@Simon *********
nope. Take Thailand, you can be a Thai tax resident but you pay tax on Thai source income and overseas income remitted to Thailand. If you're non-resident for tax then you still pay tax in Thailand on your Thai sourced income.
If you're a UK tax resident you pay tax on UK source income and your worldwide income. If non-residenr you pay tax on UK sourced income.
Americans are different in that they're taxed in the US on their worldwide income regardless of whether they're a resident or not. They can use double tax treaties, where they are in force, to reduce their tax in the US.
By tax exile I am assuming you are referring to self employed as opposed to an actual employee? In which case yes, I agree to an extent. It will depend upon their nationality, where their business is and also just how honest they do or don't want to be.