@Christopher ******
If someone is holding your passport and doesn't want to give it back for whatever, then just go to your embassy and report it lost or stolen. You get emergency documents and the one who has your passport has a document that's of no use to him.
That's why it makes more sense to keep a copy of your passport and cash as a deposit and if you don't pay up it's easy to stop you at the border when you leave the country.