Do I have to leave Thailand within 24 hours after my Non-B visa and work permit are canceled?

@Bob *********

... since always!

@Alex ******

, Usually, your employer gives you a termination letter. You bring it to the labour office that issued your work permit to cancel WP, then bring the WP cancellation receipt (and letter if you get it back) to the immigration office to cancel the extension. In many provinces, you can do it a few days or weeks in advance, but some provinces accept the WP cancellation only on the last day of your work. The order of applications might be different in some provinces.

You need to talk with your employer for a detailed procedure. Some employers accompany you for part or whole of applications. Some employers cancel your WP first, and then give you a cancellation receipt. A work permit can be cancelled by the employer, but an extension cannot be cancelled without you and your original passport.