Yes you can do that. But just incase if you would like to have a work Visa again I'm sure that you will need a copy of the work permit cancellation docs or letter from the company to cancel your Visa.
you need to leave country and get a tourist Visa if you want to stay in Thailand. AND if you would like to work again in Thailand need to go to immigration to get a cancellation stamp for your current Visa too.