It pretty much as you have described, no need for an invasion some countries just let you in. I know this concept is hard to understand but it’s true! 😂
To answer your question he can leave on a UK passport, he will not need a Thai one. Once it’s explained to the immigration staff that he’s just been born and therefore there is no entry stamp in the passport they will make a hand written note to this effect. That’s what happened with my Son.