Immigration can give you a replacement Re-entry permit in your new passport but I doubt it can be done at the airport. You'll need to use the old passport with the visa in it anyway, so the Re-entry stamp in the old passport is still valid.
The baby only needs an Extension if they entered Thailand on a foreign passport. If they were born here they are fine until their parents decide to leave and get them a foreign passport.
Where was your baby born? If in Thailand, he/she is fine and legal, as they will have a Thai birth certificate. When they leave a note is made in their (foreign) passport.