Can I apply for an extension of my Non-Immigrant O visa on the expiration date or do I need to do it earlier?

You can apply for a 1-year extension until the day on your extension stamp, 14 Jan 2024 in your case. The best is for you to go to your local immigration office on or before 12 Jan as you said. There is no grace period after that date, but they let you apply for a 1-year extension by paying an overstay fine if you are late a few days at many offices. But you better check in advance.

If you travel abroad around the new year, at the very LEAST you have to be stamped in on your current extension on or before 14 Jan with a re-entry permit. If you enter Thailand after 14 Jan, you will be stamped in on visa-exempt or whatever you are eligible but an extension being spouse to a Thai, even if you have a re-entry permit (it would be expired anyway), you have to do the whole process from scratch over again. In this case, the overstay trick cannot work.

As Chiang Mai office allows you to apply for an extension max 45 days in advance from your due date. So you may be able to apply in early Dec 2023, if you are in Thailand.