Yes, if the Extension of Stay expires while you are abroad, you have to start from the beginning with a new Non Immigrant O visa based on marriage and later for a new Extension of Stay.
There is no Covid Visa, there is an Extension of Stay based on being unable to travel due to Covid. Just to mention it.
If you entered Thailand using a Tourist Visa and are stil in the first 60 days of your Admitted to Stay period, you must first apply for the usual 30 day Extension of Stay based on Tourism.
I guess you have an Extension of Stay based on being married with a Thai National.
You have to visit the Immigration Office to transfer the stamps into your new passport, you need the letter from your Embassy if you applied for the new passport inside Thailand. As you are in the Immigration Office to transfer the stamp, you can ask if the update of the address registration is needed.
The 30 Day Visa Exempt Entry can be extended ONCE with another 30 day based on tourism. You can do this on the 3rd floor of IT square Laksi Plaza Cheangwattana road in Bangkok.