Can my husband and I renew our non immigrant OA multi-entry visas from the UK or do we need to apply for new ones?

The great benefit of these visas is every time you enter Thailand you get a 1-year visa up until the expiry date.

If you leave the country the week before the visa expires, and come back before the expiry date, you will get another one year.

To keep that expiry date in 2024, before you leave get a reentry permit.

No need to get a new visa or insurance until August 2024 when you go back to England and get another OA visa.

You cannot apply for an extension unless you're in Thailand. You said your visa expires in August but you'll be in UK until October so you're going to need to apply for a new visa if you want one.