What visa options do European passport holders have for staying in Thailand?

You could look into the DTV.

There isn't a European passport..

First of all there's no such thing as a European citizen, you have a passport from a single country and that's how Thailand sees you.

Second on all, that doesn't even matter. Thailand doesn't care where you are from. They are looking at you as abusing tourist entries and they know you're trying to live in Thailand without a long term visa, which you cannot do. You need to find an appropriate long term visa if you want to live in Thailand. Just because you want to, doesn't mean you can. If you don't qualify for a visa, then you cannot live in Thailand.

Third, you did not make a visa run, which is where you go to a Thai embassy and apply for a pay for a visa. You did a border bounce because you likely wouldn't have been questioned if you did a visa run and returned with a visa.