Can I change my COVID extension visa to a Non-ED visa without going back to my home country?

It's been done, but if your school won't support it then it's not possible. It's up to the immigration office to decide if they will allow, and the visa you had before you went on Covid extension also makes a difference, because you are essentially still on this same reason, just prolonged with covid extensions.

So first your school has to say they will support a conversion application (a huge amount of paperwork, probably close to 200 pages), and then the immigration office of the province of the school also has to allow these conversions. Without both, then it's not possible to convert.

If that's the case, then you'd need to go to a neighboring country and apply, which is the easiest option and what was always done before Covid.