Can I save my DTV visa application mid-way and continue later?

You can save and go back and forth unlimitedly BEFORE you submit and pay.

you can save it in between and continue later if needed.

and do you live IN Japan long term? because it's mostly only japanese nationals that are using the consulates there due to the requirements they have for getting visas thru them

You select your nationality and then select the consulate you will use in Japan, and then you start uploading documentation and entering the data, you can stop and save it at any time in your profile and go back to it, edit anything but your nationality and the consulate you're applying at

You can save and edit the application UNTIL you submit it to the consulate, then it is locked and if there's a mistake, you are stuck