as in, if they deny the application, they just hand you back the papers and passport without doing anything. If they take the application and then not approve it, you will get a new stamp in the passport.
it means if they deny your application it won't be processed at all. If they process it and don't approve it, you will get a 7 day stamp to leave the country.
just remember she has to leave on the same passport she entered on, and that you can only switch the passports by air travel. So leave on the US passport. Enter whatever country you are going to, by air, with the passport that makes sense. Re-enter Thailand, by air, with the Thai passport.
one thing to also keep in mind is the timing. You didn't mention her time of flight. If her flight is by any chance in the early am of the 18th, she could be passing passport control before midnight on the 17th, and there will be no overstay at all, as she would officially have left Thailand at the time she went through passport control.
on the other hand, as I understand, not all thai embassies sells those 90 day non o, but only the year long oa, which has extra requirements of insurance for example. And I guess that's why many chose to just come on a tourist visa or 30 day visa exemption, and then buying the 90 day non o inside Thailand. For that though, you WILL need to show the 800K came from abroad with international transfer showing up in your thai account.
you can have the money in your account in your home country when applying for the initial non o visa there, before you go to Thailand. You can then, however you want/can, get those 800k to a thai account in your name while in Thailand if you wish to apply for a year extension. No need to prove where it came from then, but it does have to be in the account for minimum 2 full months before you can apply, and then 3 months after too, and not below 400k the rest of the year, if your intention is to keep applying for yearly extensions.
You can, if you buy a non o before coming here. If you come on tourist visa, or visa exempt, and want to buy the initial 90 day non o inside Thailand, you will have to prove an 800k international transfer to an account in your name only. Customs declaration and anything else wont work. Only an international transfer showing up in your bankbook/bank account. If you already have the non o when entering Thailand, you can just deposit the funds at a bank, in an account in your name only, and have them there for at least 2 months for seasoning before applying for the year extension.