It will work, you are a UK citizen, you have just to be present in the UK at time of application, that is all. You state you are in the UK for the DTV application, staying at xyz.. blah blah.. Other non uk residents have returned to UK and done it.
Where ever you stay, you can produce a letter of confirmation. It would be shocking if you could not apply in the UK while other foreign residents can..
You can, as you are a UK born citizen. Im also a UK non resident and I contacted London Embassy about applying there and they said its ok, you have to show you are in the UK at time of application, flights in/out of the UK. Also Tokyo Embassy told me to go there and apply when they were not allowing foreign residents to apply.
Did you check the gym used by the agency was govt approved and submitted all course docs, 6 plus months course? Some agents use non regulated gyms. Malaysia is the toughest embassy to get DTV approval. Much easier doing it yourself in Vietnam, Laos Indonesia.