Padang Basar is in my experience easy and fast. Stamp out of Thailand jump on one of the many motorbike taxis and they take you over go through the car line at the Malaysian side so no queuing and same on way out. 50 baht well spent.
You can get a bus or van from Hat Yai to Pedang Basar border crossing then take a motorbike taxi to the Malaysian side and back it’s 50 baht for the bike taxi.