While it's frustrating for the renter, I understand the landlord. He's renting out and doesn't want to hear about this room again for the duration of the contract.
Your travels are hassle to him, as he cannot forbid them, all he can do is demand compensation for his inconvenience.
I know I know, you're all gonna slug me, but even landlords are humans with busy lives sometimes