So after reading multiple guides and being told by the staff at my building. I went to the police station to report a lost departure card prior to going to immigration. And now they said no need to worry if you have a stamp in your passport... Sigh.
TLDR : Answer Summary
The original poster was advised by their apartment staff to report a lost departure card at the police station before visiting immigration. However, they were informed at the police station that having a stamp in the passport is sufficient, thus alleviating their initial worry.