Compiling 32-bit app on x86_64 gcc increases app size drastically compared to x32 gcc

Question

I have a small C program that was is intended to be 32-bit app for compatibility. Later I changed my gcc to x86_64 from x32. And now my program compiles in 64-bit app, which I d