Sorry that there are many similar questions, but I do find that Googling for CMake queries always yields similar-but-not-the-same scenarios, conflicting CMake commands and s
CMAKE_CXX_FLAGS only affects the C++ compiler. You probably also have to set the flag for the C compiler:
CMAKE_CXX_FLAGS
set (CMAKE_C_FLAGS "-m32")