Which path should I pass to the --prefix
option when doing cross compiling: should I specify the path on my build machine or the path on the target platform?
Yes you are right, --prefix is the path for working environment. Just use --prefix=/usr. You can check in which folder path make install command will install your binary by installing in DESTDIR. For example if you use --prefix=/usr and make install DESTDIR=/home/me/arm/build/target_fs, then the binaries will be installed in the folder /home/me/arm/build/target_fs/usr. And If you just run make install, then the binary will be installed in your prefix i.e. in "/usr".
As you are cross-compiling, I think it doesn't matter which prefix you use, because anyways you will be installing in DESTDIR and then copying the binary files manually to your target.