I am doing a project to modify the Linux kernel. I have a desktop Linux machine and I have no problem building kernel on it.
However, I am going on a trip and I want to
This is a common question (well, usually it's building on Windows, but the answer is the same).
Just don't do it. You're going to have so much trouble getting anything to build correctly, it's just not worth it. Use a virtual machine, as you said yourself. It's a little slower, but not all that much, and at least building will work correctly.
Kernel development is hard enough without introducing additional, unnecessary problems.