I recently had to solve the same problem and have written up a blog post with my solution. For ease of reference I am re-posting the TLDR; version here.
Paste this code into a cell in Colab and run it to install all of the dependencies.
%%bash
# install required system dependencies
apt-get install -y xvfb x11-utils
# install required python dependencies (might need to install additional gym extras depending)
pip install gym[box2d]==0.17.* pyvirtualdisplay==0.2.* PyOpenGL==3.1.* PyOpenGL-accelerate==3.1.*
And then start a virtual display in the background.
import pyvirtualdisplay
_display = pyvirtualdisplay.Display(visible=False, # use False with Xvfb
size=(1400, 900))
_ = _display.start()
In the blog post I also provide a sample simulation demo that demonstrates that the above actually works.