How to run OpenAI Gym .render() over a server

Question

I am running a python 2.7 script on a p2.xlarge AWS server through Jupyter (Ubuntu 14.04). I would like to be able to render my simulations.

Minimal working example<

Answer 1

Got a simple solution working:

If on a linux server, open jupyter with

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

In Jupyter

import matplotlib.pyplot as plt
%matplotlib inline
from IPython import display

After each step

def show_state(env, step=0, info=""):
    plt.figure(3)
    plt.clf()
    plt.imshow(env.render(mode='rgb_array'))
    plt.title("%s | Step: %d %s" % (env._spec.id,step, info))
    plt.axis('off')

    display.clear_output(wait=True)
    display.display(plt.gcf())

Note: if your environment is not unwrapped, pass env.env to show_state.

Answer 2

I had the same problem and I_like_foxes solution to reinstall nvidia drivers with no opengl fixed things. Here are the commands I used for Ubuntu 16.04 and GTX 1080ti https://gist.github.com/8enmann/931ec2a9dc45fde871d2139a7d1f2d78

Answer 3

I created this mini-package which allows you to render your environment onto a browser by just adding one line to your code.

Put your code in a function and replace your normal env.render() with yield env.render(mode='rgb_array'). Encapsulate this function with the render_browser decorator.

import gym
from render_browser import render_browser

@render_browser
def test_policy(policy):
    # Your function/code here.
    env = gym.make('Breakout-v0')
    obs = env.reset()

    while True:
        yield env.render(mode='rgb_array')
        # ... run policy ...
        obs, rew, _, _ = env.step(action)

test_policy(policy)    

When you visit your_ip:5000 on your browser, test_policy() will be called and you'll be able to see the rendered environment on your browser window.

Answer 4

This GitHub issue gave an answer that worked great for me. It's nice because it doesn't require any additional dependencies (I assume you already have matplotlib) or configuration of the server.

Just run, e.g.:

import gym
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0') # insert your favorite environment
render = lambda : plt.imshow(env.render(mode='rgb_array'))
env.reset()
render()

Using mode='rgb_array' gives you back a numpy.ndarray with the RGB values for each position, and matplotlib's imshow (or other methods) displays these nicely.

Note that if you're rendering multiple times in the same cell, this solution will plot a separate image each time. This is probably not what you want. I'll try to update this if I figure out a good workaround for that.

Update to render multiple times in one cell

Based on this StackOverflow answer, here's a working snippet (note that there may be more efficient ways to do this with an interactive plot; this way seems a little laggy on my machine):

import gym
from IPython import display
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
for _ in range(100):
    plt.imshow(env.render(mode='rgb_array'))
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Update to increase efficiency

On my machine, this was about 3x faster. The difference is that instead of calling imshow each time we render, we just change the RGB data on the original plot.

import gym
from IPython import display
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

env = gym.make('Breakout-v0')
env.reset()
img = plt.imshow(env.render(mode='rgb_array')) # only call this once
for _ in range(100):
    img.set_data(env.render(mode='rgb_array')) # just update the data
    display.display(plt.gcf())
    display.clear_output(wait=True)
    action = env.action_space.sample()
    env.step(action)

Answer 5

In my IPython environment, Andrew Schreiber's solution can't plot image smoothly. The following is my solution:

If on a linux server, open jupyter with

$ xvfb-run -s "-screen 0 1400x900x24" jupyter notebook

In Jupyter

import matplotlib.pyplot as plt
%matplotlib inline
%matplotlib notebook
from IPython import display

Display iteration:

done = False
obs = env.reset()

fig = plt.figure()
ax = fig.add_subplot(111)
plt.ion()

fig.show()
fig.canvas.draw()

while not done:
    # action = pi.act(True, obs)[0] # pi means a policy which produces an action, if you have
    # obs, reward, done, info = env.step(action) # do action, if you have
    env_rnd = env.render(mode='rgb_array')
    ax.clear()
    ax.imshow(env_rnd)
    fig.canvas.draw()
    time.sleep(0.01)

Answer 6

I managed to run and render openai/gym (even with mujoco) remotely on a headless server.

# Install and configure X window with virtual screen
sudo apt-get install xserver-xorg libglu1-mesa-dev freeglut3-dev mesa-common-dev libxmu-dev libxi-dev
# Configure the nvidia-x
sudo nvidia-xconfig -a --use-display-device=None --virtual=1280x1024
# Run the virtual screen in the background (:0)
sudo /usr/bin/X :0 &
# We only need to setup the virtual screen once

# Run the program with vitural screen
DISPLAY=:0 <program>

# If you dont want to type `DISPLAY=:0` everytime
export DISPLAY=:0

Usage:

DISPLAY=:0 ipython2

Example:

import gym
env = gym.make('Ant-v1')
arr = env.render(mode='rgb_array')
print(arr.shape)
# plot or save wherever you want
# plt.imshow(arr) or scipy.misc.imsave('sample.png', arr)