I am running a python 2.7 script on a p2.xlarge AWS server through Jupyter (Ubuntu 14.04). I would like to be able to render my simulations.
Minimal working example<
I think we should just capture renders as video by using OpenAI Gym wrappers.Monitor
and then display it within the Notebook.
Example:
!apt install python-opengl
!apt install ffmpeg
!apt install xvfb
!pip3 install pyvirtualdisplay
# Virtual display
from pyvirtualdisplay import Display
virtual_display = Display(visible=0, size=(1400, 900))
virtual_display.start()
import gym
from gym import wrappers
env = gym.make("SpaceInvaders-v0")
env = wrappers.Monitor(env, "/tmp/SpaceInvaders-v0")
for episode in range(2):
observation = env.reset()
step = 0
total_reward = 0
while True:
step += 1
env.render()
action = env.action_space.sample()
observation, reward, done, info = env.step(action)
total_reward += reward
if done:
print("Episode: {0},\tSteps: {1},\tscore: {2}"
.format(episode, step, total_reward)
)
break
env.close()
import os
import io
import base64
from IPython.display import display, HTML
def ipython_show_video(path):
"""Show a video at `path` within IPython Notebook
"""
if not os.path.isfile(path):
raise NameError("Cannot access: {}".format(path))
video = io.open(path, 'r+b').read()
encoded = base64.b64encode(video)
display(HTML(
data="""
<video alt="test" controls>
<source src="data:video/mp4;base64,{0}" type="video/mp4" />
</video>
""".format(encoded.decode('ascii'))
))
ipython_show_video("/tmp/SpaceInvaders-v0/openaigym.video.4.10822.video000000.mp4")
I hope it helps. ;)
I was looking for a solution that works in Colaboratory and ended up with this
from IPython import display
import numpy as np
import time
import gym
env = gym.make('SpaceInvaders-v0')
env.reset()
import PIL.Image
import io
def showarray(a, fmt='png'):
a = np.uint8(a)
f = io.BytesIO()
ima = PIL.Image.fromarray(a).save(f, fmt)
return f.getvalue()
imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')
while True:
time.sleep(0.01)
env.step(env.action_space.sample()) # take a random action
display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')
EDIT 1:
You could use xvfbwrapper for the Cartpole environment.
from IPython import display
from xvfbwrapper import Xvfb
import numpy as np
import time
import pyglet
import gym
import PIL.Image
import io
vdisplay = Xvfb(width=1280, height=740)
vdisplay.start()
env = gym.make('CartPole-v0')
env.reset()
def showarray(a, fmt='png'):
a = np.uint8(a)
f = io.BytesIO()
ima = PIL.Image.fromarray(a).save(f, fmt)
return f.getvalue()
imagehandle = display.display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')
for _ in range(1000):
time.sleep(0.01)
observation, reward, done, info = env.step(env.action_space.sample()) # take a random action
display.update_display(display.Image(data=showarray(env.render(mode='rgb_array')), width=450), display_id='gymscr')
vdisplay.stop()
If you're working with standard Jupyter, there's a better solution though. You can use the CommManager to send messages with updated Data URLs to your HTML output.
IPython Inline Screen Example
In Colab the CommManager is not available. The more restrictive output module has a method called eval_js() which seems to be kind of slow.
I ran into this myself.
Using xvfb as X-server somehow clashes with the Nvidia drivers.
But finally this post pointed me into the right direction.
Xvfb works without any problems if you install the Nvidia driver with the -no-opengl-files
option and CUDA with --no-opengl-libs
option.
If you know this, it should work. But as it took me quite some time till I figured this out and it seems like I'm not the only one running into problems with xvfb and the nvidia drivers.
I wrote down all necessary steps to set everything up on an AWS EC2 instance with Ubuntu 16.04 LTS here.