How to do interactive 2D scatter plot zoom / point selection in Vaex?

Question

I saw that it is possible to do it during the demo: https://youtu.be/2Tt0i823-ec?t=769

There, the presenter has a huge dataset, and can quickly zoom in by selecting a rectangle with the mouse.

I also saw the "Interactive Widgets" section of the tutorial: https://docs.vaex.io/en/latest/tutorial.html#Interactive-widgets

However, I was not able to easily replicate that setup. What are the minimal steps to achieve it?

On Ubuntu 19.04 vaex 2.0.2, I have tried:

python3 -m pip install --user vaex scipy pandas vaex-jupyter
jupyter nbextension enable --py widgetsnbextension
jupyter nbextension enable --py bqplot
jupyter nbextension enable --py ipyvolume
jupyter nbextension enable --py ipympl
jupyter nbextension enable --py ipyleaflet
jupyter nbextension enable --py ipyvuetify
jupyter notebook

Then I created a notebook and paste in the notebook:

import vaex
import vaex.jupyter
import numpy as np
import pylab as plt
%matplotlib inline
df = vaex.example()
df.plot_widget(df.x, df.y, f='log1p', backend='bqplot')

but all I get is no graph and the message:

Plot2dDefault(w=None, what='count(*)', x='x', y='y', z=None)

If instead I do:

df.plot(df.x, df.y, f='log1p')

then I do get an plot, but it is just a non-interactive image.

I also tried to git clone the notebook that is the source of the read the docs page: https://github.com/vaexio/vaex/blob/0247f0673c5c0473001b0b66adcbc716560536aa/docs/source/tutorial.ipynb but the result was the same.

My motivation is to find a plotting program that can handle large number of points as mentioned at: Large plot: ~20 million samples, gigabytes of data

Answer 1

Use virtualenv

Not sure why, but this fixed it. I think it is because the Jupyter executable was on Python 2 and could not find Python 3 extensions.

virtualenv --python=python3 .venv
. .venv/bin/activate
python3 -m pip install vaex scipy pandas vaex-jupyter
jupyter nbextension enable --py widgetsnbextension
jupyter nbextension enable --py bqplot
jupyter nbextension enable --py ipyvolume
jupyter nbextension enable --py ipympl
jupyter nbextension enable --py ipyleaflet
jupyter nbextension enable --py ipyvuetify
jupyter notebook

and inside a new notebook:

import vaex
df = vaex.example()
df.plot_widget(df.x, df.y, f='log1p', backend='bqplot')

and now I see the interactive widget with zoom!

Versions:

pandas==0.25.0
scipy==1.3.0
vaex==2.0.2
vaex-jupyter==0.3.0

Answer 2

Jupyter widgets exist of front-end code and kernel code, the kernel code (Python in this case) is usually not the problem, if you can import for instance the ipywidgets module, you should be fine.

The biggest problem is making sure the front-end (classical notebook or Jupyter lab) has the matching javascript libraries loaded. Assuming you are working from the classical notebook, the way to debug is as following:

Since the javascript code of ipywidgets libraries are added to the frontend using the nbextension mechanism, you can check if all libraries are enabled and validated using

$ jupyter nbextension list

Which should show all green 'OK' and 'enabled' (unless you explicitly disabled it).

If not properly installed, you may want to do an installation of the front-end code, e.g.:

$ jupyter nbextension install --py --symlink --sys-prefix ipywidgets
$ jupyter nbextension install --py --symlink --sys-prefix ipyvuetify

If for some reason the extension is not automatically enabled (older versions of the notebook), run:

$ jupyter nbextension enable bqplot --py --sys-prefix
$ jupyter nbextension enable ipyvuetify --py --sys-prefix

If things aren't working, you should test which library isn't working, for instance, start with ipywidgets:

import ipywidgets as widgets
widget.FloatSlider()

If this doesn't show a slider, you are in trouble. Continue with the other libraries, and see when it fails. When it fails, check the javascript console (google that for your browser) for error messages, and see if you get hints from that.

Lastly, make sure you use a modern browser, Internet Explorer won't cut it anymore, it's too old. I recommend Chrome/Chromium or Firefox.