How can I write unit tests against code that uses matplotlib?

Question

I'm working on a python (2.7) program that produce a lot of different matplotlib figure (the data are not random). I'm willing to implement some test (using unittest) to be sure that the generated figures are correct. For instance, I store the expected figure (data or image) in some place, I run my function and compare the result with the reference. Is there a way to do this ?

Answer 1

In my experience, image comparison tests end up bring more trouble than they are worth. This is especially the case if you want to run continuous integration across multiple systems (like TravisCI) that may have slightly different fonts or available drawing backends. It can be a lot of work to keep the tests passing even when the functions work perfectly correctly. Furthermore, testing this way requires keeping images in your git repository, which can quickly lead to repository bloat if you're changing the code often.

A better approach in my opinion is to (1) assume matplotlib is going to actually draw the figure correctly, and (2) run numerical tests against the data returned by the plotting functions. (You can also always find this data inside the Axes object if you know where to look.)

For example, say you want to test a simple function like this:

import numpy as np
import matplotlib.pyplot as plt
def plot_square(x, y):
    y_squared = np.square(y)
    return plt.plot(x, y_squared)

Your unit test might then look like

def test_plot_square1():
    x, y = [0, 1, 2], [0, 1, 2]
    line, = plot_square(x, y)
    x_plot, y_plot = line.get_xydata().T
    np.testing.assert_array_equal(y_plot, np.square(y))

Or, equivalently,

def test_plot_square2():
    f, ax = plt.subplots()
    x, y = [0, 1, 2], [0, 1, 2]
    plot_square(x, y)
    x_plot, y_plot = ax.lines[0].get_xydata().T
    np.testing.assert_array_equal(y_plot, np.square(y))

Answer 2

Matplotlib has a testing infrastructure. For example:

import numpy as np
import matplotlib
from matplotlib.testing.decorators import image_comparison
import matplotlib.pyplot as plt

@image_comparison(baseline_images=['spines_axes_positions'])
def test_spines_axes_positions():
    # SF bug 2852168
    fig = plt.figure()
    x = np.linspace(0,2*np.pi,100)
    y = 2*np.sin(x)
    ax = fig.add_subplot(1,1,1)
    ax.set_title('centered spines')
    ax.plot(x,y)
    ax.spines['right'].set_position(('axes',0.1))
    ax.yaxis.set_ticks_position('right')
    ax.spines['top'].set_position(('axes',0.25))
    ax.xaxis.set_ticks_position('top')
    ax.spines['left'].set_color('none')
    ax.spines['bottom'].set_color('none')

From the docs:

The first time this test is run, there will be no baseline image to compare against, so the test will fail. Copy the output images (in this case result_images/test_category/spines_axes_positions.*) to the correct subdirectory of baseline_images tree in the source directory (in this case lib/matplotlib/tests/baseline_images/test_category). When rerunning the tests, they should now pass.

Answer 3

You can also use unittest.mock to mock matplotlib.pyplot and check that appropriate calls with appropriate arguments are made to it. Let's say you have a plot_data(data) function inside module.py that you want to test and which looks like this:

import matplotlib.pyplot as plt

def plot_data(x, y, title):
        plt.figure()
        plt.title(title)
        plt.plot(x, y)
        plt.show()

In order to test this function in your test_module.py file you need to:

import numpy as np

from unittest import mock
import module as my_module


@mock.patch("%s.my_module.plt" % __name__)
def test_module(mock_plt):
    x = np.arange(0, 5, 0.1)
    y = np.sin(x)
    my_module.plot_data(x, y, "my title")

    assert mock_plt.title.call_args_list[0][0][0] == "my title"
    assert mock_plt.figure.called

This checks if a title method is called with an argument my title and that the figure method is invoked inside plot_data.