mock.patch and multiprocessing

问题

I'm struggling to use mock.patch in a multiprocessing environment while without multiprocessing mock.patch works fine. Filename: test_mp.py

import multiprocessing
import mock

def inner():
    return sub()

def sub():
    return "abc"

def test_local():
    assert inner()=="abc"

def test_mp():    
    with multiprocessing.Pool() as pool:
        assert pool.apply(inner,args=[])=='abc'

def test_mock():    
    with mock.patch('test_mp.sub', return_value='xxx') as xx:
        assert inner()=="xxx"
        xx.assert_called_once()

def test_mp_mock():    
    with multiprocessing.Pool() as pool:
        with mock.patch('test_mp.sub', return_value='xyz') as xx:
            assert pool.apply(inner,args=[])=='xyz'
            xx.assert_called_once()

Test test_local, test_mock and test_mock finishes successfully
However test_mp_mock fails with

=================================== FAILURES ===================================
_________________________________ test_mp_mock _________________________________

    def test_mp_mock():
        with multiprocessing.Pool() as pool:
            with mock.patch('test_mp.sub', return_value='xyz') as xx:
>               assert pool.apply(inner,args=[])=='xyz'
E               AssertionError: assert 'abc' == 'xyz'
E                 - abc
E                 + xyz

projects/mfhealth/test_mp.py:25: AssertionError

Update:

Based on https://medium.com/uckey/how-mock-patch-decorator-works-in-python-37acd8b78ae and with help of sharedmock https://github.com/elritsch/python-sharedmock I was able to get further, but still not complete.

I extended test_mp.py for

from sharedmock.mock import SharedMock

def inner2(sm):
    with mock.patch('test_mp.sub', sm) as xx:
        return inner()


def test_mp_smock():
    with multiprocessing.Pool() as pool:
        sm=SharedMock()
        sm.return_value="xyz"
        with mock.patch('test_mp.sub', sm) as xx:
            assert pool.apply(inner2,args=[sm])=='xyz'
            assert xx.call_count == 1

def test_mp_mock2():
    with multiprocessing.Pool() as pool:
        sm=mock.Mock()
        sm.return_value="xyz"
        print(f"before patch {sub}, {locals()}")
        with mock.patch('test_mp.sub', sm) as xx:
            print(f"after patch {sub}")
            assert pool.apply(inner2,args=[sm])=='xyz'
            assert xx.call_count == 1

With the following results:

test_mp_smock finishes successfully.
test_mp_mock2 fails with _pickle.PicklingError: Can't pickle <class 'mock.mock.Mock'>: it's not the same object as mock.mock.Mock

The main disadvantage of test_mp_smock is that a new method inner2 must be introduced to activate patching via mock.patch. Any idea of how to propagate patching from test_mp_smock to code under test without introducing a wrapper method inner2 because I cannot override.

pool.apply(inner,args=[])

回答1:

Finally, I got it working.

SharedMock is not that flexible like Mock, eg. missing assert_called_once_with, ..., but you can set returned value or check the number of calls or its arguments.

import multiprocessing
import mock
from sharedmock.mock import SharedMock


def inner():
    return sub(x="xxx")

def sub(x=""):
    return f"abc"

def fun_under_test():
    with multiprocessing.Pool() as pool:
        assert pool.apply(inner,args=[])=='xyz'

def test_final():
    sm=SharedMock()
    sm.return_value="xyz"
    with mock.patch('test_mp.sub', sm) as xx:
        fun_under_test()
        assert xx.call_count == 1 #number of calls of sub function
        assert xx.mock_calls[0][2]['x']=="xxx" # value of parameters ie sub(x="xxx")

来源：https://stackoverflow.com/questions/65492977/mock-patch-and-multiprocessing

标签

python

unit-testing

mocking

multiprocessing