Pickle exception for cv2.Boost when using multiprocessing

Question

I'm working on project named "Faciel Actions Units Detection" I'm using python2.7 and opencv 2.4

The error:

pickle.PicklingError: Can't pickle <type 'cv2.Boost'>: it's not the same object as cv2.Boost

A partial traceback, transcribed from a screenshot:

Loading classifier for action unit 27
Traceback (most recent call last):
  File "C:\Python27\audetect-master\audetect-interactive.py", line 59, in <module>
    main()
  File "C:\Python27\audetect-master\audetect-interactive.py", line 18, in main
    active_aus = detector.detect()
  File "C:\Python27\audetect-master\detect.py", line 67, in detect
    initial_points = self.ffdetector.locate_features(first)
  File "C:\Python27\audetect-master\detect.py", line 183, in locate_features
    thread.start()
  File "C:\Python27\lib\multiprocessing\process.py", line 130, in start
    self._popen = Popen(self)
  File "C:\Python27\lib\multiprocessing\forking.py", line 227, in __init__
    dump(process_obj, to_child, HIGHEST_PROTOCOL)
  File "C:\Python27\lib\multiprocessing\forking.py", line 199, in dump
    ForkingPickler(file, protocol).dump(obj)
  File "C:\Python27\lib\pickle.py", line 224, in dump
    self.save(obj)
  File "C:\Python27\lib\pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 425, in save_reduce
    save(state)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 655, in save_dict
    self._batch_setitems(obj.iteritems())
  File "C:\Python27\lib\pickle.py", line 687, in _batch_setitems
    save(v)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\multiprocessing\forking.py", line 67, in dispatcher
    self.save_reduce(obj=obj, *rv)
  File "C:\Python27\lib\pickle.py", line 401, in save_reduce
    save(args)
  File "C:\Python27\lib\pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "C:\Python27\lib\pickle.py", line 554, in save_tuple
    save(element)

Answer 1

Pickle is used by the multiprocessing module to communicate between the different parts, and in the programming guidelines it explains that you must ensure that all your data that you pass between processes must be compatible with pickling:

Picklability: Ensure that the arguments to the methods of proxies are picklable.

You are using data that is not picklable.

Specifically, what is going wrong is that the cv2.Boost class doesn't quite tell the truth on how you can create more copies of the class. pickle stores references to classes and functions, not their definition, because that's way more efficient. This means that instances only need to store the data for that instance, not also all of the class hierarchy and method definitions.

In order to do this, pickle takes the module a class or function is defined in, and the name of the object, and together that's the reference to the class or function. It then double-checks that it can use that name to load the same class or function back again.

That sanity check failed for the cv2.Boost class. You have instances of a class is named Boost and that claims to have come from the cv2 module, but when pickle then goes to the cv2 module and looks up the Boost attribute of that module it found a different object. This means your data could not be unpickled.

There are ways to correct this; you need to teach the pickle module to use a different function to load the same data again, using the copyreg.pickle() function; if such a registration exists for the cv2.Boost class then pickle will not make the above check:

import copyreg
import cv2

def _pickle_boost(boost):
    return cv2.Boost, (
        boost.trainData,
        boost.tflag,
        boost.responses, 
        boost.varIdx,
        boost.sampleIdx,
        boost.varType,
        boost.missingDataMask,
        boost.paramsd,
    )

copyreg.pickle(cv2.Boost().__class__, _pickle_boost)

WARNING: I didn't actually test if the above will work, because I don't have a 2.4.x version of cv2 installed locally; I merely referred to the cv2.Boost() documentation to guess at what attributes such a class would have. You'll probably need to adjust it. The idea is that for the cv2.Boost().__class__ type, the _pickle_boost() function is called, returning a callable (cv2.Boost) to create a new instance, and the arguments that you want to pass to that callable.

If any of the above values are themselves more cv2 types that exhibit the same problem, then you need to register more functions.