Importing tensorflow when embedding python in c++ returns null

问题

I've a question regarding embedding python into a C++ application. The setup is as follows: I have a large C++ application which generates some data (renders images in real-time) and displays them. I have also trained a neural network in python using tensorflow which will accept these images.

My idea was to embed python and send data as a numpy array, predict using the neural network and get back another processed numpy array to display (in C++). I've made some basic tests without tensorflow on python side to get a feel for embedding python in c and such and it seems to work.

However, once i place "import tensorflow" into any python script that I want to import, I'll get a NULL from PyImport_ImportModule in c++ part.

e.g.

import numpy as np
def foo(img):
    return np.clip(img * 2.0, 0, 255).astype(np.uint8)

works fine. But the following doesn't:

import numpy as np
import tensorflow as tf #this causes the fail

def foo(img):
    return np.clip(img * 2.0, 0, 255).astype(np.uint8)

In the second case, I still get the message in stdout from tensorflow that it has found cuda etc, but then the module import fails.

My setup is on Windows 10 x64, Anaconda Python 3.5, tensorflow-0.12 and CUDA 8. Has anyone ran across a similar problem? Other modules that I've tested (numpy, pil, scipy) seem to load fine.

If it looks like it cannot be resolved, I'll resort to IPC of some sort between the c++ part and python.

回答1:

I've resolved the issue. I needed to set argc and argv with PySys_SetArgv. I've uncovered this using PyErr_Occurred() and PyErr_Print() right after the failed import to see the problem.

回答2:

ref: https://docs.python.org/3.5/extending/embedding.html

main.cpp

#include <Python.h>
#include <iostream>
#include <QString>
#include <QDir>
#include <cstring>

using namespace std;

int main(int argc, char *argv[])
{
    PyObject *pName, *pModule, *pDict, *pFunc;
    PyObject *pArgs, *pValue;
    int i;

    if (argc < 3) {
        fprintf(stderr,"Usage: call pythonfile funcname [args]\n");
        return 1;
    }

    Py_SetProgramName((wchar_t*)L"test");

    Py_Initialize();

    PySys_SetArgv(argc, (wchar_t**)argv);
    PyRun_SimpleString("import tensorflow as tf\n"
                       "print(tf.__version__)\n");

    PyRun_SimpleString("import cv2\n"
                       "print(cv2.__version__)\n");

    QString qs = QDir::currentPath();
    std::wstring ws = qs.toStdWString();
    PySys_SetPath(ws.data());
    pName = PyUnicode_DecodeFSDefault(argv[1]);
    /* Error checking of pName left out */

    pModule = PyImport_Import(pName);
    Py_DECREF(pName);

    if (pModule != NULL) {
        pFunc = PyObject_GetAttrString(pModule, argv[2]);
        /* pFunc is a new reference */

        if (pFunc && PyCallable_Check(pFunc)) {
            pArgs = PyTuple_New(argc - 3);
            for (i = 0; i < argc - 3; ++i) {
                pValue = PyLong_FromLong(atoi(argv[i + 3]));
                if (!pValue) {
                    Py_DECREF(pArgs);
                    Py_DECREF(pModule);
                    fprintf(stderr, "Cannot convert argument\n");
                    return 1;
                }
                /* pValue reference stolen here: */
                PyTuple_SetItem(pArgs, i, pValue);
            }
            pValue = PyObject_CallObject(pFunc, pArgs);
            Py_DECREF(pArgs);
            if (pValue != NULL) {
                printf("Result of call: %ld\n", PyLong_AsLong(pValue));
                Py_DECREF(pValue);
            }
            else {
                Py_DECREF(pFunc);
                Py_DECREF(pModule);
                PyErr_Print();
                fprintf(stderr,"Call failed\n");
                return 1;
            }
        }
        else {
            if (PyErr_Occurred())
                PyErr_Print();
            fprintf(stderr, "Cannot find function \"%s\"\n", argv[2]);
        }
        Py_XDECREF(pFunc);
        Py_DECREF(pModule);
    }
    else {
        PyErr_Print();
        fprintf(stderr, "Failed to load \"%s\"\n", argv[1]);
        return 1;
    }
    Py_Finalize();
    return 0;
}

multiply.py

import tensorflow as tf
import cv2

def multiply(a,b):
    print(tf.__version__) 
    print(cv2.__version__) 
    print("Will compute", a, "times", b)
    c = 0
    for i in range(0, a):
        c = c + b
    return c

来源：https://stackoverflow.com/questions/42183255/importing-tensorflow-when-embedding-python-in-c-returns-null

标签

python

c++

tensorflow

embedding