Why does the Python/C API crash on PyRun_SimpleFile?

Question

I've been experimenting with embedding different scripting languages in a C++ application, currently I'm trying Stackless Python 3.1. I've tried several tutorials and examples, what few I can find, to try and run a simple script from an application.

Py_Initialize();

FILE* PythonScriptFile = fopen("Python Scripts/Test.py", "r");
if(PythonScriptFile)
{
    PyRun_SimpleFile(PythonScriptFile, "Python Scripts/Test.py");
    fclose(PythonScriptFile);
}

Py_Finalize();

For some odd reason, running this piece of code results in an access violation at:

    PyRun_SimpleFile(PythonScriptFile, "Python Scripts/Test.py");

I've searched online for others with a similar problem and found only one. Their only solution was a workaround that only seems possible in an older version of Python: Creating a python file object and returning the FILE* from that python file object into PyRun_SimpleFile. Such function calls are not available however, the Python 3.1 API creates file objects from a file descriptor and returns file descriptors, but the PyRun_SimpleFile function still requires a FILE*.

I'm at a loss as to how to run any scripts from file, short of loading the entire file into memory manually and running it as a giant string, certainly not a practical solution.

What gives? How can I accomplish this task if the API has an internal error?

Update: I've managed to build Stackless Python 3.1 from the source and yet the crash remains completely unchanged, despite using the same C runtime library. Both my project and the Stackless Python 3.1 source are built with Visual Studio 2010's C++ compiler and C runtime. I no longer have any inkling as to what might solve this problem, short of modifying Python to use a file name and not a FILE*. Another terrible workaround.

Answer 1

I was getting a similar crash & did the below:

   PyObject* PyFileObject = PyFile_FromString("test.py", "r");
   PyRun_SimpleFileEx(PyFile_AsFile(PyFileObject), "test.py", 1);

Note that this was in python 2.7 though. I don't know if the API has changed in 3.x.

Answer 2

This works for me on Python 3:

 PyObject *obj = Py_BuildValue("s", "test.py");
 FILE *file = _Py_fopen_obj(obj, "r+");
 if(file != NULL) {
     PyRun_SimpleFile(file, "test.py");
 }

I hope It would be useful.

Answer 3

Your code works correctly on my installed version of Python 2.6. I also built stackless 3.1.2 from source and it worked correctly. This was with g++ 4.4.3 on Ubuntu 10.04. If you're on windows, you might want to check that both stackless and your code are built against the same C runtime.

Answer 4

This sounds like a problem of mismatched APIs. If your code and the Python runtime were compiled with different compilers, or even different compiler options, then accessing the FILE* could result in an access violation. Can you double-check that you've build your C code properly?

You mention that you're embedding Python into your C++ application. Keep in mind that Python is C code, compiled as C code. Perhaps that is the source of the problem?

Answer 5

If you built your test with VC 2010, you will definitely have problems - VC9 (VS 2008) and VC10 (VS 2010) have mutually incompatible support DLLs that are usually required (implement printf, file i/o and that sort of thing). You cannot mix them if they include the standard libraries, which the python build does.

You always have the option of using gcc (e.g. Cygwin or mingw) or downloading Visual Studio 2008 express, which should work fine for experimentation into python embedding. I have used both with the standard Python 2.7.6 build.

Answer 6

And what about this solution:

Py_SetProgramName(argv[0]);
Py_Initialize();
PyRun_SimpleString("execfile(\"ex30.py\")");
Py_Finalize();

Where ex30.py it the name of the python script I am running.

Answer 7

The below code will execute the test.py module. Python will search the module in the path set. So the path should be handled first.

Py_Initialize();

string path = "Python Scripts/";

//Set the path
PyRun_SimpleString("import sys");
string str = "sys.path.append('" + path + "')";
PyRun_SimpleString(str.c_str());

//Dont use test.py as it actually searches sub module test>>py
PyObject * moduleName = PyUnicode_FromString("test");
PyObject * pluginModule = PyImport_Import(moduleName);

if (pluginModule == nullptr)
{
    PyErr_Print();
    return "";
}

//Do the executions here

//clean up
Py_DECREF(moduleName);
Py_DECREF(pluginModule);
Py_DECREF(transformFunc);
Py_DECREF(result);

Py_Finalize();