问题
While writing my first django application I've faced the following problem with boost::python. From python code, I need to pass io.BytesIO to the C++ class which takes std::istream.
I have a legacy C++ library for reading files of certain format. Let's call is somelib. The interface of this library uses std::istream as an input. Something like this:
class SomeReader
{
public:
bool read_from_stream(std::istream&);
};
And I want to wrap it, so that I can use my lib from python in the following way:
reader = somelib.SomeReader()
print ">>Pyhton: reading from BytesIO"
buf = io.BytesIO("Hello Stack Overflow")
reader.read(buf)
I found out how to do it for actual python file object. But it is not clear how to do it for arbitrary file-like object. This is definition of the python bindings I have so far:
using namespace boost::python;
namespace io = boost::iostreams;
struct SomeReaderWrap: SomeReader, wrapper<SomeReader>
{
bool read(object &py_file)
{
if (PyFile_Check(py_file.ptr()))
{
FILE* handle = PyFile_AsFile(py_file.ptr());
io::stream_buffer<io::file_descriptor_source> fpstream (fileno(handle), io::never_close_handle);
std::istream in(&fpstream);
return this->read_from_stream(in);
}
else
{
//
// How do we implement this???
//
throw std::runtime_error("Not a file, have no idea how to read this!");
}
}
};
BOOST_PYTHON_MODULE(somelib)
{
class_<SomeReaderWrap, boost::noncopyable>("SomeReader")
.def("read", &SomeReaderWrap::read);
}
Is there a more or less generic way of converting python IO object to the C++ stream?
Thank you in advance.
As a result of my experiments I've created a small github repo which illustrates this problem.
回答1:
Instead of converting the Python io.BytesIO object, consider implementing a model of the Boost.IOStreams Source concept that is capable of reading from a Python io.BytesIO object. This would allow one to construct a boost::iostreams::stream and be useable by SomeReader::read_from_stream().
This tutorial demonstrates how to create and use a custom Boost.IOStream Source. Overall, the process should be fairly straight forward. One just needs to implement the Source concept's read() function in terms of io.BufferedIOBase.read():
/// Type that implements the Boost.IOStream's Source concept for reading
/// data from a Python object supporting read(size).
class PythonInputDevice
: public boost::iostreams::source // Use convenience class.
{
public:
explicit
PythonInputDevice(boost::python::object object)
: object_(object)
{}
std::streamsize read(char_type* buffer, std::streamsize buffer_size)
{
namespace python = boost::python;
// Read data through the Python object's API. The following is
// is equivalent to:
// data = object_.read(buffer_size)
boost::python::object py_data = object_.attr("read")(buffer_size);
std::string data = python::extract<std::string>(py_data);
// If the string is empty, then EOF has been reached.
if (data.empty())
{
return -1; // Indicate end-of-sequence, per Source concept.
}
// Otherwise, copy data into the buffer.
copy(data.begin(), data.end(), buffer);
return data.size();
}
private:
boost::python::object object_;
};
then create a boost::iostreams::stream using the Source device:
boost::iostreams::stream<PythonInputDevice> input(py_object);
SomeReader reader;
reader.read_from_stream(input);
As PythonInputDevice is implemented in terms of object.read(), duck typing allows PythonInputDevice to be used with any Python object supporting a read() method that has the same pre and post-conditions. This include's the built-in Python file object, making it no longer necessary to have conditional branching based on type within SomeReaderWrap::read().
Here is a complete minimal example based on the original code:
#include <algorithm> // std::copy
#include <iosfwd> // std::streamsize
#include <iostream>
#include <boost/python.hpp>
#include <boost/iostreams/concepts.hpp> // boost::iostreams::source
#include <boost/iostreams/stream.hpp>
class SomeReader
{
public:
bool read_from_stream(std::istream& input)
{
std::string content(std::istreambuf_iterator<char>(input.rdbuf()),
(std::istreambuf_iterator<char>()));
std::cout << "SomeReader::read_from_stream(): " << content << std::endl;
return true;
}
};
/// Type that implements a model of the Boost.IOStream's Source concept
/// for reading data from a Python object supporting:
/// data = object.read(size).
class PythonInputDevice
: public boost::iostreams::source // Use convenience class.
{
public:
explicit
PythonInputDevice(boost::python::object object)
: object_(object)
{}
std::streamsize read(char_type* buffer, std::streamsize buffer_size)
{
namespace python = boost::python;
// Read data through the Python object's API. The following is
// is equivalent to:
// data = object_.read(buffer_size)
boost::python::object py_data = object_.attr("read")(buffer_size);
std::string data = python::extract<std::string>(py_data);
// If the string is empty, then EOF has been reached.
if (data.empty())
{
return -1; // Indicate end-of-sequence, per Source concept.
}
// Otherwise, copy data into the buffer.
copy(data.begin(), data.end(), buffer);
return data.size();
}
private:
boost::python::object object_;
};
struct SomeReaderWrap
: SomeReader,
boost::python::wrapper<SomeReader>
{
bool read(boost::python::object& object)
{
boost::iostreams::stream<PythonInputDevice> input(object);
return this->read_from_stream(input);
}
};
BOOST_PYTHON_MODULE(example)
{
namespace python = boost::python;
python::class_<SomeReaderWrap, boost::noncopyable>("SomeReader")
.def("read", &SomeReaderWrap::read)
;
}
Interactive usage:
$ echo -n "test file" > test_file
$ python
>>> import example
>>> with open('test_file') as f:
... reader = example.SomeReader()
... reader.read(f)
...
SomeReader::read_from_stream(): test file
True
>>> import io
>>> with io.BytesIO("Hello Stack Overflow") as f:
... reaader = example.SomeReader()
... reader.read(f)
...
SomeReader::read_from_stream(): Hello Stack Overflow
True
来源：https://stackoverflow.com/questions/24225442/converting-python-io-object-to-stdistream-when-using-boostpython