C++ binary file I/O to/from containers (other than char *) using STL algorithms

Question

I\'m attempting a simple test of binary file I/O using the STL copy algorithm to copy data to/from containers and a binary file. See below:

 1 #include 

        

Answer 1

To write binary data using std::copy().
I would do this:

template<typename T>
struct oi_t: public iterator<output_iterator_tag, void, void, void, void>
{
  oi_t(std::ostream& str)
    :m_str(str)
  {}
  oi_t& operator++()   {return *this;}  // increment does not do anything.
  oi_t& operator++(int){return *this;}
  oi_t& operator*()    {return *this;}  // Dereference returns a reference to this
                                       // So that when the assignment is done we
                                       // actually write the data from this class
  oi_t& operator=(T const& data)
  {
    // Write the data in a binary format
    m_str.write(reinterpret_cast<char const*>(&data),sizeof(T));
    return *this;
  }

  private:
    std::ostream&   m_str;
};

Thus the call to std::copy is:

copy (vd.begin(), vd.end(), oi_t<double>(output));

The input iterator is slightly more complicated as we have to test for the end of the stream.

template<typename T>
struct ii_t: public iterator<input_iterator_tag, void, void, void, void>
{
  ii_t(std::istream& str)
    :m_str(&str)
  {}
  ii_t()
    :m_str(NULL)
  {}
  ii_t& operator++()   {return *this;}  // increment does nothing.
  ii_t& operator++(int){return *this;}
  T& operator*()
  {
    // On the de-reference we actuall read the data into a local //// static ////
    // Thus we can return a reference
    static T result;
    m_str->read(reinterpret_cast<char*>(&result),sizeof(T));
    return result;
  }
  // If either iterator has a NULL pointer then it is the end() of stream iterator.
  // Input iterators are only equal if they have read past the end of stream.
  bool operator!=(ii_t const& rhs)
  {
      bool lhsPastEnd  = (m_str == NULL)     || (!m_str->good());
      bool rhsPastEnd  = (rhs.m_str == NULL) || (!rhs.m_str->good());

      return !(lhsPastEnd && rhsPastEnd);
  } 

  private:
    std::istream*   m_str;
};

The call to read the input is now:

ii_t<double> ii(input);
copy (ii, ii_t<double>(), back_inserter(vi));

Answer 2

You could set the precision using setprecision as Tristram pointed out, and do you need a delimiter. See the cppreference to see how the operator= functions. There is no format set, so you will need to set it on output:

ofstream output ("temp.bin", ios::binary);
output.flags(ios_base::fixed);  //or output << fixed;
copy(vd.begin(), vd.end(), oi_t(output, " "));
output.close();

I would tend to favor using fixed to eliminate precision problems. There have been many cases were someone thought "we'll never need more than 5 digits" so they hardcoded a precision everywhere. Those are costly bugs to have to correct.

Answer 3

For the question 1) You need to specify a separator (for example a space). The non-decimal part was stuck to the decimal part of the previous number. Casting and using NULL is generally wrong in C++. Should have been a hint ;)

copy (vd.begin(), vd.end(), oi_t(output, " ")); 

For the question 2)

#include <iomanip>
output << setprecision(9);

Answer 4

I have come up with a better design for binary I/O. The fundamental approach is to have three methods: size_on_stream, load_from_buffer, and store_to_buffer. These go into an interface class so that all classes that support binary I/O inherit it.

The size_on_stream method returns the size of the data as transmitted on the stream. Generally, this does not include padding bytes. This should be recursive such that a class calls the method on all of its members.

The load_from_buffer method is passed a reference to a pointer to a buffer ( unsigned char * & ). The method loads the object's data members from the buffer, incrementing the pointer after every member (or incrementing once after all the members).

The store_to_buffer method stores data into the given buffer and increments the pointer.

The client calls size_on_stream to determine the size of all the data. A buffer of this size is dynamically allocated. Another pointer to this buffer is passed to the store_to_buffer to store the object's members into the buffer. Finally, the client uses a binary write (fwrite or std::ostream::write) to transfer the buffer to the stream.

Some of the benefits of this technique are: packing, abstraction and block I/O. The objects pack their members into the buffer. The process for writing into the buffer is hidden from the client. The client can use block I/O functions which are always more efficient than transferring individual members.

This design is also more portable, as the objects can take care of the Endianess. There is a simple method for this, which is left up to the reader.

I have expanded this concept to incorporate POD (Plain Old Data) types as well, which is left as an exercise for the reader.