Is there a C++ iterator that can iterate over a file line by line?

Question

I would like to get an istream_iterator-style iterator that returns each line of the file as a string rather than each word. Is this possible?

Answer 1

The standard library does not provide iterators to do this (although you can implement something like that on your own), but you can simply use the getline function (not the istream method) to read a whole line from an input stream to a C++ string.

Example:

#include <iostream>
#include <fstream>
#include <string>
#include <algorithm>

using namespace std;

int main()
{
    ifstream is("test.txt");
    string str;
    while(getline(is, str))
    {
        cout<<str<<endl;
    }
    return 0;
}

Answer 2

You could write your own iterator. It's not that hard. An iterator is just a class on which (simply speaking) the increment and * operators are defined.

Look at http://www.drdobbs.com/cpp/184401417 to get started writing your own iterators.

Answer 3

You can use istreambuf_iterator instead of istream_iterator. It doesn't ignore control characters like istream_iterator.

code.cpp:

#include <iterator>
#include <iostream>
#include <fstream>

using namespace std;

int main()
{
    ifstream file("input.txt");

    istreambuf_iterator<char> i_file(file);

    istreambuf_iterator<char> eof;

    std::string buffer;
    while(i_file != eof)
    {
        buffer += *i_file;
        if(*i_file == '\n')
        {
            std::cout << buffer;
            buffer.clear();
        }
        ++i_file;
    }

    return 0;
}

input.txt:

ahhhh test *<-- There is a line feed here*
bhhhh second test *<-- There is a line feed here*

output:

ahhhh test
bhhhh second test

Answer 4

Here is a pretty clean approach that uses boost::tokenizer. This returns an object providing begin() and end() member functions; for a complete interface, see the documentation of the tokenizer class.

#include <boost/tokenizer.hpp>
#include <iostream>
#include <iterator> 


using istream_tokenizer = boost::tokenizer<boost::char_separator<char>,
                                           std::istreambuf_iterator<char>>;

istream_tokenizer line_range(std::istream& is);
{
    using separator = boost::char_separator<char>;

    return istream_tokenizer{std::istreambuf_iterator<char>{is},
                             std::istreambuf_iterator<char>{},
                             separator{"\n", "", boost::keep_empty_tokens}};
}

This hardcodes char as the stream's character type, but this could be templatized.

The function can be used as follows:

#include <sstream>

std::istringstream is{"A\nBB\n\nCCC"};

auto lines = line_range(is);
std::vector<std::string> line_vec{lines.begin(), lines.end()};
assert(line_vec == (std::vector<std::string>{{"A", "BB", "", "CCC"}}));

Naturally, it can also be used with an std::ifstream created by opening a file:

std::ifstream ifs{"filename.txt"};
auto lines = line_range(ifs);

Answer 5

In a related thread iterate-over-cin-line-by-line quoted above, Jerry Coffin described "another possibility (which) uses a part of the standard library most people barely even know exists." The following applies that method (which was what I was looking for) to solve the iterate-over-file-line-by-line problem as requested in the current thread.

First a snippet copied directly from Jerry's answer in the related thread:

struct line_reader: std::ctype<char> {
line_reader(): std::ctype<char>(get_table()) {}
static std::ctype_base::mask const* get_table() {
    static std::vector<std::ctype_base::mask> rc(table_size, std::ctype_base::mask());
    rc['\n'] = std::ctype_base::space;
    return &rc[0];
}}; 

And now, imbue the ifstream with the custom locale as described by Jerry, and copy from infstream to ofstream.

ifstream is {"fox.txt"};
is.imbue(locale(locale(), new line_reader()));
istream_iterator<string> ii {is};
istream_iterator<string> eos {};

ofstream os {"out.txt"};
ostream_iterator<string> oi {os,"\n"};

vector<string> lines {ii,eos};
copy(lines.begin(), lines.end(), oi);

The output file ("out.txt") will be exactly the same as the input file ("fox.txt").

Answer 6

Here is a solution. The exemple print the input file with @@ at the end of each line.

#include <iostream>
#include <iterator>
#include <fstream>
#include <string>

using namespace std;

class line : public string {};

std::istream &operator>>(std::istream &is, line &l)
{
    std::getline(is, l);
    return is;
}

int main()
{
    std::ifstream inputFile("input.txt");

    istream_iterator<line> begin(inputFile);
    istream_iterator<line> end;

    for(istream_iterator<line> it = begin; it != end; ++it)
    {
        cout << *it << "@@\n";
    }

    getchar();
}

Edit : Manuel has been faster.