What is the correct way of using C++11's range-based for?

Question

What is the correct way of using C++11\'s range-based for?

What syntax should be used? for (auto elem : container), or for (auto&

Answer 1

There is no correct way to use for (auto elem : container), or for (auto& elem : container) or for (const auto& elem : container). You just express what you want.

Let me elaborate on that. Let's take a stroll.

for (auto elem : container) ...

This one is syntactic sugar for:

for(auto it = container.begin(); it != container.end(); ++it) {

    // Observe that this is a copy by value.
    auto elem = *it;

}

You can use this one if it your container contains elements which are cheap to copy.

for (auto& elem : container) ...

This one is syntactic sugar for:

for(auto it = container.begin(); it != container.end(); ++it) {

    // Now you're directly modifying the elements
    // because elem is an lvalue reference
    auto& elem = *it;

}

Use this when you want to write to the elements in the container directly, for example.

for (const auto& elem : container) ...

This one is syntactic sugar for:

for(auto it = container.begin(); it != container.end(); ++it) {

    // You just want to read stuff, no modification
    const auto& elem = *it;

}

As the comment says, just for reading. And that's about it, everything is "correct" when used properly.

Answer 2

The correct means is always

for(auto&& elem : container)

This will guarantee the preservation of all semantics.

Answer 3

Let's start differentiating between observing the elements in the container vs. modifying them in place.

Observing the elements

Let's consider a simple example:

vector<int> v = {1, 3, 5, 7, 9};

for (auto x : v)
    cout << x << ' ';

The above code prints the elements (ints) in the vector:

1 3 5 7 9

Now consider another case, in which the vector elements are not just simple integers, but instances of a more complex class, with custom copy constructor, etc.

// A sample test class, with custom copy semantics.
class X
{
public:
    X() 
        : m_data(0) 
    {}

    X(int data)
        : m_data(data)
    {}

    ~X() 
    {}

    X(const X& other) 
        : m_data(other.m_data)
    { cout << "X copy ctor.\n"; }

    X& operator=(const X& other)
    {
        m_data = other.m_data;       
        cout << "X copy assign.\n";
        return *this;
    }

    int Get() const
    {
        return m_data;
    }

private:
    int m_data;
};

ostream& operator<<(ostream& os, const X& x)
{
    os << x.Get();
    return os;
}

If we use the above for (auto x : v) {...} syntax with this new class:

vector<X> v = {1, 3, 5, 7, 9};

cout << "\nElements:\n";
for (auto x : v)
{
    cout << x << ' ';
}

the output is something like:

[... copy constructor calls for vector<X> initialization ...]

Elements:
X copy ctor.
1 X copy ctor.
3 X copy ctor.
5 X copy ctor.
7 X copy ctor.
9

As it can be read from the output, copy constructor calls are made during range-based for loop iterations.
This is because we are capturing the elements from the container by value (the auto x part in for (auto x : v)).

This is inefficient code, e.g., if these elements are instances of std::string, heap memory allocations can be done, with expensive trips to the memory manager, etc. This is useless if we just want to observe the elements in a container.

So, a better syntax is available: capture by const reference, i.e. const auto&:

vector<X> v = {1, 3, 5, 7, 9};

cout << "\nElements:\n";
for (const auto& x : v)
{ 
    cout << x << ' ';
}

Now the output is:

 [... copy constructor calls for vector<X> initialization ...]

Elements:
1 3 5 7 9

Without any spurious (and potentially expensive) copy constructor call.

So, when observing elements in a container (i.e., for read-only access), the following syntax is fine for simple cheap-to-copy types, like int, double, etc.:

for (auto elem : container) 

Else, capturing by const reference is better in the general case, to avoid useless (and potentially expensive) copy constructor calls:

for (const auto& elem : container) 

Modifying the elements in the container

If we want to modify the elements in a container using range-based for, the above for (auto elem : container) and for (const auto& elem : container) syntaxes are wrong.

In fact, in the former case, elem stores a copy of the original element, so modifications done to it are just lost and not stored persistently in the container, e.g.:

vector<int> v = {1, 3, 5, 7, 9};
for (auto x : v)  // <-- capture by value (copy)
    x *= 10;      // <-- a local temporary copy ("x") is modified,
                  //     *not* the original vector element.

for (auto x : v)
    cout << x << ' ';

The output is just the initial sequence:

1 3 5 7 9

Instead, an attempt of using for (const auto& x : v) just fails to compile.

g++ outputs an error message something like this:

TestRangeFor.cpp:138:11: error: assignment of read-only reference 'x'
          x *= 10;
            ^

The correct approach in this case is capturing by non-const reference:

vector<int> v = {1, 3, 5, 7, 9};
for (auto& x : v)
    x *= 10;

for (auto x : v)
    cout << x << ' ';

The output is (as expected):

10 30 50 70 90

This for (auto& elem : container) syntax works also for more complex types, e.g. considering a vector<string>:

vector<string> v = {"Bob", "Jeff", "Connie"};

// Modify elements in place: use "auto &"
for (auto& x : v)
    x = "Hi " + x + "!";

// Output elements (*observing* --> use "const auto&")
for (const auto& x : v)
    cout << x << ' ';

the output is:

Hi Bob! Hi Jeff! Hi Connie!

The special case of proxy iterators

Suppose we have a vector<bool>, and we want to invert the logical boolean state of its elements, using the above syntax:

vector<bool> v = {true, false, false, true};
for (auto& x : v)
    x = !x;

The above code fails to compile.

g++ outputs an error message similar to this:

TestRangeFor.cpp:168:20: error: invalid initialization of non-const reference of
 type 'std::_Bit_reference&' from an rvalue of type 'std::_Bit_iterator::referen
ce {aka std::_Bit_reference}'
     for (auto& x : v)
                    ^

The problem is that std::vector template is specialized for bool, with an implementation that packs the bools to optimize space (each boolean value is stored in one bit, eight "boolean" bits in a byte).

Because of that (since it's not possible to return a reference to a single bit), vector<bool> uses a so-called "proxy iterator" pattern. A "proxy iterator" is an iterator that, when dereferenced, does not yield an ordinary bool &, but instead returns (by value) a temporary object, which is a proxy class convertible to bool. (See also this question and related answers here on StackOverflow.)

To modify in place the elements of vector<bool>, a new kind of syntax (using auto&&) must be used:

for (auto&& x : v)
    x = !x;

The following code works fine:

vector<bool> v = {true, false, false, true};

// Invert boolean status
for (auto&& x : v)  // <-- note use of "auto&&" for proxy iterators
    x = !x;

// Print new element values
cout << boolalpha;        
for (const auto& x : v)
    cout << x << ' ';

and outputs:

false true true false

Note that the for (auto&& elem : container) syntax also works in the other cases of ordinary (non-proxy) iterators (e.g. for a vector<int> or a vector<string>).

(As a side note, the aforementioned "observing" syntax of for (const auto& elem : container) works fine also for the proxy iterator case.)

Summary

The above discussion can be summarized in the following guidelines:

1. For observing the elements, use the following syntax:

   for (const auto& elem : container)    // capture by const reference
   

   • If the objects are cheap to copy (like ints, doubles, etc.), it's possible to use a slightly simplified form:

     for (auto elem : container)    // capture by value
     

2. For modifying the elements in place, use:

   for (auto& elem : container)    // capture by (non-const) reference
   

   • If the container uses "proxy iterators" (like std::vector<bool>), use:

     for (auto&& elem : container)    // capture by &&
     

Of course, if there is a need to make a local copy of the element inside the loop body, capturing by value (for (auto elem : container)) is a good choice.

---

Additional notes on generic code

In generic code, since we can't make assumptions about generic type T being cheap to copy, in observing mode it's safe to always use for (const auto& elem : container).
(This won't trigger potentially expensive useless copies, will work just fine also for cheap-to-copy types like int, and also for containers using proxy-iterators, like std::vector<bool>.)

Moreover, in modifying mode, if we want generic code to work also in case of proxy-iterators, the best option is for (auto&& elem : container).
(This will work just fine also for containers using ordinary non-proxy-iterators, like std::vector<int> or std::vector<string>.)

So, in generic code, the following guidelines can be provided:

1. For observing the elements, use:

   for (const auto& elem : container)
   

2. For modifying the elements in place, use:

   for (auto&& elem : container)
   

Answer 4

While the initial motivation of the range-for loop might have been ease of iterating over the elements of a container, the syntax is generic enough to be useful even for objects that are not purely containers.

The syntactic requirement for the for-loop is that range_expression support begin() and end() as either functions -- either as member functions of the type that it evaluates to or as non-member functions what take an instance of the type.

As a contrived example, one can generate a range of numbers and iterate over the range using the following class.

struct Range
{
   struct Iterator
   {
      Iterator(int v, int s) : val(v), step(s) {}

      int operator*() const
      {
         return val;
      }

      Iterator& operator++()
      {
         val += step;
         return *this;
      }

      bool operator!=(Iterator const& rhs) const
      {
         return (this->val < rhs.val);
      }

      int val;
      int step;
   };

   Range(int l, int h, int s=1) : low(l), high(h), step(s) {}

   Iterator begin() const
   {
      return Iterator(low, step);
   }

   Iterator end() const
   {
      return Iterator(high, 1);
   }

   int low, high, step;
}; 

With the following main function,

#include <iostream>

int main()
{
   Range r1(1, 10);
   for ( auto item : r1 )
   {
      std::cout << item << " ";
   }
   std::cout << std::endl;

   Range r2(1, 20, 2);
   for ( auto item : r2 )
   {
      std::cout << item << " ";
   }
   std::cout << std::endl;

   Range r3(1, 20, 3);
   for ( auto item : r3 )
   {
      std::cout << item << " ";
   }
   std::cout << std::endl;
}

one would get the following output.

1 2 3 4 5 6 7 8 9 
1 3 5 7 9 11 13 15 17 19 
1 4 7 10 13 16 19