Why can't iterator methods take either 'ref' or 'out' parameters?

Question

I tried this earlier today:

public interface IFoo
{
    IEnumerable GetItems_A( ref int somethingElse );
    IEnumerable GetItems_B( ref in         


        

Answer 1

Others have explained why your iterator can't have a ref parameter. Here's a simple alternative:

public interface IFoo
{
    IEnumerable<int> GetItems( int[] box );
    ...
}

public class Bar : IFoo
{
    public IEnumerable<int> GetItems( int[] box )
    {
        int value = box[0];
        // use and change value and yield to your heart's content
        box[0] = value;
    }
}

If you have several items to pass in and out, define a class to hold them.

Answer 2

I've gotten around this problem using functions, when the value that I need to return is derived from the iterated items:

// One of the problems with Enumerable.Count() is
// that it is a 'terminator', meaning that it will
// execute the expression it is given, and discard
// the resulting sequence. To count the number of
// items in a sequence without discarding it, we 
// can use this variant that takes an Action<int>
// (or Action<long>), invokes it and passes it the
// number of items that were yielded.
//
// Example: This example allows us to find out
//          how many items were in the original
//          source sequence 'items', as well as
//          the number of items consumed by the
//          call to Sum(), without causing any 
//          LINQ expressions involved to execute
//          multiple times.
// 
//   int start = 0;    // the number of items from the original source
//   int finished = 0; // the number of items in the resulting sequence
//
//   IEnumerable<KeyValuePair<string, double>> items = // assumed to be an iterator
//
//   var result = items.Count( i => start = i )
//                   .Where( p => p.Key = "Banana" )
//                      .Select( p => p.Value )
//                         .Count( i => finished = i )
//                            .Sum();
//
//   // by getting the count of items operated 
//   // on by Sum(), we can calculate an average:
// 
//   double average = result / (double) finished; 
//
//   Console.WriteLine( "started with {0} items", start );
//   Console.WriteLine( "finished with {0} items", finished );
//

public static IEnumerable<T> Count<T>( 
    this IEnumerable<T> source, 
    Action<int> receiver )
{
  int i = 0;
  foreach( T item in source )
  {
    yield return item;
    ++i ;
  }
  receiver( i );
}

public static IEnumerable<T> Count<T>( 
    this IEnumerable<T> source, 
    Action<long> receiver )
{
  long i = 0;
  foreach( T item in source )
  {
    yield return item;
    ++i ;
  }
  receiver( i );
}

Answer 3

At a highish level, A ref variable can point to many locations including to value types that are on the stack. The time at which the iterator is initially created by calling the iterator method and when the ref variable would be assigned are two very different times. It is not possible to guarantee that the variable which originally was passed by reference is still around when the iterator actually executes. Hence it is not allowed (or verifiable)

Answer 4

If you want to return both an iterator and an int from your method, a workaround is this:

public class Bar : IFoo
{
    public IEnumerable<int> GetItems( ref int somethingElse )
    {
        somethingElse = 42;
        return GetItemsCore();
    }

    private IEnumerable<int> GetItemsCore();
    {
        yield return 7;
    }
}

You should note that none of the code inside an iterator method (i.e. basically a method that contains yield return or yield break) is executed until the MoveNext() method in the Enumerator is called. So if you were able to use out or ref in your iterator method, you would get surprising behavior like this:

// This will not compile:
public IEnumerable<int> GetItems( ref int somethingElse )
{
    somethingElse = 42;
    yield return 7;
}

// ...
int somethingElse = 0;
IEnumerable<int> items = GetItems( ref somethingElse );
// at this point somethingElse would still be 0
items.GetEnumerator().MoveNext();
// but now the assignment would be executed and somethingElse would be 42

This is a common pitfall, a related issue is this:

public IEnumerable<int> GetItems( object mayNotBeNull ){
  if( mayNotBeNull == null )
    throw new NullPointerException();
  yield return 7;
}

// ...
IEnumerable<int> items = GetItems( null ); // <- This does not throw
items.GetEnumerators().MoveNext();                    // <- But this does

So a good pattern is to separate iterator methods into two parts: one to execute immediately and one that contains the code that should be lazily executed.

public IEnumerable<int> GetItems( object mayNotBeNull ){
  if( mayNotBeNull == null )
    throw new NullPointerException();
  // other quick checks
  return GetItemsCore( mayNotBeNull );
}

private IEnumerable<int> GetItemsCore( object mayNotBeNull ){
  SlowRunningMethod();
  CallToDatabase();
  // etc
  yield return 7;
}    
// ...
IEnumerable<int> items = GetItems( null ); // <- Now this will throw

EDIT: If you really want the behavior where moving the iterator would modify the ref-parameter, you could do something like this:

public static IEnumerable<int> GetItems( Action<int> setter, Func<int> getter )
{
    setter(42);
    yield return 7;
}

//...

int local = 0;
IEnumerable<int> items = GetItems((x)=>{local = x;}, ()=>local);
Console.WriteLine(local); // 0
items.GetEnumerator().MoveNext();
Console.WriteLine(local); // 42

Answer 5

C# iterators are state machines internally. Every time you yield return something, the place where you left off should be saved along with the state of local variables so that you could get back and continue from there.

To hold this state, C# compiler creates a class to hold local variables and the place it should continue from. It's not possible to have a ref or out value as a field in a class. Consequently, if you were allowed to declare a parameter as ref or out, there would be no way to keep the complete snapshot of the function at the time we had left off.

EDIT: Technically, not all methods that return IEnumerable<T> are considered iterators. Just those that use yield to produce a sequence directly are considered iterators. Therefore, while the splitting the iterator into two methods is a nice and common workaround, it doesn't contradict with what I just said. The outer method (that doesn't use yield directly) is not considered an iterator.