Enumerator as an infinite generator in Ruby

Question

I\'m reading one resource explaining how Enumerators can be used as generators, which as an example like:

triangular_numbers = Enumerator.new do |yielder|
  numb         


        

Answer 1

I think I've found something that you may find interesting.

This article: 'Ruby 2.0 Works Hard So You Can Be Lazy' by Pat Shaughnessy explains the ideas behind Eager and Lazy evaluation, and also explains how that relates to the "framework classes" like Enumerale, Generator or Yielder. It is mostly focused on explaining how to achieve LazyEvaluation, but still, it's quite detailed.

---

Original Source: 'Ruby 2.0 Works Hard So You Can Be Lazy' by Pat Shaughnessy

Ruby 2.0 implements lazy evaluation using an object called Enumerator::Lazy. What makes this special is that it plays both roles! It is an enumerator, and also contains a series of Enumerable methods. It calls each to obtain data from an enumeration source, and it yields data to the rest of an enumeration. Since Enumerator::Lazy plays both roles, you can chain them up together to produce a single enumeration.

This is the key to lazy evaluation in Ruby. Each value from the data source is yielded to my block, and then the result is immediately passed along down the enumeration chain. This enumeration is not eager – the Enumerator::Lazy#collect method does not collect the values into an array. Instead, each value is passed one at a time along the chain of Enumerator::Lazy objects, via repeated yields. If I had chained together a series of calls to collect or other Enumerator::Lazy methods, each value would be passed along the chain from one of my blocks to the next, one at a time

Enumerable#first both starts the iteration by calling each on the lazy enumerators, and ends the iteration by raising an exception when it has enough values.

At the end of the day, this is the key idea behind lazy evaluation: the function or method at the end of a calculation chain starts the execution process, and the program’s flow works backwards through the chain of function calls until it obtains just the data inputs it needs. Ruby achieves this using a chain of Enumerator::Lazy objects.

Answer 2

The Yielder is just a piece of code that returns the value and wait until the next call.

This can be easily achieve by using the Ruby Fiber Class. See the following example that creates a SimpleEnumerator class:

class SimpleEnumerator

  def initialize &block
    # creates a new Fiber to be used as an Yielder
    @yielder  = Fiber.new do
      yield Fiber # call the block code. The same as: block.call Fiber
      raise StopIteration # raise an error if there is no more calls
    end
  end

  def next
    # return the value and wait until the next call
    @yielder.resume
  end

end

triangular_numbers = SimpleEnumerator.new do |yielder|
  number  = 0
  count   = 1
  loop do
    number  += count
    count   += 1
    yielder.yield number
  end
end

print triangular_numbers.next, " " 
print triangular_numbers.next, " " 
print triangular_numbers.next, " " 

I just replaced Enumerator.new in your code by SimpleEnumerator.new and the results are the same.

There is a "light weight cooperative concurrency"; using the Ruby documentation words, where the programmer schedules what should be done, in other words, the programmer can pause and resume the code block.

Answer 3

I found a nice concise answer in Ruby Cookbook:

https://books.google.com/books?id=xBmkBwAAQBAJ&pg=PT463&lpg=PT463&dq=upgrade+ruby+1.8+generator&source=bl&ots=yyVBoNUhNj&sig=iYXXR_8QqVMasFnS53sbUzGAbTc&hl=en&sa=X&ei=fOM-VZb0BoXSsAWulIGIAw&ved=0CFcQ6AEwBw#v=onepage&q=upgrade%20ruby%201.8%20generator&f=false

This shows how to make a Ruby 1.8 style Generator using the Ruby 2.0+ Enumerator class.

my_array = ['v1', 'v2']

my_generator = Enumerator.new do |yielder|
    index = 0
    loop do
        yielder.yield(my_array[index])
        index += 1
    end
end

my_generator.next    # => 'v1'
my_generator.next    # => 'v2'
my_generator.next    # => nil

Answer 4

Suppose we want to print the first three triangular numbers. A naive implementation would be to use a function:

def print_triangular_numbers steps
  number = 0
  count = 1
  steps.times do
    number += count
    count += 1
    print number, " "
  end
end

print_triangular_numbers(3)

The disadvantage here is that we're mixing the printing logic with the counting logic. If we don't want to print the numbers, this isn't useful. We can improve this by instead yielding the numbers to a block:

def triangular_numbers steps
  number = 0
  count = 1
  steps.times do
    number += count
    count += 1
    yield number
  end
end

triangular_numbers(3) { |n| print n, " " }

Now suppose we want to print a few triangular numbers, do some other stuff, then continue printing them. Again, a naive solution:

def triangular_numbers steps, start = 0
  number = 0
  count = 1
  (steps + start).times do
    number += count
    yield number if count > start
    count += 1
  end
end

triangular_numbers(4) { |n| print n, " " }

# do other stuff

triangular_numbers(3, 4) { |n| print n, " " }

This has the disadvantage that every time we want to resume printing triangular numbers, we need to start from scratch. Inefficient! What we need is a way to remember where we left off so that we can resume later. Variables with a proc make an easy solution:

number = 0
count = 1
triangular_numbers = proc do |&blk|
  number += count
  count += 1
  blk.call number
end

4.times { triangular_numbers.call { |n| print n, " " } }

# do other stuff

3.times { triangular_numbers.call { |n| print n, " " } }

But this is one step forward and two steps back. We can easily resume, but there's no encapsulation of the logic (we could accidentally change number and ruin everything!). What we really want is an object where we can store the state. This is exactly what Enumerator is for.

triangular_numbers = Enumerator.new do |yielder|
  number = 0
  count = 1
  loop do
    number += count
    count += 1
    yielder.yield number
  end
end

4.times { print triangular_numbers.next, " " }

# do other stuff

3.times { print triangular_numbers.next, " " }

Since blocks are closures in Ruby, the loop remembers the state of number and count between calls. This is what makes it seem like the enumerator is running in parallel.

Now we get to the yielder. Note that it replaces blk.call number from the previous example where we used a proc. blk.call worked, but it was inflexible. In Ruby, you don't always have to provide enumerators with blocks. Sometimes you just want to enumerate one step at a time or chain enumerators together, in those cases having your enumerator simply pass a value to a block is inconvenient. Enumerator makes enumerators much simpler to write by providing the agnostic Enumerator::Yielder interface. When you give a value to the yielder (yielder.yield number or yielder << number), you're telling the enumerator "Whenever someone asks for the next value (be it in a block, with next, each, or passed directly to another enumerator), give them this." The yield keyword simply wouldn't cut it here because it is only for yielding values to blocks.