Python: yield-and-delete

Question

How do I yield an object from a generator and forget it immediately, so that it doesn\'t take up memory?

For example, in the following function:

def grou         


        

Answer 1

After yield chunk, the variable value is never used again in the function, so a good interpreter/garbage collector will already free chunk for garbage collection (note: cpython 2.7 seems not do this, pypy 1.6 with default gc does). Therefore, you don't have to change anything but your code example, which is missing the second argument to grouper.

Note that garbage collection is non-deterministic in Python. The null garbage collector, which doesn't collect free objects at all, is a perfectly valid garbage collector. From the Python manual:

Objects are never explicitly destroyed; however, when they become unreachable they may be garbage-collected. An implementation is allowed to postpone garbage collection or omit it altogether — it is a matter of implementation quality how garbage collection is implemented, as long as no objects are collected that are still reachable.

Therefore, it can not be decided whether a Python program does or "doesn't take up memory" without specifying Python implementation and garbage collector. Given a specific Python implementation and garbage collector, you can use the gc module to test whether the object is freed.

That being said, if you really want no reference from the function (not necessarily meaning the object will be garbage-collected), here's how to do it:

def grouper(iterable, chunksize):
    i = iter(iterable)
    while True:
        tmpr = [list(itertools.islice(i, int(chunksize)))]
        if not tmpr[0]:
            break
        yield tmpr.pop()

Instead of a list, you can also use any other data structure that with a function which removes and returns an object, like Owen's wrapper.