How can I, in python, iterate over multiple 2d lists at once, cleanly?

Question

If I\'m making a simple grid based game, for example, I might have a few 2d lists. One might be for terrain, another might be for objects, etc. Unfortunately, when I need to ite

Answer 1

I'd start by writing a generator method:

def grid_objects(alist, blist):
    for i in range(len(alist)):
        for j in range(len(alist[i])):
            yield(alist[i][j], blist[i][j])

Then whenever you need to iterate over the lists your code looks like this:

for (a, b) in grid_objects(alist, blist):
    if a.is_whatever():
        b.do_something()

Answer 2

As a slight style change, you could use enumerate:

for i, arow in enumerate(alist):
    for j, aval in enumerate(arow):
        if aval.isWhatever():
            blist[i][j].doSomething()

I don't think you'll get anything significantly simpler unless you rearrange your data structures as Federico suggests. So that you could turn the last line into something like "aval.b.doSomething()".

Answer 3

When you are operating with grids of numbers and want really good performance, you should consider using Numpy. It's surprisingly easy to use and lets you think in terms of operations with grids instead of loops over grids. The performance comes from the fact that the operations are then run over whole grids with optimised SSE code.

For example here is some numpy using code that I wrote that does brute force numerical simulation of charged particles connected by springs. This code calculates a timestep for a 3d system with 100 nodes and 99 edges in 31ms. That is over 10x faster than the best pure python code I could come up with.

from numpy import array, sqrt, float32, newaxis
def evolve(points, velocities, edges, timestep=0.01, charge=0.1, mass=1., edgelen=0.5, dampen=0.95):
    """Evolve a n body system of electrostatically repulsive nodes connected by
       springs by one timestep."""
    velocities *= dampen

    # calculate matrix of distance vectors between all points and their lengths squared
    dists = array([[p2 - p1 for p2 in points] for p1 in points])
    l_2 = (dists*dists).sum(axis=2)

    # make the diagonal 1's to avoid division by zero
    for i in xrange(points.shape[0]):
        l_2[i,i] = 1

    l_2_inv = 1/l_2
    l_3_inv = l_2_inv*sqrt(l_2_inv)

    # repulsive force: distance vectors divided by length cubed, summed and multiplied by scale
    scale = timestep*charge*charge/mass
    velocities -= scale*(l_3_inv[:,:,newaxis].repeat(points.shape[1], axis=2)*dists).sum(axis=1)

    # calculate spring contributions for each point
    for idx, (point, outedges) in enumerate(izip(points, edges)):
        edgevecs = point - points.take(outedges, axis=0)
        edgevec_lens = sqrt((edgevecs*edgevecs).sum(axis=1))
        scale = timestep/mass
        velocities[idx] += (edgevecs*((((edgelen*scale)/edgevec_lens - scale))[:,newaxis].repeat(points.shape[1],axis=1))).sum(axis=0)

    # move points to new positions
    points += velocities*timestep

Answer 4

Generator expressions and izip from itertools module will do very nicely here:

from itertools import izip
for a, b in (pair for (aline, bline) in izip(alist, blist) 
             for pair in izip(aline, bline)):
    if a.isWhatever:
        b.doSomething()

The line in for statement above means:

• take each line from combined grids alist and blist and make a tuple from them (aline, bline)
• now combine these lists with izip again and take each element from them (pair).

This method has two advantages:

• there are no indices used anywhere
• you don't have to create lists with zip and use more efficient generators with izip instead.

Answer 5

Are you sure that the objects in the two matrices you are iterating in parallel are instances of conceptually distinct classes? What about merging the two classes ending up with a matrix of objects that contain both isWhatever() and doSomething()?

Answer 6

If anyone is interested in performance of the above solutions, here they are for 4000x4000 grids, from fastest to slowest:

• Brian: 1.08s (modified, with izip instead of zip)
• John: 2.33s
• DzinX: 2.36s
• ΤΖΩΤΖΙΟΥ: 2.41s (but object initialization took 62s)
• Eugene: 3.17s
• Robert: 4.56s
• Brian: 27.24s (original, with zip)

EDIT: Added Brian's scores with izip modification and it won by a large amount!

John's solution is also very fast, although it uses indices (I was really surprised to see this!), whereas Robert's and Brian's (with zip) are slower than the question creator's initial solution.

So let's present Brian's winning function, as it is not shown in proper form anywhere in this thread:

from itertools import izip
for a_row,b_row in izip(alist, blist):
    for a_item, b_item in izip(a_row,b_row):
        if a_item.isWhatever:
            b_item.doSomething()