Create a random order of (x, y) pairs, without repeating/subsequent x's

Question

Say I have a list of valid X = [1, 2, 3, 4, 5] and a list of valid Y = [1, 2, 3, 4, 5].

I need to generate all combinations of every element in

Answer 1

Distribute the x values (5 times each value) evenly across your output:

import random

def random_combo_without_x_repeats(xvals, yvals):
    # produce all valid combinations, but group by `x` and shuffle the `y`s
    grouped = [[x, random.sample(yvals, len(yvals))] for x in xvals]
    last_x = object()  # sentinel not equal to anything
    while grouped[0][1]:  # still `y`s left
        for _ in range(len(xvals)):
            # shuffle the `x`s, but skip any ordering that would
            # produce consecutive `x`s.
            random.shuffle(grouped)
            if grouped[0][0] != last_x:
                break
        else:
            # we tried to reshuffle N times, but ended up with the same `x` value
            # in the first position each time. This is pretty unlikely, but
            # if this happens we bail out and just reverse the order. That is
            # more than good enough.
            grouped = grouped[::-1]
        # yield a set of (x, y) pairs for each unique x
        # Pick one y (from the pre-shuffled groups per x
        for x, ys in grouped:
            yield x, ys.pop()
        last_x = x

This shuffles the y values per x first, then gives you a x, y combination for each x. The order in which the xs are yielded is shuffled each iteration, where you test for the restriction.

This is random, but you'll get all numbers between 1 and 5 in the x position before you'll see the same number again:

>>> list(random_combo_without_x_repeats(range(1, 6), range(1, 6)))
[(2, 1), (3, 2), (1, 5), (5, 1), (4, 1),
 (2, 4), (3, 1), (4, 3), (5, 5), (1, 4),
 (5, 2), (1, 1), (3, 3), (4, 4), (2, 5),
 (3, 5), (2, 3), (4, 2), (1, 2), (5, 4),
 (2, 2), (3, 4), (1, 3), (4, 5), (5, 3)]

(I manually grouped that into sets of 5). Overall, this makes for a pretty good random shuffling of a fixed input set with your restriction.

It is efficient too; because there is only a 1-in-N chance that you have to re-shuffle the x order, you should only see one reshuffle on average take place during a full run of the algorithm. The whole algorithm stays within O(N*M) boundaries therefor, pretty much ideal for something that produces N times M elements of output. Because we limit the reshuffling to N times at most before falling back to a simple reverse we avoid the (extremely unlikely) posibility of endlessly reshuffling.

The only drawback then is that it has to create N copies of the M y values up front.

Answer 2

A simple solution to ensure an average O(N*M) complexity:

def pseudorandom(M,N):
    l=[(x+1,y+1) for x in range(N) for y in range(M)]
    random.shuffle(l)
    for i in range(M*N-1):
            for j in range (i+1,M*N): # find a compatible ...
                if l[i][0] != l[j][0]:
                    l[i+1],l[j] = l[j],l[i+1]
                    break  
            else:   # or insert otherwise.
                while True:
                    l[i],l[i-1] = l[i-1],l[i]
                    i-=1
                    if l[i][0] != l[i-1][0]: break  
    return l

Some tests:

In [354]: print(pseudorandom(5,5))
[(2, 2), (3, 1), (5, 1), (1, 1), (3, 2), (1, 2), (3, 5), (1, 5), (5, 4),\
(1, 3), (5, 2), (3, 4), (5, 3), (4, 5), (5, 5), (1, 4), (2, 5), (4, 4), (2, 4),\ 
(4, 2), (2, 1), (4, 3), (2, 3), (4, 1), (3, 3)]

In [355]: %timeit pseudorandom(100,100)
10 loops, best of 3: 41.3 ms per loop

Answer 3

Here is an evolutionary algorithm approach. It first evolves a list in which the elements of X are each repeated len(Y) times and then it randomly fills in each element of Y len(X) times. The resulting orders seem fairly random:

import random

#the following fitness function measures
#the number of times in which
#consecutive elements in a list
#are equal

def numRepeats(x):
    n = len(x)
    if n < 2: return 0
    repeats = 0
    for i in range(n-1):
        if x[i] == x[i+1]: repeats += 1
    return repeats

def mutate(xs):
    #swaps random pairs of elements
    #returns a new list
    #one of the two indices is chosen so that
    #it is in a repeated pair
    #and swapped element is different

    n = len(xs)
    repeats = [i for i in range(n) if (i > 0 and xs[i] == xs[i-1]) or (i < n-1 and xs[i] == xs[i+1])]
    i = random.choice(repeats)
    j = random.randint(0,n-1)
    while xs[j] == xs[i]: j = random.randint(0,n-1)
    ys = xs[:]
    ys[i], ys[j] = ys[j], ys[i]
    return ys

def evolveShuffle(xs, popSize = 100, numGens = 100):
    #tries to evolve a shuffle of xs so that consecutive
    #elements are different
    #takes the best 10% of each generation and mutates each 9
    #times. Stops when a perfect solution is found
    #popsize assumed to be a multiple of 10

    population = []

    for i in range(popSize):
        deck = xs[:]
        random.shuffle(deck)
        fitness = numRepeats(deck)
        if fitness == 0: return deck
        population.append((fitness,deck))

    for i in range(numGens):
        population.sort(key = (lambda p: p[0]))
        newPop = []
        for i in range(popSize//10):
            fit,deck = population[i]
            newPop.append((fit,deck))
            for j in range(9):
                newDeck = mutate(deck)
                fitness = numRepeats(newDeck)
                if fitness == 0: return newDeck
                newPop.append((fitness,newDeck))
        population = newPop
    #if you get here :
    return [] #no special shuffle found

#the following function takes a list x
#with n distinct elements (n>1) and an integer k
#and returns a random list of length nk
#where consecutive elements are not the same

def specialShuffle(x,k):
    n = len(x)
    if n == 2:
        if random.random() < 0.5:
            a,b = x
        else:
            b,a = x
        return [a,b]*k
    else:
        deck = x*k
        return evolveShuffle(deck)

def randOrder(x,y):
    xs = specialShuffle(x,len(y))
    d = {}
    for i in x:
        ys = y[:]
        random.shuffle(ys)
        d[i] = iter(ys)

    pairs = []
    for i in xs:
        pairs.append((i,next(d[i])))
    return pairs

for example:

>>> randOrder([1,2,3,4,5],[1,2,3,4,5])
[(1, 4), (3, 1), (4, 5), (2, 2), (4, 3), (5, 3), (2, 1), (3, 3), (1, 1), (5, 2), (1, 3), (2, 5), (1, 5), (3, 5), (5, 5), (4, 4), (2, 3), (3, 2), (5, 4), (2, 4), (4, 2), (1, 2), (5, 1), (4, 1), (3, 4)]

As len(X) and len(Y) gets larger this has more difficulty finding a solution (and is designed to return the empty list in that eventuality), in which case the parameters popSize and numGens could be increased. As is, it is able to find 20x20 solutions very rapidly. It takes about a minute when X and Y are of size 100 but even then is able to find a solution (in the times that I have run it).

Answer 4

An interesting question! Here is my solution. It has the following properties:

• If there is no valid solution it should detect this and let you know
• The iteration is guaranteed to terminate so it should never get stuck in an infinite loop
• Any possible solution is reachable with nonzero probability

I do not know the distribution of the output over all possible solutions, but I think it should be uniform because there is no obvious asymmetry inherent in the algorithm. I would be surprised and pleased to be shown otherwise, though!

import random

def random_without_repeats(xs, ys):
    pairs = [[x,y] for x in xs for y in ys]
    output = [[object()], [object()]]
    seen = set()
    while pairs:
        # choose a random pair from the ones left
        indices = list(set(xrange(len(pairs))) - seen)
        try:
            index = random.choice(indices)
        except IndexError:
            raise Exception('No valid solution exists!')
        # the first element of our randomly chosen pair
        x = pairs[index][0]
        # search for a valid place in output where we slot it in
        for i in xrange(len(output) - 1):
            left, right = output[i], output[i+1]
            if x != left[0] and x != right[0]:
                output.insert(i+1, pairs.pop(index))
                seen = set()
                break
        else:
            # make sure we don't randomly choose a bad pair like that again
            seen |= {i for i in indices if pairs[i][0] == x}
    # trim off the sentinels
    output = output[1:-1]
    assert len(output) == len(xs) * len(ys)
    assert not any(L==R for L,R in zip(output[:-1], output[1:]))
    return output


nx, ny = 5, 5       # OP example
# nx, ny = 2, 10      # output must alternate in 1st index
# nx, ny = 4, 13      # shuffle 'deck of cards' with no repeating suit
# nx, ny = 1, 5       # should raise 'No valid solution exists!' exception

xs = range(1, nx+1)
ys = range(1, ny+1)

for pair in random_without_repeats(xs, ys):
    print pair

Answer 5

For completeness, I guess I will throw in the super-naive "just keep shuffling till you get one" solution. It's not guaranteed to even terminate, but if it does, it will have a good degree of randomness, and you did say one of the desired qualities was succinctness, and this sure is succinct:

import itertools
import random

x = range(5)  # this is a list in Python 2
y = range(5)
all_pairs = list(itertools.product(x, y))

s = list(all_pairs)  # make a working copy
while any(s[i][0] == s[i + 1][0] for i in range(len(s) - 1)):
    random.shuffle(s)
print s

As was commented, for small values of x and y (especially y!), this is actually a reasonably quick solution. Your example of 5 for each completes in an average time of "right away". The deck of cards example (4 and 13) can take much longer, because it will usually require hundreds of thousands of shuffles. (And again, is not guaranteed to terminate at all.)

Answer 6

Interesting restriction! I probably overthought this, solving a more general problem: shuffling an arbitrary list of sequences such that (if possible) no two adjacent sequences share a first item.

from itertools import product
from random import choice, randrange, shuffle

def combine(*sequences):
    return playlist(product(*sequences))

def playlist(sequence):
    r'''Shuffle a set of sequences, avoiding repeated first elements.
    '''#"""#'''
    result = list(sequence)
    length = len(result)
    if length < 2:
        # No rearrangement is possible.
        return result
    def swap(a, b):
        if a != b:
            result[a], result[b] = result[b], result[a]
    swap(0, randrange(length))
    for n in range(1, length):
        previous = result[n-1][0]
        choices = [x for x in range(n, length) if result[x][0] != previous]
        if not choices:
            # Trapped in a corner: Too many of the same item are left.
            # Backtrack as far as necessary to interleave other items.
            minor = 0
            major = length - n
            while n > 0:
                n -= 1
                if result[n][0] == previous:
                    major += 1
                else:
                    minor += 1
                if minor == major - 1:
                    if n == 0 or result[n-1][0] != previous:
                        break
            else:
                # The requirement can't be fulfilled,
                # because there are too many of a single item.
                shuffle(result)
                break

            # Interleave the majority item with the other items.
            major = [item for item in result[n:] if item[0] == previous]
            minor = [item for item in result[n:] if item[0] != previous]
            shuffle(major)
            shuffle(minor)
            result[n] = major.pop(0)
            n += 1
            while n < length:
                result[n] = minor.pop(0)
                n += 1
                result[n] = major.pop(0)
                n += 1
            break
        swap(n, choice(choices))
    return result

This starts out simple, but when it discovers that it can't find an item with a different first element, it figures out how far back it needs to go to interleave that element with something else. Therefore, the main loop traverses the array at most three times (once backwards), but usually just once. Granted, each iteration of the first forward pass checks each remaining item in the array, and the array itself contains every pair, so the overall run time is O((NM)**2).

For your specific problem:

>>> X = Y = [1, 2, 3, 4, 5]
>>> combine(X, Y)
[(3, 5), (1, 1), (4, 4), (1, 2), (3, 4),
 (2, 3), (5, 4), (1, 5), (2, 4), (5, 5),
 (4, 1), (2, 2), (1, 4), (4, 2), (5, 2),
 (2, 1), (3, 3), (2, 5), (3, 2), (1, 3),
 (4, 3), (5, 3), (4, 5), (5, 1), (3, 1)]

By the way, this compares x values by equality, not by position in the X array, which may make a difference if the array can contain duplicates. In fact, duplicate values might trigger the fallback case of shuffling all pairs together if more than half of the X values are the same.