Shuffle list with some conditions

Question

I have a list of elements which can be easily compared using Equals(). I have to shuffle the list, but the shuffle must satisfy one condition:

The i\'th

Answer 1

Your requirements eliminate the real shuffling alternative: there is no randomness, or there is controlled randomness. Here is one special approach

1. Sort the original list L
2. Find the mode M, and its frequency n
3. If n is even, n++.
4. Create k (= 5*n - 1) lists (prime to n, and 5 times n) L₁ through L_k
   (5 is chosen to avoid two previous elements and two next elements)
5. Assign all elements into the k lists in round robin fashion
6. Individually shuffle all k lists.
7. Collate the contents of k lists in the following order: a. pick randomly +5 or -5 as x.
   b. pick a random number j.
   c. repeat k times:
   i. add all contents from L_j.
   ii. j <- (j + x) mod k

[5, 6, 7, 7, 8, 8, 9, 10, 12, 13, 13, 14, 14, 14, 17, 18, 18, 19, 19, 20, 21, 21, 21, 21, 24, 24, 26, 26, 26, 27, 27, 27, 29, 29, 30, 31, 31, 31, 31, 32, 32, 32, 33, 35, 35, 37, 38, 39, 40, 42, 43, 44, 44, 46, 46, 47, 48, 50, 50, 50, 50, 51, 52, 53, 54, 55, 56, 57, 57, 58, 60, 60, 60, 61, 62, 63, 63, 64, 64, 65, 65, 65, 68, 71, 71, 72, 72, 73, 74, 74, 74, 74, 75, 76, 76, 76, 77, 77, 77, 78, 78, 78, 79, 79, 80, 81, 82, 86, 88, 88, 89, 89, 90, 91, 92, 92, 92, 93, 93, 94, 94, 95, 96, 99, 99, 100, 102, 102, 103, 103, 105, 106, 106, 107, 108, 113, 115, 116, 118, 119, 123, 124, 125, 127, 127, 127, 128, 131, 133, 133, 134, 135, 135, 135, 137, 137, 137, 138, 139, 141, 143, 143, 143, 145, 146, 147, 153, 156, 157, 158, 160, 164, 166, 170, 173, 175, 181, 181, 184, 185, 187, 188, 190, 200, 200, 215, 217, 234, 238, 240]

Frequency of mode = 4, so 19 slots (#0 - #18)

0: [7, 21, 32, 50, 65, 77, 93, 115, 137, 173]
1: [8, 21, 33, 51, 65, 78, 93, 116, 137, 175]
2: [8, 24, 35, 52, 65, 78, 94, 118, 138, 181]
3: [9, 24, 35, 53, 68, 78, 94, 119, 139, 181]
4: [10, 26, 37, 54, 71, 79, 95, 123, 141, 184]
5: [12, 26, 38, 55, 71, 79, 96, 124, 143, 185]
6: [13, 26, 39, 56, 72, 80, 99, 125, 143, 187]
7: [13, 27, 40, 57, 72, 81, 99, 127, 143, 188]
8: [14, 27, 42, 57, 73, 82, 100, 127, 145, 190]
9: [14, 27, 43, 58, 74, 86, 102, 127, 146, 200]
10: [14, 29, 44, 60, 74, 88, 102, 128, 147, 200]
11: [17, 29, 44, 60, 74, 88, 103, 131, 153, 215]
12: [18, 30, 46, 60, 74, 89, 103, 133, 156, 217]
13: [18, 31, 46, 61, 75, 89, 105, 133, 157, 234]
14: [19, 31, 47, 62, 76, 90, 106, 134, 158, 238]
15: [19, 31, 48, 63, 76, 91, 106, 135, 160, 240]
16: [5, 20, 31, 50, 63, 76, 92, 107, 135, 164]
17: [6, 21, 32, 50, 64, 77, 92, 108, 135, 166]
18: [7, 21, 32, 50, 64, 77, 92, 113, 137, 170]

Shuffling individual lists, and picking lists 5 slots apart (start randomly at #16):

16: [31, 135, 92, 76, 107, 5, 164, 63, 20, 50]
2: [52, 24, 35, 78, 181, 8, 138, 94, 118, 65]
7: [57, 143, 99, 81, 40, 13, 127, 72, 188, 27]
12: [46, 30, 60, 89, 133, 74, 156, 18, 103, 217]
17: [64, 50, 135, 92, 21, 32, 108, 77, 166, 6]
3: [9, 94, 181, 119, 24, 35, 139, 68, 53, 78]
8: [145, 27, 14, 57, 42, 100, 190, 82, 73, 127]
13: [89, 18, 75, 61, 157, 234, 133, 105, 31, 46]
18: [113, 21, 7, 92, 64, 32, 137, 50, 170, 77]
4: [71, 10, 37, 26, 123, 54, 184, 79, 95, 141]
9: [27, 74, 86, 14, 102, 146, 127, 43, 58, 200]
14: [62, 106, 158, 134, 19, 47, 238, 31, 76, 90]
0: [7, 77, 65, 21, 50, 93, 173, 115, 32, 137]
5: [96, 79, 26, 185, 12, 71, 124, 143, 55, 38]
10: [29, 14, 147, 60, 128, 88, 74, 44, 102, 200]
15: [106, 240, 63, 48, 91, 19, 160, 31, 76, 135]
1: [65, 33, 21, 51, 137, 8, 175, 93, 116, 78]
6: [143, 26, 13, 56, 99, 72, 39, 80, 187, 125]
11: [103, 88, 29, 60, 74, 44, 17, 153, 131, 215]

[31, 135, 92, 76, 107, 5, 164, 63, 20, 50, 52, 24, 35, 78, 181, 8, 138, 94, 118, 65, 57, 143, 99, 81, 40, 13, 127, 72, 188, 27, 46, 30, 60, 89, 133, 74, 156, 18, 103, 217, 64, 50, 135, 92, 21, 32, 108, 77, 166, 6, 9, 94, 181, 119, 24, 35, 139, 68, 53, 78, 145, 27, 14, 57, 42, 100, 190, 82, 73, 127, 89, 18, 75, 61, 157, 234, 133, 105, 31, 46, 113, 21, 7, 92, 64, 32, 137, 50, 170, 77, 71, 10, 37, 26, 123, 54, 184, 79, 95, 141, 27, 74, 86, 14, 102, 146, 127, 43, 58, 200, 62, 106, 158, 134, 19, 47, 238, 31, 76, 90, 7, 77, 65, 21, 50, 93, 173, 115, 32, 137, 96, 79, 26, 185, 12, 71, 124, 143, 55, 38, 29, 14, 147, 60, 128, 88, 74, 44, 102, 200, 106, 240, 63, 48, 91, 19, 160, 31, 76, 135, 65, 33, 21, 51, 137, 8, 175, 93, 116, 78, 143, 26, 13, 56, 99, 72, 39, 80, 187, 125, 103, 88, 29, 60, 74, 44, 17, 153, 131, 215]

Answer 2

• permute valid 5 in list, if none unsolvable
• remove permutation from list
• create a cycle graph of that 5
• order the list(remaining list) by count of valid positions in new graph (if you dont do this you can end up with a wrong non solvable, because putting items that can go on more positions increases the possible position count of items with less)
• continue picking items in the list, add items to cycle graph at valid positions
• if there is no valid position in graph or graph cannot be created continue on next
• if graph created revert back to start iteration where graph has been created
• continue creating other graphs
• save all full graphs in a list
• pick a random from that list
• if list is empty unsolvable

Answer 3

It can be accomplished with a simple, two-step process. First use a Fisher-Yates (Knuth) shuffle in place. Then iterate over the list once, copying its elements to a new list. As you encounter an element, insert it into the first legal position in your new list. (This will be much more efficient with a linked list than with an array as your destination.) If there are no legal position to insert to, the instance of the problem is unsolvable. If you manage to fill your destination list, problem solved. This will take O(n) in the best case and O(n^2) in the worst.

Answer 4

Optimized version

To my disappointment, my optimized function only runs 7x faster than the LINQ 'straightforward' version. Unoptimized LINQ 1m43s Optimized 14.7s.

• linux 32-bit
• Mono 2.11 (C# 4.0) compiler with -optimize+,
• 1,000,000 TESTITERATIONS
• VERBOSE not #define-d

What has been optimized:

• assume arrays for input and output
• work in-place on input array
• 'manually' analyze runs of identical values, instead of using GroupBy (using ValueRun struct)
• have these ValueRun structs in an Array instead of Enumerable (List); sort/shuffle in-place
• use unsafe blocks and pointers (no discernable difference...)
• use modulo indexing instead of MAGIC Linq code
• generate the output by iterative append instead of nested LINQ. This probably had the most effect. Actually, it would be even better when we could shortcut the ValueRuns that have a countruns collection is being ordered by this count, it seemed pretty easy to do; however, the transposed indexing (needed for the cyclic constraints) is complicating things. The gain of somehow applying this optimization anyway will be bigger with larger inputs and many unique values and some highly duplicated values.

Here is the code with optimized version. _Additional speed gain can be had by removing the seeding of RNGs; these are only there to make it possible to regression test the output.

[... old code removed as well ...]

---

Original response (partial)

If I'm getting you right, you are trying to devise a shuffle that prevents duplicates from ending up consecutive in the output (with a minimum interleave of 2 elements).

This is not solvable in the general case. Imagine an input of only identical elements :)

Update: Troubled state of affairs

Like I mention in the notes, I think I wasn't on the right track all the time. Either I should invoke graph theory (anyone?) or use a simple 'bruteforcey' algorithm instead, much a long Erick's suggestion.

Anyway, so you can see what I've been up to, and also what the problems are (enable the randomized samples to quickly see the problems):

#define OUTPUT       // to display the testcase results
#define VERIFY       // to selfcheck internals and verify results
#define SIMPLERANDOM
// #define DEBUG        // to really traces the internals
using System;
using System.Linq;
using System.Collections.Generic;

public static class Q5899274
{
    // TEST DRIVER CODE
    private const int TESTITERATIONS = 100000;
    public static int Main(string[] args)
    {
        var testcases = new [] {
            new [] {0,1,1,2,2,2,3,3,4,4,4,4,5,5,5,6,6,6,7,7,7,7,8,8,8,8,8,9,9,9,9,10},
            new [] {0,1,1,1,2,2,2,3,3,3,4,4,4,5,5,6,6,6,7,7,7,7,8,8,8,8,9,9,9,9,9,10},
            new [] { 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41, 42, 42, 42, },
            new [] {1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4},
        }.AsEnumerable();

        // // creating some very random testcases
        // testcases = Enumerable.Range(0, 10000).Select(nr => Enumerable.Range(GROUPWIDTH, _seeder.Next(GROUPWIDTH, 400)).Select(el => _seeder.Next(-40, 40)).ToArray());

        foreach (var testcase in testcases)
        {
            // _seeder = new Random(45); for (int i=0; i<TESTITERATIONS; i++) // for benchmarking/regression
            {
                try
                {
                    var output = TestOptimized(testcase);
#if OUTPUT
                    Console.WriteLine("spread\t{0}", string.Join(", ", output));
#endif
#if VERIFY
                    AssertValidOutput(output);
#endif
                } catch(Exception e)
                {
                    Console.Error.WriteLine("Exception for input {0}:", string.Join(", ", testcase));
                    Console.Error.WriteLine("Sequence length {0}: {1} groups and remainder {2}", testcase.Count(), (testcase.Count()+GROUPWIDTH-1)/GROUPWIDTH, testcase.Count() % GROUPWIDTH);
                    Console.Error.WriteLine("Analysis: \n\t{0}", string.Join("\n\t", InternalAnalyzeInputRuns(testcase)));
                    Console.Error.WriteLine(e);
                }
            }
        }

        return 0;
    }

#region Algorithm Core
    const int GROUPWIDTH = 3; /* implying a minimum distance of 2
                                 (GROUPWIDTH-1) values in between duplicates
                                 must be guaranteed*/

    public static T[] TestOptimized<T>(T[] input, bool doShuffle = false)
        where T: IComparable<T>
    {
        if (input.Length==0)
            return input;

        var runs = InternalAnalyzeInputRuns(input);
#if VERIFY
        CanBeSatisfied(input.Length, runs); // throws NoValidOrderingExists if not
#endif
        var transpositions = CreateTranspositionIndex(input.Length, runs);

        int pos = 0;
        for (int run=0; run<runs.Length; run++)
            for (int i=0; i<runs[run].runlength; i++)
                input[transpositions[pos++]] = runs[run].value;

        return input;
    }

    private static ValueRun<T>[] InternalAnalyzeInputRuns<T>(T[] input)
    {
        var listOfRuns = new List<ValueRun<T>>();
        Array.Sort(input);
        ValueRun<T> current = new ValueRun<T> { value = input[0], runlength = 1 };

        for (int i=1; i<=input.Length; i++)
        {
            if (i<input.Length && input[i].Equals(current.value))
                current.runlength++;
            else
            {
                listOfRuns.Add(current);
                if (i<input.Length)
                    current = new ValueRun<T> { value = input[i], runlength = 1 };
            }
        }

#if SIMPLERANDOM
        var rng = new Random(_seeder.Next());
        listOfRuns.ForEach(run => run.tag = rng.Next()); // this shuffles them
#endif
        var runs = listOfRuns.ToArray();
        Array.Sort(runs);

        return runs;
    }

    // NOTE: suboptimal performance 
    //   * some steps can be done inline with CreateTranspositionIndex for
    //   efficiency
    private class NoValidOrderingExists : Exception { public NoValidOrderingExists(string message) : base(message) { } }
    private static bool CanBeSatisfied<T>(int length, ValueRun<T>[] runs)
    {
        int groups = (length+GROUPWIDTH-1)/GROUPWIDTH;
        int remainder = length % GROUPWIDTH;

        // elementary checks
        if (length<GROUPWIDTH)
            throw new NoValidOrderingExists(string.Format("Input sequence shorter ({0}) than single group of {1})", length, GROUPWIDTH));
        if (runs.Length<GROUPWIDTH)
            throw new NoValidOrderingExists(string.Format("Insufficient distinct values ({0}) in input sequence to fill a single group of {1})", runs.Length, GROUPWIDTH));

        int effectivewidth = Math.Min(GROUPWIDTH, length);

        // check for a direct exhaustion by repeating a single value more than the available number of groups (for the relevant groupmember if there is a remainder group)
        for (int groupmember=0; groupmember<effectivewidth; groupmember++)
        {
            int capacity = remainder==0? groups : groups -1;

            if (capacity < runs[groupmember].runlength)
                throw new NoValidOrderingExists(string.Format("Capacity exceeded on groupmember index {0} with capacity of {1} elements, (runlength {2} in run of '{3}'))",
                            groupmember, capacity, runs[groupmember].runlength, runs[groupmember].value));
        }

        // with the above, no single ValueRun should be a problem; however, due
        // to space exhaustion duplicates could end up being squeezed into the
        // 'remainder' group, which could be an incomplete group; 
        // In particular, if the smallest ValueRun (tail) has a runlength>1
        // _and_ there is an imcomplete remainder group, there is a problem
        if (runs.Last().runlength>1 && (0!=remainder))
            throw new NoValidOrderingExists("Smallest ValueRun would spill into trailing incomplete group");

        return true;
    }

    // will also verify solvability of input sequence
    private static int[] CreateTranspositionIndex<T>(int length, ValueRun<T>[] runs)
        where T: IComparable<T>
    {
        int remainder = length % GROUPWIDTH;

        int effectivewidth = Math.Min(GROUPWIDTH, length);

        var transpositions = new int[length];
        {
            int outit = 0;
            for (int groupmember=0; groupmember<effectivewidth; groupmember++)
                for (int pos=groupmember; outit<length && pos<(length-remainder) /* avoid the remainder */; pos+=GROUPWIDTH)
                    transpositions[outit++] = pos;

            while (outit<length)
            {
                transpositions[outit] = outit;
                outit += 1;
            }

#if DEBUG
            int groups = (length+GROUPWIDTH-1)/GROUPWIDTH;
            Console.WriteLine("Natural transpositions ({1} elements in {0} groups, remainder {2}): ", groups, length, remainder);
            Console.WriteLine("\t{0}", string.Join(" ", transpositions));
            var sum1 = string.Join(":", Enumerable.Range(0, length));
            var sum2 = string.Join(":", transpositions.OrderBy(i=>i));
            if (sum1!=sum2)
                throw new ArgumentException("transpositions do not cover range\n\tsum1 = " + sum1 + "\n\tsum2 = " + sum2);
#endif
        }

        return transpositions;
    }

#endregion // Algorithm Core

#region Utilities
    private struct ValueRun<T> : IComparable<ValueRun<T>>
    {
        public T value;
        public int runlength;
        public int tag;         // set to random for shuffling

        public int CompareTo(ValueRun<T> other) { var res = other.runlength.CompareTo(runlength); return 0==res? tag.CompareTo(other.tag) : res; }
        public override string ToString() { return string.Format("[{0}x {1}]", runlength, value); }
    }

    private static /*readonly*/ Random _seeder = new Random(45);
#endregion // Utilities

#region Error detection/verification
    public static void AssertValidOutput<T>(IEnumerable<T> output)
        where T:IComparable<T>
    {
        var repl = output.Concat(output.Take(GROUPWIDTH)).ToArray();

        for (int i=1; i<repl.Length; i++) 
            for (int j=Math.Max(0, i-(GROUPWIDTH-1)); j<i; j++)
                if (repl[i].Equals(repl[j]))
                    throw new ArgumentException(String.Format("Improper duplicate distance found: (#{0};#{1}) out of {2}: value is '{3}'", j, i, output.Count(), repl[j]));
    }
#endregion

}