Performance issue with generation of random unique numbers

Question

I have a situation where by I need to create tens of thousands of unique numbers. However these numbers must be 9 digits and cannot contain any 0\'s. My current approach is

Answer 1

Something like this?

public List<string> generateIdentifiers2(int quantity)
        {
            var uniqueIdentifiers = new List<string>(quantity);
            while (uniqueIdentifiers.Count < quantity)
            {
                var sb = new StringBuilder();
                sb.Append(random.Next(11, 100));
                sb.Append(" ");
                sb.Append(random.Next(11, 100));
                sb.Append(" ");
                sb.Append(random.Next(11, 100));

                var id = sb.ToString();
                id = new string(id.ToList().ConvertAll(x => x == '0' ? char.Parse(random.Next(1, 10).ToString()) : x).ToArray());

                if (!uniqueIdentifiers.Contains(id))
                {
                    uniqueIdentifiers.Add(id);
                }
            }
            return uniqueIdentifiers;
        }

Answer 2

I think @slugster is broadly right - although you could run two parallel processes, one to generate numbers, the other to verify them and add them to the list of accepted numbers when verified. Once you have enough, signal the original process to stop.

Combine this with other suggestions - using more efficient and appropriate data structures - and you should have something that works acceptably.

However the question of why you need such numbers is also significant - this requirement seems like one that should be analysed.

Answer 3

The trick here is that you only need ten thousand unique numbers. Theoretically you could have almost 9,0E+08 possibilities, but why care if you need so many less?

Once you realize that you can cut down on the combinations that much then creating enough unique numbers is easy:

long[] numbers = { 1, 3, 5, 7 }; //note that we just take a few numbers, enough to create the number of combinations we might need
var list = (from i0 in numbers
            from i1 in numbers
            from i2 in numbers
            from i3 in numbers
            from i4 in numbers
            from i5 in numbers
            from i6 in numbers
            from i7 in numbers
            from i8 in numbers
            from i9 in numbers
            select i0 + i1 * 10 + i2 * 100 + i3 * 1000 + i4 * 10000 + i5 * 100000 + i6 * 1000000 + i7 * 10000000 + i8 * 100000000 + i9 * 1000000000).ToList();

This snippet creates a list of more than a 1,000,000 valid unique numbers pretty much instantly.

Answer 4

This suggestion may or may not be popular.... it depends on people's perspective. Because you haven't been too specific about what you need them for, how often, or the exact number, I will suggest a brute force approach.

I would generate a hundred thousand numbers - shouldn't take very long at all, maybe a few seconds? Then use Parallel LINQ to do a Distinct() on them to eliminate duplicates. Then use another PLINQ query to run a regex against the remainder to eliminate any with zeroes in them. Then take the top x thousand. (PLINQ is brilliant for ripping through large tasks like this). If needed, rinse and repeat until you have enough for your needs.

On a decent machine it will just about take you longer to write this simple function than it will take to run it. I would also query why you have 400K entries to test when you state you actually need "tens of thousands"?

Answer 5

Try avoiding checks making sure that you always pick up a unique number:

static char[] base9 = "123456789".ToCharArray();

static string ConvertToBase9(int value) {
    int num = 9;
    char[] result = new char[9];
    for (int i = 8; i >= 0; --i) { 
        result[i] = base9[value % num];
        value = value / num;
    }
    return new string(result);
}

public static void generateIdentifiers(int quantity) {
    var uniqueIdentifiers = new List<string>(quantity);
    // we have 387420489 (9^9) possible numbers of 9 digits in base 9.
    // if we choose a number that is prime to that we can easily get always
    // unique numbers
    Random random = new Random();
    int inc = 386000000;
    int seed = random.Next(0, 387420489);
    while (uniqueIdentifiers.Count < quantity) {
        uniqueIdentifiers.Add(ConvertToBase9(seed));
        seed += inc;
        seed %= 387420489;
    }
}

I'll try to explain the idea behind with small numbers...

Suppose you have at most 7 possible combinations. We choose a number that is prime to 7, e.g. 3, and a random starting number, e.g. 4.

At each round, we add 3 to our current number, and then we take the result modulo 7, so we get this sequence:

4 -> 4 + 3 % 7 = 0
0 -> 0 + 3 % 7 = 3
3 -> 3 + 3 % 7 = 6
6 -> 6 + 6 % 7 = 5

In this way, we generate all the values from 0 to 6 in a non-consecutive way. In my example, we are doing the same, but we have 9^9 possible combinations, and as a number prime to that I choose 386000000 (you just have to avoid multiples of 3).

Then, I pick up the number in the sequence and I convert it to base 9.

I hope this is clear :)

I tested it on my machine, and generating 400k unique values took ~ 1 second.

Answer 6

use string array or stringbuilder, wjile working with string additions.

more over, your code is not efficient because after generating many id's your list may hold new generated id, so that the while loop will run more than you need.

use for loops and generate your id's from this loop without randomizing. if random id's are required, use again for loops and generate more than you need and give an generation interval, and selected from this list randomly how much you need.

use the code below to have a static list and fill it at starting your program. i will add later a second code to generate random id list. [i'm a little busy]

    public static Random RANDOM = new Random();
    public static List<int> randomNumbers = new List<int>();
    public static List<string> randomStrings = new List<string>();

    private void fillRandomNumbers()
    {
        int i = 100;
        while (i < 1000)
        {
            if (i.ToString().Contains('0') == false)
            {
                randomNumbers.Add(i);
            }
        }
    }