random number generator test

Question

How will you test if the random number generator is generating actual random numbers?

My Approach: Firstly build a hash of size M, where M is the prime number. Then

Answer 1

You should be aware that you cannot guarantee the random number generator is working properly. Note that even a perfect uniform distribution in range [1,10] - there is a 10^-10 chance of getting 10 times 10 in a random sampling of 10 numbers.

Is it likely? Of course not.

So - what can we do?

We can statistically prove that the combination (10,10,....,10) is unlikely if the random number generator is indeed uniformly distributed. This concept is called Hypothesis testing. With this approach we can say "with certainty level of x% - we can reject the hypothesis that the data is taken from a uniform distribution".

A common way to do it, is using Pearson's Chi-Squared test, The idea is similar to yours - you fill in a table - check what is the observed (generated) number of numbers for each cell, and what is the expected number of numbers for each cell under the null hypothesis (in your case, the expected is k/M - where M is the range's size, and k is the total number of numbers taken).
You then do some manipulation on the data (see the wikipedia article for more info what this manipulation is exactly) - and get a number (the test statistic). You then check if this number is likely to be taken from a Chi-Square Distribution. If it is - you cannot reject the null hypothesis, if it is not - you can be certain with x% certainty that the data is not taken from a uniform random generator.

EDIT: example:
You have a cube, and you want to check if it is "fair" (uniformly distributed in [1,6]). Throw it 200 times (for example) and create the following table:

number:                1       2         3         4          5          6
empirical occurances: 37       41        30        27         32         33
expected occurances: 33.3      33.3      33.3      33.3       33.3       33.3

Now, according to Pearson's test, the statistic is:

X = ((37-33.3)^2)/33.3 + ((41-33.3)^2)/33.3 + ... + ((33-33.3)^2)/33.3 
X = (18.49 + 59.29 + 10.89 + 39.69 + 1.69 + 0.09) / 33.3
X = 3.9

For a random C~ChiSquare(5), the probability of being higher then 3.9 is ~0.45 (which is not improbable)¹.

So we cannot reject the null hypothesis, and we can conclude that the data is probably uniformly distributed in [1,6]

---

(1) We usually reject the null hypothesis if the value is smaller then 0.05, but this is very case dependent.

Answer 2

Let's say you want to generate a uniform distribution on the interval [0, 1].

Then one possible test is

for i from 1 to sample-size
when a < random-being-tested() < b
counter +1
return counter/sample-size

And see if the result is closed to b-a (b minus a).

Of course you should define a function taking a, b between 0 and 1 as inputs, and return the difference between the counter/sample-size and b-a. Loop through possible a, b, say of the multiples of 0.01, a < b. Print out a, b when the difference is larger than a preset epsilon, say 0.001.

Those are the a, b for which there are too many outliers.

If you let sample-size be 5000. Your random-being-tested will be called about 5000 * 5050 times in total, hopefully not too bad.

Answer 3

I had the same problem. when I finish to write my code (using an external RNG engine)

I looked on the results and found that all of them fail Chi-Square test whenever I have to many results.

my code generated a random number and hold buckets of the amount of each result range. I don't know why the Chi-square test fail when i have a lot of results.

during my research I saw that the C# Random.next() fail in any range of random and that some of the numbers have better odds than the other, further more i saw that the RNGCryptoServiceProvider random provider is not supporting good on big numbers.

when trying to get numbers in the range of 0-1,000,000,000 the numbers in the lower range 0-300M have better odds to appear...

as a result I'm using the RNGCryptoServiceProvider and if my range is higher than 100M i'm combine the number my self (RandomHigh*100M + RandomLow) and the ranges of both randoms is smaller than 100M so it good.

Good Luck!

Answer 4

My naive idea:
The generator is following a distribution. (At least it should.) Do a reasonable amount of runs then plot the values on a graph. Fit a regression curve on the points. If it correlates with the shape of the distribution you're good. (This is also possible in 1D with projections and histograms. And fully automatable with the correct tool, e.g. MatLab)
You can also use the diehard tests as it was mentioned before, that is surely better but involves much less intuition, at least on your side.