Convert integer to a random but deterministically repeatable choice

问题

How do I convert an unsigned integer (representing a user ID) to a random looking but actually a deterministically repeatable choice? The choice must be selected with equal probability (irrespective of the distribution of the the input integers). For example, if I have 3 choices, i.e. [0, 1, 2], the user ID 123 may always be randomly assigned choice 2, whereas the user ID 234 may always be assigned choice 1.

Cross-language and cross-platform algorithmic reproducibility is desirable. I'm inclined to use a hash function and modulo unless there is a better way. Here is what I have:

>>> num_choices = 3
>>> id_num = 123
>>> int(hashlib.sha256(str(id_num).encode()).hexdigest(), 16) % num_choices
2

I'm using the latest stable Python 3. Please note that this question is similar but not exactly identical to the related question to convert a string to random but deterministically repeatable uniform probability.

回答1:

Using hash and modulo

import hashlib

def id_to_choice(id_num, num_choices):
    id_bytes = id_num.to_bytes((id_num.bit_length() + 7) // 8, 'big')
    id_hash = hashlib.sha512(id_bytes)
    id_hash_int = int.from_bytes(id_hash.digest(), 'big')  # Uses explicit byteorder for system-agnostic reproducibility
    choice = id_hash_int % num_choices  # Use with small num_choices only
    return choice

>>> id_to_choice(123, 3)
0
>>> id_to_choice(456, 3)
1

Notes:

The built-in hash method must not be used because it can preserve the input's distribution, e.g. with hash(123). Alternatively, it can return values that differ when Python is restarted, e.g. with hash('123').
For converting an int to bytes, bytes(id_num) works but is grossly inefficient as it returns an array of null bytes, and so it must not be used. Using int.to_bytes is better. Using str(id_num).encode() works but wastes a few bytes.
Admittedly, using modulo doesn't offer exactly uniform probability,[1][2] but this shouldn't bias much for this application because id_hash_int is expected to be very large and num_choices is assumed to be small.

Using random

The random module can be used with id_num as its seed, while addressing concerns surrounding both thread safety and continuity. Using randrange in this manner is comparable to and simpler than hashing the seed and taking modulo.

With this approach, not only is cross-language reproducibility a concern, but reproducibility across multiple future versions of Python could also be a concern. It is therefore not recommended.

import random

def id_to_choice(id_num, num_choices):
    localrandom = random.Random(id_num)
    choice = localrandom.randrange(num_choices)
    return choice

>>> id_to_choice(123, 3)
0
>>> id_to_choice(456, 3)
2

回答2:

An alternative is to encrypt the user ID. If you keep the encryption key the same, then each input number will encrypt to a different output number up to the block size of the cipher you use. DES uses 64 bit blocks which cover IDs 000000 to 18446744073709551615. That will give a random appearing replacement for the user ID, which is guaranteed not to give two different user IDs the same 'random' number because encryption is a one-to-one permutation of the block values.

回答3:

I apologize I don't have Python implementation but I do have very clear, readable and self evident implementation in Java which should be easy to translate into Python with minimal effort. The following produce long predictable evenly distributed sequences covering all range except zero

XorShift ( http://www.arklyffe.com/main/2010/08/29/xorshift-pseudorandom-number-generator )

public int nextQuickInt(int number) {
    number ^= number << 11;
    number ^= number >>> 7;
    number ^= number << 16;
    return number;
}

public short nextQuickShort(short number) {
    number ^= number << 11;
    number ^= number >>> 5;
    number ^= number << 3;
    return number;
}

public long nextQuickLong(long number) {
    number ^= number << 21;
    number ^= number >>> 35;
    number ^= number << 4;
    return number;
}

or XorShift128Plus (need to re-seed state0 and state1 to non-zero values before using, http://xoroshiro.di.unimi.it/xorshift128plus.c)

public class XorShift128Plus {

private long state0, state1; // One of these shouldn't be zero

public long nextLong() {
    long state1 = this.state0;
    long state0 = this.state0 = this.state1;
    state1 ^= state1 << 23;
    return (this.state1 = state1 ^ state0 ^ (state1 >> 18) ^ (state0 >> 5)) + state0;
}

public void reseed(...) {
    this.state0 = ...;
    this.state1 = ...;
}

}

or XorOshiro128Plus (http://xoroshiro.di.unimi.it/)

public class XorOshiro128Plus {

private long state0, state1; // One of these shouldn't be zero

public long nextLong() {
    long state0 = this.state0;
    long state1 = this.state1;
    long result = state0 + state1;
    state1 ^= state0;
    this.state0 = Long.rotateLeft(state0, 55) ^ state1 ^ (state1 << 14);
    this.state1 = Long.rotateLeft(state1, 36);
    return result;
}

public void reseed() {

}

}

or SplitMix64 (http://xoroshiro.di.unimi.it/splitmix64.c)

public class SplitMix64 {

private long state;

public long nextLong() {
    long result = (state += 0x9E3779B97F4A7C15L);
    result = (result ^ (result >> 30)) * 0xBF58476D1CE4E5B9L;
    result = (result ^ (result >> 27)) * 0x94D049BB133111EBL;
    return result ^ (result >> 31);
}

public void reseed() {
    this.state = ...;
}
}

or XorShift1024Mult (http://xoroshiro.di.unimi.it/xorshift1024star.c) or Pcg64_32 (http://www.pcg-random.org/, http://www.pcg-random.org/download.html)

回答4:

The simplest method is to modulo user_id by number of options:

choice = user_id % number_of_options

It's very easy and fast. However if you know user_id's you may to guess an algorithm.

Also, pseudorandom sequences can be obtained from random seeded with user constants (e.g. user_id):

>>> import random
>>> def generate_random_value(user_id):
...     random.seed(user_id)
...     return random.randint(1, 10000)
...
>>> [generate_random_value(x) for x in range(20)]
[6312, 2202, 927, 3899, 3868, 4186, 9402, 5306, 3715, 7586, 9362, 7412, 7776, 4244, 1751, 3424, 5924, 8553, 2970, 709]
>>> [generate_random_value(x) for x in range(20)]
[6312, 2202, 927, 3899, 3868, 4186, 9402, 5306, 3715, 7586, 9362, 7412, 7776, 4244, 1751, 3424, 5924, 8553, 2970, 709]
>>>

来源：https://stackoverflow.com/questions/41255711/convert-integer-to-a-random-but-deterministically-repeatable-choice

标签

python

random

hashlib