Hash Collision - what are the chances? [closed]

Answer 1

Use a symmetric encryption scheme and a private server key to encrypt the ID (and other values) when you send them to the client and decrypt again on reception. Take care that your cryptographic function provides both confidentiality and integrity checks.

This allows you to use sensible values when talking to the DB without any collision, great security when talking to the client and reduces your probability to land on thedailyWTF by approximatly 2^160.

See also Pounding A Nail: Old Shoe or Glass Bottle?!

Answer 2

What are the chances of a collision?

I see not accurate answer to this specific question, so here I provide the probability of collision for some Nº of records:

• Nº of entries: 2^54 Probability of collision: 1e-16
• Nº of entries: 2^64 Probability of collision: 1e-10
• Nª of entries: 2^71 Probability of collision: 2e-06
• Nª of entries: 2^76 Probability of collision: 2e-03
• Nª of entries: 2^80 Probability of collision: 0.39
• Nª of entries: 2^82 Probability of collision: >0.99

Interpretation examples:

• If you have less than 2^54 (18,014,398,509,481,984) records you can be as certain as anyone can be that you won't have a collision.
• If you somehow manage to have 2^76 (75,557,863,725,914,323,419,136) records you can only be somewhat sure (there are 2 in a million chances that you may have a collision!)

Now you can decide for yourself if the probability chances are good enough for you.

Answer 3

If you assume that SHA-1 does a good job, you can conclude that there's a 1 in 2^160 chance that two given messages have the same hash (since SHA-1 produces a 160-bit hash).

2^160 is a ridiculously large number. It's roughly 10^48. Even if you have a million entries in your database, that's still a 1 in 10^42 chance that a new entry will share the same hash.

SHA-1 has proved to be fairly good, so I don't think you need to worry about collisions at all.

As a side note, use PHP's raw_output feature when you use SHA-1 as this will lead to a shorter string and hence will make your database operations a bit faster.

EDIT: To address the birthday paradox, a database with 10^18 (a million million million) entries has a chance of about 1 in 0.0000000000003 of a collision. Really not worth worrying about.

Answer 4

Ask the question what will it cost you if there is a collision. If this is a free site fine. If you are running a money making business and an overrite will cost you a million dollar contract then I would think again.

I think you are going about this the wrong way.
I think you need to keep the unique ID but you want to make sure that the users can not manually change the ID.

One way to to do this is to put the ID and the hash of the ID (with some extra data) in the link.

For Example: (my PHP is rusty so general algorithm would be:)

id   = 5;
hash = hash("My Private String " + id)
link = "http://mySite.com/resource?id=" + id + "&hash=" + hash

Then when you receive a request just validate that you can regenerate the hash from ID. This does leave you open to an attack to work out "My Private String" but that will be quite computationally hard and you could always append something else unique that is not directly available to the user (like the session ID).

Answer 5

The other comments have covered you on the probabilities, however if you look at this pragmatically then you can get a definite answer for yourself.

You said yourself that you are going to be hashing your sequential IDs. It would be easy to code up a test case. Iterate through ~100,000,000 ids and check for collisions. That would not take long to do. On the other hand you might run out of memory quarter of the way through.

Answer 6

From first principles:

SHA-1 produces a 160-bit digest. Assuming it uses the entire bit-space evenly (which is presumably what it was designed to do), that is only a 2^-160 chance on each insert that you would get a collision.

So for each insert, it should be safe to assume there is no collision, and deal with the error if there is.

That is not to say that you can ignore the chance of collision entirely.

The Birthday Paradox suggests the chance of there being at least one collision in your database is higher than you would guess, because of the O(N^2) possible collisions.