Is it possible to do CRC-32 calculation in splits?

Question

I use this trivial function to calculate the CRC checksum of a given file:

long i, j = 0;
int k = 0;
uint crc = 0xFFFFFFFF;
FileInfo file_info = new FileInfo(fil         


        

Answer 1

It is indeed possible to parallelize the CRC-32 computation, but the details are messy and I'd need to spend about a day to come up with the code.

Let's look at the basic CRC algorithm, where there is no negation and no bit reversal.

For the string of bytes that you want to compute the CRC of, let's call it the message. The basic idea is that you treat the message as a polynomial in GF(2), and you compute its remainder modulo the CRC polynomial.

The basic CRC algorithm is additive/linear. If you have two messages a and b of the same length, then CRC(a XOR b) = CRC(a) XOR CRC(b).

Furthermore, if you pad the message on the right side with n zeros, the new CRC will be the old CRC times x^n mod the CRC polynomial.

 

With all that said, the only way to solve your problem is to really understand the mathematics behind the CRC algorithm and write your own custom code. Here's a long but very complete explanation of CRCs: http://www.ross.net/crc/download/crc_v3.txt