Calculate/validate bz2 (bzip2) CRC32 in Python

Question

I\'m trying to calculate/validate the CRC32 checksums for compressed bzip2 archives.

.magic:16                       = \'BZ\' signature/magic number
.version:8           


        

Answer 1

The following is the CRC algorithm used by bzip2, written in Python:

crcVar = 0xffffffff # Init
    for cha in list(dataIn):
        crcVar = crcVar & 0xffffffff # Unsigned
        crcVar = ((crcVar << 8) ^ (BZ2_crc32Table[(crcVar >> 24) ^ (ord(cha))]))

    return hex(~crcVar & 0xffffffff)[2:-1].upper()

(C code definitions can be found on lines 155-172 in bzlib_private.h)

BZ2_crc32Table array/list can be found in crctable.c from the bzip2 source code. This CRC checksum algorithm is, quoting: "..vaguely derived from code by Rob Warnock, in Section 51 of the comp.compression FAQ..." (crctable.c)

The checksums are calculated over the uncompressed data.

Sources can be downloaded here: http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz