Get MD5 Checksum for Very Large Files

Question

I\'ve written a script that reads through all files in a directory and returns md5 hash for each file. However, it renders nothing for a rather large file. I assume that t

Answer 1

If you're hitting an execution time limit or maximum execution time, PHP should be throwing an error message to that effect. Check your error logs. If you are hitting a limit, you can set the maximum values for PHP memory usage and execution time in your php.ini file:

memory_limit = 16M 

will set max memory usage to 16 megs. For maximum execution time:

max_execution_time = 30

will set maximum execution time to 30 seconds.

Answer 2

you could achieve it with command line

 shell_exec('md5sum -b '. $fileName);

Answer 3

FYI....in case someone needs a fast md5()check-sum. PHP is pretty fast even with the larger files. This returns the check-sum on Linux Mint .iso (size 880MB) in 3 sec.

<?php
// checksum
$path = $_SERVER['DOCUMENT_ROOT']; // get upload folder path
$file = $path."/somefolder/linux-mint.iso";  // any file
echo md5_file($file);
?>

Answer 4

Make sure to use escapeshellarg ( http://us3.php.net/manual/en/function.escapeshellarg.php ) if you decide to use a shell_exec() or system() call. I.e.,

shell_exec('md5sum -b ' . escapeshellarg($filename));

Answer 5

While i couldn't reproduce it with PHP 5.2 or 5.3 with a 2GB file the issue seems to come up on 32bit PHP builds.

Even so it's not a really nice solution you could try to let the system to the hasing

echo system("md5sum test.txt");

46d6a7bcbcf7ae0501da341cb3bae27c test.txt