PHP - how to read big remote files efficiently and use buffer in loop

Question

i would like to understand how to use the buffer of a read file.

Assuming we have a big file with a list of emails line by line ( delimiter is a classic \\n

Answer 1

Open the file with fopen() and read it incrementally. Probably one line at a time with fgets().

file_get_contents reads the whole file into memory, which is undesirable if the file is larger than a few megabytes

Depending on how long this takes, you may need to worry about the PHP execution time limit, or the browser timing out if it doesn't receive any output for 2 minutes.

Things you might try:

1. set_time_limit(0) to avoid running up against the PHP time limit
2. Make sure to output some data every 30 seconds or so so the browser doesn't time out; make sure to flush(); and possibly ob_flush(); so your output is actually sent over the network (this is a kludge)
3. start a separate process (e.g. via exec()) to run this in the background. Honestly, anything that takes more than a second or two is best run in the background