Export large CSV file using laravel

问题

I have to export 600k database rows with a belongsTo relationship between two tables to a csv file using a queued job in laravel 5.2, it works fine when I do not include the information from the second table, but it reaches memory limit or timeout I guess (there is no error on the output, the job just stops and the file blocks at ~8MB) when I include the second table.

Here is my handle function from my job :

public function handle()
{
    $request = $this->request;
    $ts = Carbon::now()->timestamp; 
    $file = "/path/file-" . $ts . ".csv";
    $csv = CSV::createFromPath(public_path() . $file, "w");
    $csv->insertOne(\Schema::getColumnListing('games'));


        $queryBuilder = Game::query();
        if(!empty($request['host'])) $queryBuilder->searchHost($request['host']);
        if(!empty($request['customer_code'])) $queryBuilder->searchCustomerCode($request['customer_code']);
        if(!empty($request['customer_first_name'])) $queryBuilder->searchCustomerFirstName($request['customer_first_name']);
        if(!empty($request['customer_last_name'])) $queryBuilder->searchCustomerLastName($request['customer_last_name']);
        if(!empty($request['check_1'])) $queryBuilder->searchCheckOne($request['check_1']);
        if(!empty($request['check_2'])) $queryBuilder->searchCheckTwo($request['check_2']);
        if(!empty($request['check_3'])) $queryBuilder->searchCheckThree($request['check_3']);
        if(!empty($request['check_4'])) $queryBuilder->searchCheckFour($request['check_4']);
        if(!empty($request['date_sync'])) $queryBuilder->SearchDate($request['date_sync']);     
        if(!empty($request['place'])) $queryBuilder->SearchPlace($request['place']);            

        $queryBuilder->chunk(100, function($games) use($request, $csv){
            foreach ($games as $game) {

                    $csv->insertOne($game->toArray()); 

            }
        });
        event(new GameFileReadyEvent($file));
}

This code works fine, but in the foreach loop, if I do

 $queryBuilder->chunk(100, function($games) use($request, $csv){
            foreach ($games as $game) {

                    $csv->insertOne($game->id, $game->host->id, $game->check_first...); 

            }
        });

it seems to block somewhere and I don't know why.

I am using league/csv to do the export job, laravel 5.2 and a homestead box for development, and an Ubuntu 16.04 server for production.

来源：https://stackoverflow.com/questions/40959308/export-large-csv-file-using-laravel

标签

php

mysql

laravel

csv

timeout