How to chunk results from a custom query in Laravel

Question

I have a custom query that grabs data from the old system and maps it to models in the new system. The query looks like this:

$companies = DB::connection(\'leg

Answer 1

A lot of answers on this page do a lookup of all records in order to workout how many 'pages' there are. This can be slow and is not needed as we are not paginating we are chunking. We only need to know the total number of pages when we are paginating to show the user.

So an alternative to getting a count of all records at the start is to do the following:

    $recordsRemaining = true;
    $lookupsCompleted = 0;
    $chunkSize = 200;

    while($recordsRemaining){
       $companies = DB::connection('legacy')->select("...")->skip($chunkSize*$lookupsCompleted)->take($chunkSize)->get();

       if($legacy->count() < $chunkSize){
          $recordsRemaining = false;
       }
       foreach($companies as $company){
          //Do something
       }

       $lookupsCompleted++;
    }

This does exactly the same as the accepted answer but is more efficient.

Answer 2

deyes's answer has a bug.

AS-IS

if 'legacy' table has 'id' column and there's 1,2,3,4,5 ... 100 numbered data.

<?php

$max = 10;
$total = DB::connection('legacy')->select("...")->count();
$pages = ceil($total / $max);
for ($i = 1; $i < ($pages + 1); $i++) {
    $offset = (($i - 1)  * $max);
    $start = ($offset == 0 ? 0 : ($offset + 1));
    $legacy = DB::connection('legacy')->select("...")->skip($start)->take($max)->get();
    /* Do stuff. */

    $legacyIds = $legacy->lists("id");
    echo "i = " . $i . ": \n";
    print_r($legacyIds);
}

//Result
i = 1: 
Array
(
    [0] => 1
    [1] => 2
    [2] => 3
    [3] => 4
    [4] => 5
    [5] => 6
    [6] => 7
    [7] => 8
    [8] => 9
    [9] => 10
)
i = 2: 
Array
(
    [0] => 12
    [1] => 13
    [2] => 14
    [3] => 15
    [4] => 16
    [5] => 17
    [6] => 18
    [7] => 19
    [8] => 20
    [9] => 21
) ...

TO-DO

$max = 10;
$total = DB::connection('legacy')->select("...")->count();
$pages = ceil($total / $max);
for ($i = 1; $i < ($pages + 1); $i++) {
    $offset = (($i - 1)  * $max);
    $legacy = DB::connection('legacy')->select("...")->skip($offset)->take($max)->get();
    /* Do stuff. */
}