Laravel - Collection with relations take a lot of time

半世苍凉 提交于 2019-12-04 20:58:41

--First Question--

One of the issues you might be facing is having all those huge amount of data in memory, i.e:

$timeLog = TimeLog::get();

This is already enormous. Then when you are trying to convert the collection to array:

  1. There is a loop through the collection.
  2. Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
  3. Thousands of queries are made to retrieve the related models

So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:

  1. Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:

    $timeLog = TimeLog::chunk(1000, function($logs){
        foreach ($logs as $log) {
        // Do the stuff here
        }
    }); 
    
  2. Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.

    foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
      //do the other stuffs here
    }
    
  3. This looks like the first but instead you have already narrowed your query down to what you need:

    TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
    
  4. Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)

    TimeLog::with(['board','task'],  function ($query) {
        $query->where([['board_id','>',0],['task_id', '>', 0]]);
    }])->get();
    
  5. You can simply use Transformer

    • With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.

Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.

--Second question--

Based on Laravel API here You could see that:

It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.

UPDATE

For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:

return TimeLog::paginate(1000);

I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.

Short example:

$db = DB::connection()->getPdo();

$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
                            ->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
                            ->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
                            ->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
                            ->toSql();

$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
 while ($log = $query->fetch()) {
   $log_filled = new TimeLog();
   //fill your model and push it into an array to parse it to json in future
   array_push($logs,$log_filled);
}
return response()->json($logs);
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!