debugging long running PHP script

Question

I have php script running as a cron job, extensively using third party code. Script itself has a few thousands LOC. Basically it\'s the data import / treatment script. (JSON to

Answer 1

There are three things that come to mind:

1. Set up an IDE so you can debug the PHP script line by line
2. Add some logging to the script
3. Look for long running queries in MySQL

Debug option #2 is the easiest. Since this is running as a cron job, you add a bunch of echo's in your script:

<?php

function log_message($type, $message) {
    echo "[{strtoupper($type)}, {date('d-m-Y H:i:s')}] $message";
}

log_message('info', 'Import script started');

// ... the rest of your script

log_message('info', 'Import script finished');

Then pipe stdout to a log file in the cron job command.

01 04 * * * php /path/to/script.php >> /path/to/script.log

Now you can add log_message('info|warn|debug|error', 'Message here') all over the script and at least get an idea of where the performance issue lies.

Debug option #3 is just straight investigation work in MySQL. One of your queries might be taking a long time, and it might show up in a long running query utility for MySQL.

Answer 2

Okay, basically you have two possibilities - it's either the ineffective PHP code or ineffective MySQL code. Judging by what you say, it's probably inserting into indexed table a lot of records separately, which causes the insertion time to skyrocket. You should either disable indexes and rebuild them after insertion, or optimize the insertion code.

But, about the tools.

You can configure the system to automatically log slow MySQL queries: https://dev.mysql.com/doc/refman/5.1/en/slow-query-log.html

You can also do the same with PHP scripts, but you need a PHP-FPM environment (and you probably have Apache). https://rtcamp.com/tutorials/php/fpm-slow-log/

These tools are very powerful and versatile.

P.S. 10-20 minutes for 100 records seems like A LOT.

Answer 3

I have a similar thing running each night (a cron job to update my database). I have found the most reliable way to debug is to set up a log table in the database and regularly insert / update a json string containing a multi-dimensional array with info about each record and whatever useful info you want to know about each record. This way if your cron job does not finish you still have detailed information about where it got up to and what happened along the way. Then you can write a simple page to pull out the json string, turn it back into an array and print useful data onto the page including timing and passed tests etc. When you see something as an issue you can concentrate on putting more info from that area into the json string.

Answer 4

Yes its possible and You can use Kint (PHP Debugging Script)

What is it? Kint for PHP is a tool designed to present your debugging data in the absolutely best way possible.

In other words, it's var_dump() and debug_backtrace() on steroids. Easy to use, but powerful and customizable. An essential addition to your development toolbox.

Still lost? You use it to see what's inside variables.

Act as a debug_backtrace replacer, too

you can download here or Here

Total Documentations and Help is here

Plus, it also supports almost all php framework

• CodeIgniter
• Drupal
• Symfony
• Symfony 2
• WordPress
• Yii
• framework
• Zend Framework

All the Best.... :)

Answer 5

xdebug_print_function_stack is an option, but what you can also do is to create a "function trace".There are three output formats. One is meant as a human readable trace, another one is more suited for computer programs as it is easier to parse, and the last one uses HTML for formatting the trace

http://www.xdebug.org/docs/execution_trace

Answer 6

You can use https://github.com/jmartin82/phplapse to record the application activity for determinate time.

For example start recording after n iterations with:

phplapse_start();

And stop it in next iteration with:

phplapse_stop();

With this process you was created a snapshot of execution when all seems works slow.

(I'm the author of project, don't hesitate to contact with me to improve the functionality)