preg_match appears to hit a limit when using two matches

Question

I have run up against an odd problem. it appears i am reaching some sort of limit with preg_replace while trying to use two matches using php-5.3.3

// works         


        

Answer 1

Ok, PHP's not very talkative about regex errors, it just returns false for the last case, which simply tells than an error occured, per the PHP docs.

I've reproduced the problem using PCRE (the regex engine used by preg_match) in C# (but with a much higher character count), and the error I'm getting is PCRE_ERROR_MATCHLIMIT.

This means you're hitting the backtracking limit set in PCRE. It's just a safety measure to prevent the engine from looping indefinitely, and I think your PHP configuration sets it to a low value.

To fix the issue, you can set a higher value for the pcre.backtrack_limit PHP option which controls this limit:

ini_set("pcre.backtrack_limit", "10000000"); // Actually, this is PCRE's default

On a side note:

• You probably should replace (.*) with (.*?) to get less useless backtracking and for correctness (otherwise the regex engine will get past the STOP string and will have to backtrack to reach it)
• Using ? as a pattern delimiter is a bad idea since it prevents you from using the ? metacharacter and therefore applying the above advice. Really, you should never use regex metacharacters as pattern delimiters.

---

If you're interested in more low-level details, here's the relevant bit of the PCRE docs (emphasis mine):

The match_limit field provides a means of preventing PCRE from using up a vast amount of resources when running patterns that are not going to match, but which have a very large number of possibilities in their search trees. The classic example is a pattern that uses nested unlimited repeats.

Internally, pcre_exec() uses a function called match(), which it calls repeatedly (sometimes recursively). The limit set by match_limit is imposed on the number of times this function is called during a match, which has the effect of limiting the amount of backtracking that can take place. For patterns that are not anchored, the count restarts from zero for each position in the subject string.

When pcre_exec() is called with a pattern that was successfully studied with a JIT option, the way that the matching is executed is entirely different. However, there is still the possibility of runaway matching that goes on for a very long time, and so the match_limit value is also used in this case (but in a different way) to limit how long the matching can continue.

The default value for the limit can be set when PCRE is built; the default default is 10 million, which handles all but the most extreme cases. You can override the default by suppling pcre_exec() with a pcre_extra block in which match_limit is set, and PCRE_EXTRA_MATCH_LIMIT is set in the flags field. If the limit is exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT.

A value for the match limit may also be supplied by an item at the start of a pattern of the form

 (*LIMIT_MATCH=d)

where d is a decimal number. However, such a setting is ignored unless d is less than the limit set by the caller of pcre_exec() or, if no such limit is set, less than the default.

Answer 2

About PHP not being talkative about its errors, you can use T-Regx library, which always throws an exception:

// didnt work
$pattern_2 = '/START-ONE(.*)STOP-ONE.*START-TWO(.*)STOP-TWO.*/';
$string = 'START-ONE this is head stuff STOP-ONE  START-TWO' . str_repeat('x', 959971) . 'STOP-TWO';

try {
    pattern($pattern_2)->match($string)->first();
}
catch ($e) {
    $m = $e->getMessage();
    $m // After invoking preg_match(), preg_last_error() returned PREG_BACKTRACK_LIMIT_ERROR.
}