How does {m}{n} (“exactly n times” twice) work?

Question

So, some way or another (playing around), I found myself with a regex like \\d{1}{2}.

Logically, to me, it should mean:

(A digit exac

Answer 1

Compiled structure of the regex

Kobi's answer is spot on about the behavior of Java regex (Sun/Oracle implementation) for the case "^\\d{1}{2}$", or "{1}".

Below is the internal compiled structure of "^\\d{1}{2}$":

^\d{1}{2}$
Begin. \A or default ^
Curly. Greedy quantifier {1,1}
  Ctype. POSIX (US-ASCII): DIGIT
  Node. Accept match
Curly. Greedy quantifier {2,2}
  Slice. (length=0)

  Node. Accept match
Dollar(multiline=false). \Z or default $
java.util.regex.Pattern$LastNode
Node. Accept match

Looking at the source code

From my investigation, the bug is probably due to that fact that { is not properly checked in the private method sequence().

The method sequence() calls to the atom() to parse the atom, then attach quantifier to the atom by calling closure(), and chains all atoms-with-closure together into one sequence.

For example, given this regex:

^\d{4}a(bc|gh)+d*$

Then the top-level call to sequence() will receive the compiled nodes for ^, \d{4}, a, (bc|gh)+, d*, $ and chain them together.

With that idea in mind, let us look at the source code of sequence(), copied from OpenJDK 8-b132 (Oracle uses the same code base):

@SuppressWarnings("fallthrough")
/**
 * Parsing of sequences between alternations.
 */
private Node sequence(Node end) {
    Node head = null;
    Node tail = null;
    Node node = null;
LOOP:
    for (;;) {
        int ch = peek();
        switch (ch) {
        case '(':
            // Because group handles its own closure,
            // we need to treat it differently
            node = group0();
            // Check for comment or flag group
            if (node == null)
                continue;
            if (head == null)
                head = node;
            else
                tail.next = node;
            // Double return: Tail was returned in root
            tail = root;
            continue;
        case '[':
            node = clazz(true);
            break;
        case '\\':
            ch = nextEscaped();
            if (ch == 'p' || ch == 'P') {
                boolean oneLetter = true;
                boolean comp = (ch == 'P');
                ch = next(); // Consume { if present
                if (ch != '{') {
                    unread();
                } else {
                    oneLetter = false;
                }
                node = family(oneLetter, comp);
            } else {
                unread();
                node = atom();
            }
            break;
        case '^':
            next();
            if (has(MULTILINE)) {
                if (has(UNIX_LINES))
                    node = new UnixCaret();
                else
                    node = new Caret();
            } else {
                node = new Begin();
            }
            break;
        case '$':
            next();
            if (has(UNIX_LINES))
                node = new UnixDollar(has(MULTILINE));
            else
                node = new Dollar(has(MULTILINE));
            break;
        case '.':
            next();
            if (has(DOTALL)) {
                node = new All();
            } else {
                if (has(UNIX_LINES))
                    node = new UnixDot();
                else {
                    node = new Dot();
                }
            }
            break;
        case '|':
        case ')':
            break LOOP;
        case ']': // Now interpreting dangling ] and } as literals
        case '}':
            node = atom();
            break;
        case '?':
        case '*':
        case '+':
            next();
            throw error("Dangling meta character '" + ((char)ch) + "'");
        case 0:
            if (cursor >= patternLength) {
                break LOOP;
            }
            // Fall through
        default:
            node = atom();
            break;
        }

        node = closure(node);

        if (head == null) {
            head = tail = node;
        } else {
            tail.next = node;
            tail = node;
        }
    }
    if (head == null) {
        return end;
    }
    tail.next = end;
    root = tail;      //double return
    return head;
}

Take note of the line throw error("Dangling meta character '" + ((char)ch) + "'");. This is where the error is thrown if +, *, ? are dangling and is not part of a preceding token. As you can see, { is not among the cases to throw error. In fact, it is not present in the list of cases in sequence(), and the compilation process will go by default case directly to atom().

@SuppressWarnings("fallthrough")
/**
 * Parse and add a new Single or Slice.
 */
private Node atom() {
    int first = 0;
    int prev = -1;
    boolean hasSupplementary = false;
    int ch = peek();
    for (;;) {
        switch (ch) {
        case '*':
        case '+':
        case '?':
        case '{':
            if (first > 1) {
                cursor = prev;    // Unwind one character
                first--;
            }
            break;
        // Irrelevant cases omitted
        // [...]
        }
        break;
    }
    if (first == 1) {
        return newSingle(buffer[0]);
    } else {
        return newSlice(buffer, first, hasSupplementary);
    }
}

When the process enters atom(), since it encounters { right away, it breaks from switch and for loop, and a new slice with length 0 is created (the length comes from first, which is 0).

When this slice is returned, the quantifier is parsed by closure(), resulting in what we see.

Comparing the source code of Java 1.4.0, Java 5 and Java 8, there doesn't seem to be much changes in the source code of sequence() and atom(). It seems this bug has been there since the beginning.

Standard for regular expression

The top-voted answer citing IEEE-Standard 1003.1 (or POSIX standard) is irrelevant to the discussion, since Java does not implement BRE and ERE.

There are many syntax resulting in undefined behavior according to the standard, but is well-defined behavior across many other regex flavors (though whether they agree or not is another matter). For example, \d is undefined according to the standard, but it matches digits (ASCII/Unicode) in many regex flavors.

Sadly, there is no other standard on regular expression syntax.

There is, however, a standard on Unicode Regular Expression, which focuses on features a Unicode regex engine should have. Java Pattern class more or less implements Level 1 support as described in UTS #18: Unicode Regular Expression and RL2.1 (albeit extremely buggy).