Overlapping matches with finditer() in Python

Question

I\'m using a regex to match Bible verse references in a text. The current regex is

REF_REGEX = re.compile(\'\'\'
  (?

        

Answer 1

A character consumed is consumed, you should not ask the regex engine to go back.

From your examples the verse part (e.g. :1) seems not optional. Removing that will match the last bit.

ref_regex = re.compile('''
(?<!\w)                      # Not preceeded by any words
((?i)q(?:uote)?\s+)?            # Match 'q' or 'quote' followed by many spaces
(
    (?:(?:[1-3]|I{1,3})\s*)?    # Match an arabic or roman number between 1 and 3.
    [A-Za-z]+                   # Match many alphabetics
)\.?                            # Followed by an optional dot
(?:
    \s*(\d+)                    # Match the chapter number
    (?:
        [:.](\d+)               # Match the verse number
        (?:-(\d+))?             # Match the ending verse number
    )                    # <-- no '?' here
)
(?:
    \s+
    (?:
        (?i)(?:from\s+)|        # Match the keyword 'from' or 'in'
        (?:in\s+)|
        (?P<lbrace>\()      # or stuff between (...)
    )\s*(\w+)
    (?(lbrace)\))
)?
''', re.X | re.U)

(If you're going to write a gigantic RegEx like this, please use the /x flag.)

---

If you really need overlapping matches, you could use a lookahead. A simple example is

>>> rx = re.compile('(.)(?=(.))')
>>> x = rx.finditer("abcdefgh")
>>> [y.groups() for y in x]
[('a', 'b'), ('b', 'c'), ('c', 'd'), ('d', 'e'), ('e', 'f'), ('f', 'g'), ('g', 'h')]

You may extend this idea to your RegEx.