How to cut html tag from very large multiline text file with content with use perl, sed or awk?

Question

I want to transform this text (remove $.*?$) with sed, awk or perl:

{|
|-
| colspan=\"2\"|
: $[\backslash \backslash underbrace\{\backslash \backslash col$

Answer 1

This should do it:

perl -0777 -pe 's!$.*?$!!sg' dirt-math.txt

-p says we're doing a sed-like readline/printline loop, -0777 says each "line" is actually the whole input file, and -e specifies the code to run (on each "line" (file)).

---

If your text files are too big to fit into memory (?!), you can try this:

perl -pe 's!$.*?$!!s; if ($cut) { if (s!^.*?!!) { $cut = 0 } else { $_ = "" } } if (!$cut && s!$.*!!s)\; \{\; \$cut\; =\; 1\; \}\text{'}\; dirt-math.txt$

or (slightly more readable):

perl -pe '
    s!$.*?$!!g;
    if ($cut) {
        if (s!^.*?!!) { $cut = 0 }
        else { $_ = "" }
    }
    if (!$cut && s!$.*!!s)\; \{\; \$cut\; =\; 1\; \}\; \text{'}\; dirt-math.txt$

This is effectively a little state machine.

$cut records whether we're in an unclosed $$ tag (and so need to cut out input). If so, we check whether we were able to find/remove . If so, we're done cutting (we found a closing tag); otherwise we overwrite the "current line" with the empty string ($_ = ""; this is the actual cutting part).

If, after this, we're not cutting (we're not using else to handle the case where ... not math text $$ appears on a single line), we try to remove $...$ from the input. If so, we've just seen an opening $$ tag and need to start cutting.