How to find and replace a particular character but only if it is in quotes?

Question

Problem: I have thousands of documents which contains a specific character I don\'t want. E.g. the character a. These documents contain a variety of characters, but

Answer 1

Firstly a few of considerations:

1. There could be multiple a characters within a single quote.
2. Each quote (using single or double quotation marks) consists of an opening quote character, some text and the same closing quote character. A simple approach is to assume that when the quote characters are counted sequentially, the odd ones are opening quotes and the even ones are closing quotes.
3. Following point 2, it could be worth some further thought on whether single-quoted strings should be allowed. See the following example: It's a shame 'this quoted text' isn't quoted. Here, the simple approach would think there were two quoted strings: s a shame and isn. Another: This isn't a quote ...'this is' and 'it's unclear where this quote ends'. I've avoided attempting to tackle these complexities and gone with the simple approach below.

The bad news is that point 1 presents a bit of a problem, as a capturing group with a wildcard repeat character after it (e.g. (.*)*) will only capture the last captured "thing". But the good news is there's a way of getting around this within certain limits. Many regex engines will allow up to 99 capturing groups (*). So if we can make the assumption that there will be no more than 99 as in each quote (UPDATE ...or even if we can't - see step 3), we can do the following...

(*) Unfortunately my first port of call, Notepad++ doesn't - it only allows up to 9. Not sure about VS Code. But regex101 (used for the online demos below) does.

TL;DR - What to do?

1. Search for: "([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*([^a"]*)a*"
2. Replace with: "\1\2\3\4\5\6\7\8\9\10\11\12\13\14\15\16\17\18\19\20\21\22\23\24\25\26\27\28\29\30\31\32\33\34\35\36\37\38\39\40\41\42\43\44\45\46\47\48\49\50\51\52\53\54\55\56\57\58\59\60\61\62\63\64\65\66\67\68\69\70\71\72\73\74\75\76\77\78\79\80\81\82\83\84\85\86\87\88\89\90\91\92\93\94\95\96\97\98\99"
3. (Optionally keep repeating steps the previous two steps if there's a possibility of > 99 such characters in a single quote until they've all been replaced).
4. Repeat step 1 but replacing all " with ' in the regular expression, i.e: '([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*([^a']*)a*'
5. Repeat steps 2-3.

Online demos

Please see the following regex101 demos, which could actually be used to perform the replacements if you're able to copy the whole text into the contents of "TEST STRING":

• Demo for double quotes
• Demo for single quotes.

Answer 2

"Inside double quotes" is rather tricky, because there are may complicating scenarios to consider to fully automate this.

What are your precise rules for "enclosed by quotes"? Do you need to consider multi-line quotes? Do you have quoted strings containing escaped quotes or quotes used other than starting/ending string quotation?

However there may be a fairly simple expression to do much of what you want.

Search expression: ("[^a"]*)a

Replacement expression: $1b

This doesn't consider inside or outside of quotes - you have do that visually. But it highlights text from the quote to the matching character, so you can quickly decide if this is inside or not.

If you can live with the visual inspection, then we can build up this pattern to include different quote types and upper and lower case.

Answer 3

I am using VSCode, but I'm open to any suggestions.

If you want to stay in an Editor environment, you could use
Visual Studio (>= 2012) or even notepad++ for quick fixup.
This avoids having to use a spurious script environment.

Both of these engines (Dot-Net and boost, respectively) use the \G construct.
Which is start the next match at the position where the last one left off.

Again, this is just a suggestion.

This regex doesn't check the validity of balanced quotes within the entire
string ahead of time (but it could with the addition of a single line).

It is all about knowing where the inside and outside of quotes are.

I've commented the regex, but if you need more info let me know.
Again this is just a suggestion (I know your editor uses ECMAScript).

Find (?s)(?:^([^"]*(?:"[^"a]*(?=")"[^"]*(?="))*"[^"a]*)|(?!^)\G)a([^"a]*(?:(?=a.*?")|(?:"[^"]*$|"[^"]*(?=")(?:"[^"a]*(?=")"[^"]*(?="))*"[^"a]*)))
Replace $1b$2

That's all there is to it.

https://regex101.com/r/loLFYH/1

Comments

(?s)                          # Dot-all inine modifier
 (?:
      ^                             # BOS 
      (                             # (1 start), Find first quote from BOS (written back)
           [^"]* 
           (?:                           # --- Cluster
                " [^"a]*                      # Inside quotes with no 'a'
                (?= " )
                " [^"]*                       # Between quotes, get up to next quote
                (?= " )
           )*                            # --- End cluster, 0 to many times

           " [^"a]*                      # Inside quotes, will be an 'a' ahead of here
                                         # to be sucked up by this match           
      )                             # (1 end)

   |                              # OR,

      (?! ^ )                       # Not-BOS 
      \G                            # Continue where left off from last match.
                                    # Must be an 'a' at this point
 )
 a                             # The 'a' to be replaced

 (                             # (2 start), Up to the next 'a' (to be written back)
      [^"a]* 
      (?:                           # --------------------
           (?= a .*? " )                 # If stopped before 'a', must be a quote ahead
        |                              # or,
           (?:                           # --------------------
                " [^"]* $                     # If stopped at a quote, check for EOS
             |                              # or, 
                " [^"]*                       # Between quotes, get up to next quote
                (?= " )

                (?:                           # --- Cluster
                     " [^"a]*                      # Inside quotes with no 'a'
                     (?= " )
                     " [^"]*                       # Between quotes 
                     (?= " )
                )*                            # --- End cluster, 0 to many times

                " [^"a]*                      # Inside quotes, will be an 'a' ahead of here
                                              # to be sucked up on the next match                    
           )                             # --------------------
      )                             # --------------------
 )                             # (2 end)

Answer 4

If you can use Visual Studio (instead of Visual Studio Code), it is written in C++ and C# and uses the .NET Framework regular expressions, which means you can use variable length lookbehinds to accomplish this.

(?<="[^"\n]*)a(?=[^"\n]*")

Adding some more logic to the above regular expression, we can tell it to ignore any locations where there are an even amount of " preceding it. This prevents matches for a outside of quotes. Take, for example, the string "a" a "a". Only the first and last a in this string will be matched, but the one in the middle will be ignored.

(?<!^[^"\n]*(?:(?:"[^"\n]*){2})+)(?<="[^"\n]*)a(?=[^"\n]*")

Now the only problem is this will break if we have escaped " within two double quotes such as "a\"" a "a". We need to add more logic to prevent this behaviour. Luckily, this beautiful answer exists for properly matching escaped ". Adding this logic to the regex above, we get the following:

(?<!^[^"\n]*(?:(?:"(?:[^"\\\n]|\\.)*){2})+)(?<="[^"\n]*)a(?=[^"\n]*")

I'm not sure which method works best with your strings, but I'll explain this last regex in detail as it also explains the two previous ones.

• (?<!^[^"\n]*(?:(?:"(?:[^"\\\n]|\\.)*){2})+) Negative lookbehind ensuring what precedes doesn't match the following
  • ^ Assert position at the start of the line
  • [^"\n]* Match anything except " or \n any number of times
  • (?:(?:"(?:[^"\\\n]|\\.)*){2})+ Match the following one or more times. This ensures if there are any " preceding the match that they are balanced in the sense that there is an opening and closing double quote.
    • (?:"(?:[^"\\\n]|\\.)*){2} Match the following exactly twice
    • " Match this literally
    • (?:[^"\\\n]|\\.)* Match either of the following any number of times
      • [^"\\\n] Match anything except ", \ and \n
      • \\. Matches \ followed by any character
• (?<="[^"\n]*) Positive lookbehind ensuring what precedes matches the following
  • " Match this literally
  • [^"\n]* Match anything except " or \n any number of times
• a Match this literally
• (?=[^"\n]*") Positive lookahead ensuring what follows matches the following
  • [^"\n]* Match anything except " or \n any number of times
  • " Match this literally

You can drop the \n from the above pattern as the following suggests. I added it just in case there's some sort of special cases I'm not considering (i.e. comments) that could break this regex within your text. The \A also forces the regex to match from the start of the string (or file) instead of the start of the line.

(?<!\A[^"]*(?:(?:"(?:[^"\\]|\\.)*){2})+)(?<="[^"]*)a(?=[^"]*")

You can test this regex here

This is what it looks like in Visual Studio:

Answer 5

Visual Studio Code

VS Code uses JavaScript RegEx engine for its find / replace functionality. This means you are very limited in working with regex in comparison to other flavors like .NET or PCRE.

Lucky enough that this flavor supports lookaheads and with lookaheads you are able to look for but not consume character. So one way to ensure that we are within a quoted string is to look for number of quotes down to bottom of file / subject string to be odd after matching an a:

a(?=[^"]*"[^"]*(?:"[^"]*"[^"]*)*$)

Live demo

This looks for as in a double quoted string, to have it for single quoted strings substitute all "s with '. You can't have both at a time.

There is a problem with regex above however, that it conflicts with escaped double quotes within double quoted strings. To match them too if it matters you have a long way to go:

a(?=[^"\\]*(?:\\.[^"\\]*)*"[^"\\]*(?:\\.[^"\\]*)*(?:"[^"\\]*(?:\\.[^"\\]*)*"[^"\\]*(?:\\.[^"\\]*)*)*$)

Applying these approaches on large files probably will result in an stack overflow so let's see a better approach.

I am using VSCode, but I'm open to any suggestions.

That's great. Then I'd suggest to use awk or sed or something more programmatic in order to achieve what you are after or if you are able to use Sublime Text a chance exists to work around this problem in a more elegant way.

Sublime Text

This is supposed to work on large files with hundred of thousands of lines but care that it works for a single character (here a) that with some modifications may work for a word or substring too:

Search for:

(?:"|\G(?<!")(?!\A))(?<r>[^a"\\]*+(?>\\.[^a"\\]*)*+)\K(a|"(*SKIP)(*F))(?(?=((?&r)"))\3)
                           ^              ^            ^

Replace it with: WHATEVER\3

Live demo

RegEx Breakdown:

(?: # Beginning of non-capturing group #1
    "   # Match a `"`
    |   # Or
    \G(?<!")(?!\A)  # Continue matching from last successful match
                    # It shouldn't start right after a `"`
)   # End of NCG #1
(?<r>   # Start of capturing group `r`
    [^a"\\]*+   # Match anything except `a`, `"` or a backslash (possessively)
    (?>\\.[^a"\\]*)*+   # Match an escaped character or 
                        # repeat last pattern as much as possible
)\K     # End of CG `r`, reset all consumed characters
(   # Start of CG #2 
    a   # Match literal `a`
    |   # Or
    "(*SKIP)(*F)    # Match a `"` and skip over current match
)
(?(?=   # Start a conditional cluster, assuming a positive lookahead
    ((?&r)")    # Start of CG #3, recurs CG `r` and match `"`
  )     # End of condition
  \3    # If conditional passed match CG #3
 )  # End of conditional

Three-step approach

Last but not least...

Matching a character inside quotation marks is tricky since delimiters are exactly the same so opening and closing marks can not be distinguished from each other without taking a look at adjacent strings. What you can do is change a delimiter to something else so that you can look for it later.

Step 1:

Search for: "[^"\\]*(?:\\.[^"\\]*)*"

Replace with: $0Я

Step 2:

Search for: a(?=[^"\\]*(?:\\.[^"\\]*)*"Я)

Replace with whatever you expect.

Step 3:

Search for: "Я

Replace with nothing to revert every thing.

---

Answer 6

/(["'])(.*?)(a)(.*?\1)/g

With the replace pattern:

$1$2$4

As far as I'm aware, VS Code uses the same regex engine as JavaScript, which is why I've written my example in JS.

The problem with this is that if you have multiple a's in 1 set of quotes, then it will struggle to pull out the right values, so there needs to be some sort of code behind it, or you, hammering the replace button until no more matches are found, to recurse the pattern and get rid of all the a's in between quotes

let regex = /(["'])(.*?)(a)(.*?\1)/g,
subst = `$1$2$4`,
str = `"a"
"helapke"
Not matched - aaaaaaa
"This is the way the world ends"
"Not with fire"
"ABBA"
"abba",
'I can haz cheezburger'
"This is not a match'
`;


// Loop to get rid of multiple a's in quotes
while(str.match(regex)){
    str = str.replace(regex, subst);
}

const result = str;
console.log(result);