How to escape single quotes within single quoted strings

Question

Let\'s say, you have a Bash alias like:

alias rxvt=\'urxvt\'

which works fine.

However:

Answer 1

Another way to fix the problem of too many layers of nested quotation:

You are trying to cram too much into too tiny a space, so use a bash function.

The problem is you are trying to have too many levels of nesting, and the basic alias technology is not powerful enough to accommodate. Use a bash function like this to make it so the single, double quotes back ticks and passed in parameters are all handled normally as we would expect:

lets_do_some_stuff() {
    tmp=$1                       #keep a passed in parameter.
    run_your_program $@          #use all your passed parameters.
    echo -e '\n-------------'    #use your single quotes.
    echo `date`                  #use your back ticks.
    echo -e "\n-------------"    #use your double quotes.
}
alias foobarbaz=lets_do_some_stuff

Then you can use your $1 and $2 variables and single, double quotes and back ticks without worrying about the alias function wrecking their integrity.

This program prints:

el@defiant ~/code $ foobarbaz alien Dyson ring detected @grid 10385
alien Dyson ring detected @grid 10385
-------------
Mon Oct 26 20:30:14 EDT 2015
-------------

Answer 2

If you have GNU Parallel installed you can use its internal quoting:

$ parallel --shellquote
L's 12" record
<Ctrl-D>
'L'"'"'s 12" record'
$ echo 'L'"'"'s 12" record'
L's 12" record

From version 20190222 you can even --shellquote multiple times:

$ parallel --shellquote --shellquote --shellquote
L's 12" record
<Ctrl-D>
'"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
$ eval eval echo '"'"'"'"'"'"'L'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'s 12" record'"'"'"'"'"'"'
L's 12" record

It will quote the string in all supported shells (not only bash).

Answer 3

Most of these answers hit on the specific case you're asking about. There is a general approach that a friend and I have developed that allows for arbitrary quoting in case you need to quote bash commands through multiple layers of shell expansion, e.g., through ssh, su -c, bash -c, etc. There is one core primitive you need, here in native bash:

quote_args() {
    local sq="'"
    local dq='"'
    local space=""
    local arg
    for arg; do
        echo -n "$space'${arg//$sq/$sq$dq$sq$dq$sq}'"
        space=" "
    done
}

This does exactly what it says: it shell-quotes each argument individually (after bash expansion, of course):

$ quote_args foo bar
'foo' 'bar'
$ quote_args arg1 'arg2 arg2a' arg3
'arg1' 'arg2 arg2a' 'arg3'
$ quote_args dq'"'
'dq"'
$ quote_args dq'"' sq"'"
'dq"' 'sq'"'"''
$ quote_args "*"
'*'
$ quote_args /b*
'/bin' '/boot'

It does the obvious thing for one layer of expansion:

$ bash -c "$(quote_args echo a'"'b"'"c arg2)"
a"b'c arg2

(Note that the double quotes around $(quote_args ...) are necessary to make the result into a single argument to bash -c.) And it can be used more generally to quote properly through multiple layers of expansion:

$ bash -c "$(quote_args bash -c "$(quote_args echo a'"'b"'"c arg2)")"
a"b'c arg2

The above example:

1. shell-quotes each argument to the inner quote_args individually and then combines the resulting output into a single argument with the inner double quotes.
2. shell-quotes bash, -c, and the already once-quoted result from step 1, and then combines the result into a single argument with the outer double quotes.
3. sends that mess as the argument to the outer bash -c.

That's the idea in a nutshell. You can do some pretty complicated stuff with this, but you have to be careful about order of evaluation and about which substrings are quoted. For instance, the following do the wrong things (for some definition of "wrong"):

$ (cd /tmp; bash -c "$(quote_args cd /; pwd 1>&2)")
/tmp
$ (cd /tmp; bash -c "$(quote_args cd /; [ -e *sbin ] && echo success 1>&2 || echo failure 1>&2)")
failure

In the first example, bash immediately expands quote_args cd /; pwd 1>&2 into two separate commands, quote_args cd / and pwd 1>&2, so the CWD is still /tmp when the pwd command is executed. The second example illustrates a similar problem for globbing. Indeed, the same basic problem occurs with all bash expansions. The problem here is that a command substitution isn't a function call: it's literally evaluating one bash script and using its output as part of another bash script.

If you try to simply escape the shell operators, you'll fail because the resulting string passed to bash -c is just a sequence of individually-quoted strings that aren't then interpreted as operators, which is easy to see if you echo the string that would have been passed to bash:

$ (cd /tmp; echo "$(quote_args cd /\; pwd 1\>\&2)")
'cd' '/;' 'pwd' '1>&2'
$ (cd /tmp; echo "$(quote_args cd /\; \[ -e \*sbin \] \&\& echo success 1\>\&2 \|\| echo failure 1\>\&2)")
'cd' '/;' '[' '-e' '*sbin' ']' '&&' 'echo' 'success' '1>&2' '||' 'echo' 'failure' '1>&2'

The problem here is that you're over-quoting. What you need is for the operators to be unquoted as input to the enclosing bash -c, which means they need to be outside the $(quote_args ...) command substitution.

Consequently, what you need to do in the most general sense is to shell-quote each word of the command not intended to be expanded at the time of command substitution separately, and not apply any extra quoting to the shell operators:

$ (cd /tmp; echo "$(quote_args cd /); $(quote_args pwd) 1>&2")
'cd' '/'; 'pwd' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")
/
$ (cd /tmp; echo "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
'cd' '/'; [ -e *'sbin' ] && 'echo' 'success' 1>&2 || 'echo' 'failure' 1>&2
$ (cd /tmp; bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")
success

Once you've done this, the entire string is fair game for further quoting to arbitrary levels of evaluation:

$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")"
/
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")"
/
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); $(quote_args pwd) 1>&2")")")"
/
$ bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")"
success
$ bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *sbin ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")"
success
$ bash -c "$(quote_args bash -c "$(quote_args bash -c "$(quote_args cd /tmp); $(quote_args bash -c "$(quote_args cd /); [ -e *$(quote_args sbin) ] && $(quote_args echo success) 1>&2 || $(quote_args echo failure) 1>&2")")")"
success

etc.

These examples may seem overwrought given that words like success, sbin, and pwd don't need to be shell-quoted, but the key point to remember when writing a script taking arbitrary input is that you want to quote everything you're not absolutely sure doesn't need quoting, because you never know when a user will throw in a Robert'; rm -rf /.

To better understand what is going on under the covers, you can play around with two small helper functions:

debug_args() {
    for (( I=1; $I <= $#; I++ )); do
        echo -n "$I:<${!I}> " 1>&2
    done
    echo 1>&2
}

debug_args_and_run() {
    debug_args "$@"
    "$@"
}

that will enumerate each argument to a command before executing it:

$ debug_args_and_run echo a'"'b"'"c arg2
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)"
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'> 
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''> 
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'> 
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

$ bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run bash -c "$(quote_args debug_args_and_run echo a'"'b"'"c arg2)")")")"
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'bash'"'"' '"'"'-c'"'"' '"'"''"'"'"'"'"'"'"'"'debug_args_and_run'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'echo'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'a"b'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'"'c'"'"'"'"'"'"'"'"' '"'"'"'"'"'"'"'"'arg2'"'"'"'"'"'"'"'"''"'"''> 
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'bash' '-c' ''"'"'debug_args_and_run'"'"' '"'"'echo'"'"' '"'"'a"b'"'"'"'"'"'"'"'"'c'"'"' '"'"'arg2'"'"''> 
1:<bash> 2:<-c> 3:<'debug_args_and_run' 'echo' 'a"b'"'"'c' 'arg2'> 
1:<echo> 2:<a"b'c> 3:<arg2> 
a"b'c arg2

Answer 4

In the given example, simply used double quotes instead of single quotes as outer escape mechanism:

alias rxvt="urxvt -fg '#111111' -bg '#111111'"

This approach is suited for many cases where you just want to pass a fixed string to a command: Just check how the shell will interpret the double-quoted string through an echo, and escape characters with backslash if necessary.

In the example, you'd see that double quotes are sufficient to protect the string:

$ echo "urxvt -fg '#111111' -bg '#111111'"
urxvt -fg '#111111' -bg '#111111'

Answer 5

I always just replace each embedded single quote with the sequence: '\'' (that is: quote backslash quote quote) which closes the string, appends an escaped single quote and reopens the string.

---

I often whip up a "quotify" function in my Perl scripts to do this for me. The steps would be:

s/'/'\\''/g    # Handle each embedded quote
$_ = qq['$_']; # Surround result with single quotes.

This pretty much takes care of all cases.

Life gets more fun when you introduce eval into your shell-scripts. You essentially have to re-quotify everything again!

For example, create a Perl script called quotify containing the above statements:

#!/usr/bin/perl -pl
s/'/'\\''/g;
$_ = qq['$_'];

then use it to generate a correctly-quoted string:

$ quotify
urxvt -fg '#111111' -bg '#111111'

result:

'urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''

which can then be copy/pasted into the alias command:

alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''

(If you need to insert the command into an eval, run the quotify again:

 $ quotify
 alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''

result:

'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''

which can be copy/pasted into an eval:

eval 'alias rxvt='\''urxvt -fg '\''\'\'''\''#111111'\''\'\'''\'' -bg '\''\'\'''\''#111111'\''\'\'''\'''\'''

Answer 6

Since Bash 2.04 syntax $'string' (instead of just 'string'; warning: do not confuse with $('string')) is another quoting mechanism which allows ANSI C-like escape sequences and do expansion to single-quoted version.

Simple example:

  $> echo $'aa\'bb'
  aa'bb

  $> alias myvar=$'aa\'bb'
  $> alias myvar
  alias myvar='aa'\''bb'

In your case:

$> alias rxvt=$'urxvt -fg \'#111111\' -bg \'#111111\''
$> alias rxvt
alias rxvt='urxvt -fg '\''#111111'\'' -bg '\''#111111'\'''

Common escaping sequences works as expected:

\'     single quote
\"     double quote
\\     backslash
\n     new line
\t     horizontal tab
\r     carriage return

---

Below is copy+pasted related documentation from man bash (version 4.4):

Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. Backslash escape sequences, if present, are decoded as follows:

    \a     alert (bell)
    \b     backspace
    \e
    \E     an escape character
    \f     form feed
    \n     new line
    \r     carriage return
    \t     horizontal tab
    \v     vertical tab
    \\     backslash
    \'     single quote
    \"     double quote
    \?     question mark
    \nnn   the eight-bit character whose value is the octal 
           value nnn (one to three digits)
    \xHH   the eight-bit character whose value is the hexadecimal
           value HH (one or two hex digits)
    \uHHHH the Unicode (ISO/IEC 10646) character whose value is 
           the hexadecimal value HHHH (one to four hex digits)
    \UHHHHHHHH the Unicode (ISO/IEC 10646) character whose value 
               is the hexadecimal value HHHHHHHH (one to eight 
               hex digits)
    \cx    a control-x character

The expanded result is single-quoted, as if the dollar sign had not been present.

---

See Quotes and escaping: ANSI C like strings on bash-hackers.org wiki for more details. Also note that "Bash Changes" file (overview here) mentions a lot for changes and bug fixes related to the $'string' quoting mechanism.

According to unix.stackexchange.com How to use a special character as a normal one? it should work (with some variations) in bash, zsh, mksh, ksh93 and FreeBSD and busybox sh.