I have the following snippet:
printf "%s\n%s\n%s" $(echo "'hello world' world") I would expect it to produce:
hello world world But it actually produces:
'hello world' world Why is the above command not the same as the following?
printf "%s\n%s\n%s" 'hello world' world After command substitution, only word splitting and wildcard expansion are done, but not quote processing.
From tbe Bash Reference Manual
The order of expansions is: brace expansion, tilde expansion, parameter, variable, and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and filename expansion.
The description of word splitting is:
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly
<space><tab><newline>, the default, then sequences of<space>,<tab>, and<newline>at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words. If IFS has a value other than the default, then sequences of the whitespace characters space and tab are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS (an IFS whitespace character). Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field. A sequence of IFS whitespace characters is also treated as a delimiter. If the value of IFS is null, no word splitting occurs.
No mention is made of treating quotes specially when doing this.
Bash doesn't see what you think it should see because Bash splits words after command substitution.
Your two code samples are not equivalent. In your non-substitution example, there is no command substitution performed, so the quoting from step 2 of shell operation will cause your text to be seen as two arguments. For example:
$ set -x $ printf "%s\n%s\n%s" 'hello world' world + printf '%s\n%s\n%s' 'hello world' world hello world world In your other example, Bash doesn't reinterpret the quoting characters that remain after command substitution, but does perform word-splitting after performing shell expansions. For example:
$ set -x $ printf "%s\n%s\n%s" $(echo "'hello world' world") ++ echo ''\''hello world'\'' world' + printf '%s\n%s\n%s' ''\''hello' 'world'\''' world 'hello world' So, Bash sees ''\''hello' 'world'\''' world and then splits the words according to $IFS. This leaves literal single-quotes in the first two words, and the last word is not used by your printf statement. An easier way to see this is to change the last word or remove the excess quotes:
# The third argument is never used. $ printf "%s\n%s\n%s" $(echo "'hello world' unused_word") 'hello world' # Still breaks into three words, but no quote literals are left behind! $ printf "%s\n%s\n%s" $(echo 'hello world' unused_word) hello world The following:
$(echo "'hello world' world") outputs three tokens: 'hello, world', world. Those three tokens are being passed to the printf this way:
printf "%s\n%s\n%s" ('hello) (world') (world) and not this way:
printf "%s\n%s\n%s" ('hello world') (world) (note that parentheses are here only to visually separate tokens, to illustrate the point).
Lots of discussion why, but no one has bothered to add how to work around this behavior?
So far I've found running eval $(...) to reevaluate the quotes works. Though of course use this sparingly, and not at all if you can't trust the output of the command substitution.