问题

This question already has an answer here:

echo “#!” fails — “event not found” 5 answers

I am attempting to parse the output of a VNC server startup event and have run into a problem in parsing using sed in a command substitution. Specifically, the remote VNC server is started in a manner such as the following:

address1=\"user1@lxplus.cern.ch\"
VNCServerResponse=\"$(ssh \"${address1}\" \'vncserver\' 2>&1)\"

The standard error output produced in this startup event is then to be parsed in order to extract the server and display information. At this point the content of the variable VNCServerResponse is something such as the following:

New \'lxplus0186.cern.ch:1 (user1)\' desktop is lxplus0186.cern.ch:1

Starting applications specified in /afs/cern.ch/user/u/user1/.vnc/xstartup
Log file is /afs/cern.ch/user/u/user1/.vnc/lxplus0186.cern.ch:1.log

This output can be parsed in the following way in order to extract the server and display information:

echo \"${VNCServerResponse}\" | sed \'/New.*desktop.*is/!d\' \\
    | awk -F\" desktop is \" \'{print $2}\'

The result is something such as the following:

lxplus0186.cern.ch:1

What I want to do is use this parsing in a command substitution something like the following:

VNCServerAndDisplayNumber=\"$(echo \"${VNCServerResponse}\" \\
    | sed \'/New.*desktop.*is/!d\' | awk -F\" desktop is \" \'{print $2}\')\"

On attempting to do this, I am presented with the following error:

bash: !d\': event not found

I am not sure how to address this. It appears to be a problem in the way sed is being used in the command substitution. I would appreciate guidance.

回答1:

The problem is that within double quotes, bash is trying to expand !d before passing it to the subshell. You can get around this problem by removing the double quotes but I would also propose a simplification to your script:

VNCServerAndDisplayNumber=$(echo "$VNCServerResponse" | awk '/desktop/ {print $NF}')

This simply prints the last field on the line containing the word "desktop".

On a newer bash, you can use a herestring rather than piping an echo:

VNCServerAndDisplayNumber=$(awk '/desktop/ {print $NF}' <<<"$VNCServerResponse")

回答2:

Bash history expansion is a very odd corner in the bash command line parser, and you are clearly running into an unexpected history expansion, which is explained below. However, any sort of history expansion in a script is unexpected, because normally history expansion is not enabled in scripts; not even scripts run with the source (or .) builtin.

How history expansion is enabled (or disabled)

There are two shell options which control history expansion:

set -o history: Required for the history to be recorded.
set -H (or set -o histexpand): Additionally required for history expansion to be enabled.

Both of these options must be set for history expansion to be recognized. (I found the manual unclear on this interaction, but it's logical enough.)

According to the bash manual, these options are unset for non-interactive shells, so if you want to enable history expansion in a script (and I cannot imagine a reason you would want this), you would need to set both of them:

set -o history -o histexpand

The situation for scripts run with source is more complicated (and what I'm about to say only applies to bash v4, and since it's undocumented in might change in the future). [Note 3]

History recording (and consequently expansion) is turned off in source'd scripts, but through an internal flag which, as far as I know, is not made visible. It certainly does not appear in $SHELLOPTS. Since a sourced script runs in the current bash context, it shares the current execution environment, including shell options. So in the execution of a sourced script initiated from an interactive session, you'll see both history and histexpand in $SHELLOPTS, but no history expansion will take place. In order to enable it, you need to:

set -o history

which is not a no-op because it has the side-effect of resetting the internal flag which suppresses history recording. Setting the histexpand shell option does not have this side-effect.

In short, I'm not sure how you managed to enable history expansion in a script (if, indeed, the misbehaving command was in a script and not in an interactive shell), but you might want to consider not doing so, unless you have a really good reason.

How history expansion is parsed

The bash implementation of history expansion is designed to work with readline, so that it can be performed during command input. (By default this function is bound to Meta-^; generally Meta is ESC, but you can customize that as well.) However, it is also performed immediately after each line is input, before any bash parsing is performed.

By default, the history expansion character is !, and -- as mostly documented -- that will trigger history expansion except:

when it is followed by whitespace or =
if the shell option extglob is set, and it is followed by ( [Note 1]
if it appears in a single-quoted string
if it is preceded by a \ [Note 2 and see below]
if it is preceded by $ or ${ [Note 1]
if it is preceded by [ [Note 1]
(As of bash v4.3) if it is the last character in a double-quoted string.

The immediate issue here is the precise interpretation of the third case, an ! appearing inside of a single-quoted string. Normally, bash starts a new quoting context for a command substitution ($(...) or the deprecated backtick notation). For example:

$ s=SUBSTITUTED
$ # The interior single quotes are just characters
$ echo "'Echoing $s'"
'Echoing SUBSTITUTED'
$ # The interior single quotes are single quotes
$ echo "$(echo 'Echoing $s')"
Echoing $s

However, the history expansion scanner isn't that intelligent. It keeps track of quotes, but not of command substitution. So as far as it is concerned, both of the single quotes in the above example are double-quoted single quotes, which is to say ordinary characters. So history expansion occurs in both of them:

# A no-op to indicated history expansion
$ HIST() { :; }
# Single-quoted strings inhibit history expansion
$ HIST
$ echo '!!'
!!
# Double-quoted strings allow history expansion
$ HIST
$ echo "'!!'"
echo "'HIST'"
'HIST'
# ... and it applies also to interior command substitution.
$ HIST
$ echo "$(echo '!!')"
echo "$(echo 'HIST')"
HIST

So if you have a perfectly normal command like sed '/foo/!d' file, where you would expect the single-quotes to protect you from history-expansion, and you put it inside a double-quoted command substitution:

result="$(sed '/foo/!d' file)"

you suddenly find that the ! is a history expansion character. Worse, you can't fix this by backslash escaping the exclamation point, because although "\!" inhibits history expansion, it doesn't remove the backslash:

$ echo "\!"
\!

In this particular example -- and the one in the OP -- the double quotes are completely unnecessary, because the right-hand side of a variable assignment does not undergo either filename expansion nor word splitting. However, there are other contexts in which removing the double quotes would change the semantics:

# Undesired history expansion
printf "The answer is '%s'\n" "$(sed '/foo/!d' file)"

# Undesired word splitting
printf "The answer is '%s'\n" $(sed '/foo/!d' file)

In this case, the best solution is probably to put the sed argument in a variable

# Works
sed_prog='/foo/!d'
printf "The answer is '%s'\n" "$(sed "$sed_prog" file)"

(The quotes around $sed_prog were not necessary in this case but usually they would be, and they do no harm.)

Notes:

The inhibition of history expansion when the following character is some form of open parenthesis only works if there is a corresponding close parenthesis in the rest of the string. However, it doesn't have to really match the open parenthesis. For example:
```
# No matching close parenthesis
$ echo "!("
bash: !: event not found
# The matching close parenthesis has nothing to do with the open
$ echo "!(" ")"
!( )
# An actual extended glob: files whose names don't start with a
$ echo "!(a*)"
b
```
As indicated in the bash manual, a history-expansion character is treated as an ordinary character if immediately preceded by a backslash. This is literally true; it doesn't matter whether the backslash will later be considered an escape character or not:
```
$ echo \!
!
$ echo \\!
\!
$ echo \\\!
\!
```
\ also inhibits history expansion inside double quotes, but \! is not a valid escape sequence inside the double quoted string, so the backslash is not removed:
```
$ echo "\!"
\!
$ echo "\\!"
\!
$ echo "\\\!"
\\!
```
I'm referring to the source code for bash v4.2 as I write this, so any undocumented behaviour may be completely different as of v4.3.

回答3:

Don't wrap the $(...) command substitution in double quotes. You are asking the shell to perform evaluation on the contents of the quotes and are hitting the history substitution expansion feature. Drop the quotes and you stop telling the shell to do that and you won't hit that problem.

And yes, dropping those quotes is safe on that assignment line even if the output may contain spaces or newlines or whatever. Assignments of that sort are not going to split on those the way command substitution or variable evaluation will on a normal shell execution line.

Alternatively, disable history expansion in your shell/script before you run that. (It should be off when running a script by default I believe anyway.)

回答4:

This only happens when history expansion is enabled, which it normally isn't and definitely shouldn't be for scripts.

Rather than trying to work around it, figure out why history expansion is enabled and what to do so it isn't.

If you're executing your script with . foo or source foo, use ./foo instead.
If you're writing this as a function in .bashrc or similar, consider making it a separate script.
If your script (or BASH_ENV) explicitly does set -H, don't.

回答5:

Quote it with '' or \ or disable history expansion with set +H or shopt -u -o histexpand. See History Expansion.

来源：https://stackoverflow.com/questions/25003162/how-to-address-error-bash-d-event-not-found-in-bash-command-substitution

标签

bash

parsing

sed

command-substitution

How to address error “bash: !d&#39;: event not found” in Bash command substitution [duplicate]

问题