Unix to tail solution for waiting for a specific string or quitting after a timeout

Question

I am trying to improve a service (daemon) script to display tomcat output (when run from a tty) until the application is effectively started.

Tomcat apps can take up to 10 minutes to start and it's extremely useful to see what happens.

Still, we need to be sure that this startup script will not run indefinitely if the startup confirmation message is failing to appear in the logs.

I know that the internet is full of similar questions, but so far I wasn't able to identify a clear solution, especially one that works without requiring utilities that are not available by default.

Requirements:

1. run without requiring you to install anything on recent Debian, Ubuntu, RedHat and OS X
2. no processes left behind when it ends (due to any reason)
3. display the logged lines to stdout
4. log file is readonly, you cannot touch it
5. log file can be rotated without breaking the tool
6. extra kudos if you can make it work in one line

Identified problems so far:

• read -t not available on current Debian, Illegal option -t

References to partial solutions so far:

• Shell function to tail a log file for a specific string for a specific time

Answer 1

If you want a script to monitor the tomcat output waiting for a specific string, the solution below should work, and it doesn't use read -t.

( tail -f server.log & ( (tail -f server.log | grep -l STARTED && kill -9 `pstree -pa $$ | grep -E '\-(tail|sleep),' | sed 's/^[^0-9]*\([0-9]*\).*/\1/' | grep -vw $$ | xargs` ) & sleep 900 && kill -9 `pstree -pa $$ | grep -E '\-(tail|sleep),' | sed 's/^[^0-9]*\([0-9]*\).*/\1/' | grep -vw $$ | xargs` && exit 3 ) ) ; [ $? -eq 3 ]  && echo 'server not up after 5 min'  || echo 'server up'

It is searching for the string "STARTED" inside the log file server.log, with a timeout of 15 minutes (900 seconds). I'm not sure what you have available in your Debian. You should have pstree, whose output should be similar to:

$ pstree -pa 1803
bash,1803
  └─bash,7850
      ├─bash,7852
      │   ├─bash,7853
      │   │   ├─grep,7856 --color=auto -l STARTED
      │   │   └─tail,7855 -f server.log
      │   └─sleep,7854 20
      └─tail,7851 -f server.log

The rest I'm sure you should have: kill, sed, sleep, grep, echo, xargs.

Some explanation of what is happening. This line contains one command and the final evaluation of the command results, to print if the server is up or not. The command is divided in 3 parts:

1. simply output the log content, as requested.
2. monitor the output looking for the specific string "STARTED"; if found, kill parts 1 and 3.
3. wait for 15 minutes, and if 15 minutes pass, kill parts 1 and 2, exiting with code 3 to signal to the outside evaluation that the whole command ended due to timeout.

With comments:

( tail -f server.log & # part 1 - just output the log content
  ( # part 2
    (tail -f server.log | grep -l STARTED # search for the string
     && kill -9 `pstree -pa $$ | grep -E '\-(tail|sleep),' | 
        sed 's/^[^0-9]*\([0-9]*\).*/\1/' | grep -vw $$ | 
        xargs` # if found, kill the remaining tails and sleeps
    ) & # if the command ends in part 2, exit code is not 3
  sleep 900 # part 3: sleep 15 minutes
  && kill -9 `pstree -pa $$ | grep -E '\-(tail|sleep),' | 
              sed 's/^[^0-9]*\([0-9]*\).*/\1/' | grep -vw $$ | 
              xargs` # if time passed, kill the remaining tails and sleeps
  && exit 3 # exit with code 3, so that the outside evaluation 
            # knows it ended by timeout
  )
) ; # end of command; whenever it finishes, the evaluation below is executed, 
    # checking the exit code to print the appropriate message
[ $? -eq 3 ]  && echo 'server not up after 5 min'  || echo 'server up'

Note that, due to how bash pipes work, if the "STARTED" string is found the command will actually only exit and print the message "server up" after an extra line is added to server.log after the line with "STARTED". That's because when the line "STARTED" is printed, grep will find it, print its output and exit, closing its input. However, the corresponding tail will continue to run until it has something else to write; only then tail will realize its output is closed, and then will die, moving on to the kill command that will lead to the evaluation. There's no workaround to that, AFAIK. You should make sure something is printed to the log after the "STARTED" message, if it's not already.