linux script with netcat stops working after x hours

Question

I\'ve have to scripts:

#!/bin/bash

netcat -lk -p 12345 | while read line
do
    match=$(echo $line | grep -c \'Keep-Alive\')
    if [ $match -eq 1 ]; then
              


        

Answer 1

About the loop it could look like this.

#!/bin/bash

for (( ;; ))
do
    netcat -lk -p 12345 | while read line
    do
        match=$(echo "$line" | grep -c 'Keep-Alive')
        if [ "$match" -eq 1 ]; then
            [start a command]
        fi
    done
    sleep 4s
done

with added double quotes to keep it safer.

And you could try capturing errors and add some logging with this format:

#!/bin/bash

{
    echo "[$(date "+%F %T")] Starting loop."

    for (( ;; ))
    do
        echo "[$(date "+%F %T")] Starting netcat."

        netcat -lk -p 12345 | while read line
        do
            match=$(echo "$line" | grep -c 'Keep-Alive')
            if [ "$match" -eq 1 ]; then
                [start a command]
            fi
        done

        echo "[$(date "+%F %T")] Netcat has stopped or crashed."

        sleep 4s
    done
} >> "/var/log/something.log" 2>&1

Your read command could also be better in this format since it would read lines unmodified:

... | while IFS= read -r line

Some could also suggest the use of process substitution but I don't recommend it this time since through the | while ... method the while loop would be able to run on a subshell and keep the outer for loop safe just in case it crashes. Besides there isn't really a variable from the while loop that would be needed outside of it.

I'm actually having the idea now that the issue might actually have been related to the input and how the while read line; do ...; done block handles it and not netcat itself. Your variables not being quoted properly around "" could be one of it, or could probably be the actual reason why your netcat is crashing.

Answer 2

You mentioned "after around 12 hours, the whole system stops working" - It is likely that the scripts are executing whatever you have in [start a command] and is bloating the memory. Are you sure the [start a command] is not forking out many processes very frequently and releasing memory?

Answer 3

Periodically netcat will print, not a line, but a block of binary data. The read builtin will likely fail as a result.

I think you're using this program to verify that a remote host is still connected to port 12345 and 12346 and hasn't been rebooted.

My solution for you is to pipe the output of netcat to sed, then pipe that (much reduced) line to the read builtin...

#!/bin/bash

{
    echo "[$(date "+%F %T")] Starting loop."

    for (( ;; ))
    do
        echo "[$(date "+%F %T")] Starting netcat."

        netcat -lk -p 12345 | sed 's/.*Keep-Alive.*/Keep-Alive/g' | \
        \
        while read line
        do
            match=$(echo "$line" | grep -c 'Keep-Alive')
            if [ "$match" -eq 1 ]; then
                [start a command]
            fi
        done

        echo "[$(date "+%F %T")] Netcat has stopped or crashed."

        sleep 4s
    done
} >> "/var/log/something.log" 2>&1

Also, you'll need to review some of the other startup programs in /etc/init.d to make sure they are compatible with whatever version of rc the system uses, though, it would be much easier to call your script2.sh from a copy of some simple file in init.d. As it stands script2 is the startup script but doesn't conform to the init package you use.

That sounds more complicated that I mean... Let me explain better:

/etc/init.d/syslogd        ## a standard init script that calls syslogd
/etc/init.d/start-monitor   ## a copy of a standard init script that calls script2.sh

As an additional note, I think you could bind netcat to the specific IP that you are monitoring, instead of binding it to the all address 0.0.0.0

Answer 4

If none of your commands including netcat reads input from stdin you can completely make it run independent of the terminal. Sometimes background process that are still dependent on the terminal pauses (S) when they try to read input from it on a background. Actually since you're running a daemon, you should make sure that none of your commands reads input from it (terminal).

#!/bin/bash

set +o monitor # Make sure job control is disabled.

(
    : # Make sure the shell runs a subshell.
    exec netcat -lk -p 12345 | while read line  ## Use exec to overwrite the subshell.
    do
        match=$(echo $line | grep -c 'Keep-Alive')
        if [ $match -eq 1 ]; then
            [start a command]
        fi
    done
) <&- >&- 2>&- </dev/null &>/dev/null &

TASKPID=$!
sleep 1s ## Let the task initialize a bit before we disown it.
disown "$TASKPID"

And I think we could try the logging thing again:

set +o monitor

(
    echo "[$(date "+%F %T")] Starting loop with PID $BASHPID."

    for (( ;; ))
    do
        echo "[$(date "+%F %T")] Starting netcat."

        netcat -vv -lk -p 12345 | while read line
        do
            match=$(echo "$line" | grep -c 'Keep-Alive')
            if [ "$match" -eq 1 ]; then
                [start a command]
            fi
        done

        echo "[$(date "+%F %T")] Netcat has stopped or crashed."

        sleep 4s
    done
) <&- >&- 2>&- </dev/null >> "/var/log/something.log" 2>&1 &

TASKPID=$!
sleep 1s
disown "$TASKPID"

Answer 5

you may not use the -p option in the case you will wait for an incoming connect request. (see man page of nc) Hostname and Port are the last two arguments of the command line.

May be it connects to the own port and after some hours there is some resource missing??

Answer 6

I have often experienced strange behaviour with nc or netcat. You should have a look at ncat it's almost the same tool but it behaves the same on all platforms (nc and netcat behave differently depending on distri, linux, BSD, Mac).