Log rotating with a Bash script

Question

I have the following issue:

I have an application, which continuously produces output to stderr and stdout. The output of this application is captured in a logfile (the

Answer 1

Okay, here's an idea, inspired by http://en.wikibooks.org/wiki/Bourne_Shell_Scripting/Files_and_streams

1. make a named pipe:

   mkfifo /dev/mypipe
   

2. redirect stdout and stderr to the named pipe:

   &> /dev/mypipe
   

3. read from mypipe into a file:

   cat < /dev/mypipe > /var/log/log.txt &
   

4. when you need to log-rotate, kill the cat, rotate the log, and restart the cat.

Now, I haven't tested this. Tell us how it goes.

Note: you can give the named pipe any name, like /var/tmp/pipe1 , /var/log/pipe , /tmp/abracadabra , and so on. Just make sure to re-create the pipe after booting before your logging-script runs.

---

Alternatively, don't use cat, but use a simple script file:

#!/bin/bash

while : ; do
  read line
  printf "%s\n" "$line"
done

This script guarantees an output for every newline read. (cat might not start outputting until its buffer is full or it encounters an EOF)

---

Final -- and TESTED -- attempt

IMPORTANT NOTE: Please read the comments from @andrew below. There are several situations which you need to be aware of.

Alright! Finally got access to my Linux box. Here's how:

Step 1: Make this recorder script:

#!/bin/bash

LOGFILE="/path/to/log/file"
SEMAPHORE="/path/to/log/file.semaphore"

while : ; do
  read line
  while [[ -f $SEMAPHORE ]]; do
    sleep 1s
  done
  printf "%s\n" "$line" >> $LOGFILE
done

Step 2: put the recorder into work:

1. Make a named pipe:

   mkfifo $PIPENAME
   

2. Redirect your application's STDOUT & STDERR to the named pipe:

   ...things... &> $PIPENAME
   

3. Start the recorder:

   /path/to/recorder.sh < $PIPENAME &
   

   You might want to nohup the above to make it survive logouts.

4. Done!

Step 3: If you need to logrotate, pause the recorder:

touch /path/to/log/file.semaphore
mv /path/to/log/file /path/to/archive/of/log/file
rm /path/to/log/file.semaphore

I suggest putting the above steps into its own script. Feel free to change the 2nd line to whatever log-rotating method you want to use.

---

Note : If you're handy with C programming, you might want to make a short C program to perform the function of recorder.sh. Compiled C programs will certainly be lighter than a nohup-ed detached bash script.

---

Note 2: David Newcomb provided a helpful warning in the comments: While the recorder is not running then writes to the pipe will block and may cause the program to fail unpredictably. Make sure the recorder is down (or rotating) for as short time as possible.

So, if you can ensure that rotating happens really quickly, you can replace sleep (a built-in command which accepts only integer values) with /bin/sleep (a program that accepts float values) and set the sleep period to 0.5 or shorter.