How to add a progress bar to a shell script?

Question

When scripting in bash or any other shell in *NIX, while running a command that will take more than a few seconds, a progress bar is needed.

For example, copying a b

Answer 1

I used an answer from Creating string of repeated characters in shell script for char repeating. I have two relatively small bash versions for scripts that need to display progress bar (for example, a loop that goes through many files, but not useful for big tar files or copy operations). The faster one consists of two functions, one to prepare the strings for bar display:

preparebar() {
# $1 - bar length
# $2 - bar char
    barlen=$1
    barspaces=$(printf "%*s" "$1")
    barchars=$(printf "%*s" "$1" | tr ' ' "$2")
}

and one to display a progress bar:

progressbar() {
# $1 - number (-1 for clearing the bar)
# $2 - max number
    if [ $1 -eq -1 ]; then
        printf "\r  $barspaces\r"
    else
        barch=$(($1*barlen/$2))
        barsp=$((barlen-barch))
        printf "\r[%.${barch}s%.${barsp}s]\r" "$barchars" "$barspaces"
    fi
}

It could be used as:

preparebar 50 "#"

which means prepare strings for bar with 50 "#" characters, and after that:

progressbar 35 80

will display the number of "#" characters that corresponds to 35/80 ratio:

[#####################                             ]

Be aware that function displays the bar on the same line over and over until you (or some other program) prints a newline. If you put -1 as first parameter, the bar would be erased:

progressbar -1 80

The slower version is all in one function:

progressbar() {
# $1 - number
# $2 - max number
# $3 - number of '#' characters
    if [ $1 -eq -1 ]; then
        printf "\r  %*s\r" "$3"
    else
        i=$(($1*$3/$2))
        j=$(($3-i))
        printf "\r[%*s" "$i" | tr ' ' '#'
        printf "%*s]\r" "$j"
    fi
}

and it can be used as (the same example as above):

progressbar 35 80 50

If you need progressbar on stderr, just add >&2 at the end of each printf command.

Answer 2

Many answers describe writing your own commands for printing out '\r' + $some_sort_of_progress_msg. The problem sometimes is that printing out hundreds of these updates per second will slow down the process.

However, if any of your processes produce output (eg 7z a -r newZipFile myFolder will output each filename as it compresses it) then a simpler, fast, painless and customisable solution exists.

Install the python module tqdm.

$ sudo pip install tqdm
$ # now have fun
$ 7z a -r -bd newZipFile myFolder | tqdm >> /dev/null
$ # if we know the expected total, we can have a bar!
$ 7z a -r -bd newZipFile myFolder | grep -o Compressing | tqdm --total $(find myFolder -type f | wc -l) >> /dev/null

Help: tqdm -h. An example using more options:

$ find / -name '*.py' -exec cat \{} \; | tqdm --unit loc --unit_scale True | wc -l

As a bonus you can also use tqdm to wrap iterables in python code.

https://github.com/tqdm/tqdm/blob/master/README.rst#module

Answer 3

for me easiest to use and best looking so far is command pv or bar like some guy already wrote

for example: need to make a backup of entire drive with dd

normally you use dd if="$input_drive_path" of="$output_file_path"

with pv you can make it like this :

dd if="$input_drive_path" | pv | dd of="$output_file_path"

and the progress goes directly to STDOUT as this:

    7.46GB 0:33:40 [3.78MB/s] [  <=>                                            ]

after it is done summary comes up

    15654912+0 records in
    15654912+0 records out
    8015314944 bytes (8.0 GB) copied, 2020.49 s, 4.0 MB/s

Answer 4

Based on the work of Edouard Lopez, I created a progress bar that fits the size of the screen, whatever it is. Check it out.

It's also posted on Git Hub.

#!/bin/bash
#
# Progress bar by Adriano Pinaffo
# Available at https://github.com/adriano-pinaffo/progressbar.sh
# Inspired on work by Edouard Lopez (https://github.com/edouard-lopez/progress-bar.sh)
# Version 1.0
# Date April, 28th 2017

function error {
  echo "Usage: $0 [SECONDS]"
  case $1 in
    1) echo "Pass one argument only"
    exit 1
    ;;
    2) echo "Parameter must be a number"
    exit 2
    ;;
    *) echo "Unknown error"
    exit 999
  esac
}

[[ $# -ne 1 ]] && error 1
[[ $1 =~ ^[0-9]+$ ]] || error 2

duration=${1}
barsize=$((`tput cols` - 7))
unity=$(($barsize / $duration))
increment=$(($barsize%$duration))
skip=$(($duration/($duration-$increment)))
curr_bar=0
prev_bar=
for (( elapsed=1; elapsed<=$duration; elapsed++ ))
do
  # Elapsed
prev_bar=$curr_bar
  let curr_bar+=$unity
  [[ $increment -eq 0 ]] || {  
    [[ $skip -eq 1 ]] &&
      { [[ $(($elapsed%($duration/$increment))) -eq 0 ]] && let curr_bar++; } ||
    { [[ $(($elapsed%$skip)) -ne 0 ]] && let curr_bar++; }
  }
  [[ $elapsed -eq 1 && $increment -eq 1 && $skip -ne 1 ]] && let curr_bar++
  [[ $(($barsize-$curr_bar)) -eq 1 ]] && let curr_bar++
  [[ $curr_bar -lt $barsize ]] || curr_bar=$barsize
  for (( filled=0; filled<=$curr_bar; filled++ )); do
    printf "▇"
  done

  # Remaining
  for (( remain=$curr_bar; remain<$barsize; remain++ )); do
    printf " "
  done

  # Percentage
  printf "| %s%%" $(( ($elapsed*100)/$duration))

  # Return
  sleep 1
  printf "\r"
done
printf "\n"
exit 0

Enjoy

Answer 5

I prefer to use dialog with the --gauge param. Is used very often in .deb package installations and other basic configuration stuff of many distros. So you don't need to reinvent the wheel... again

Just put an int value from 1 to 100 @stdin. One basic and silly example:

for a in {1..100}; do sleep .1s; echo $a| dialog --gauge "waiting" 7 30; done

I have this /bin/Wait file (with chmod u+x perms) for cooking purposes :P

#!/bin/bash
INIT=`/bin/date +%s`
NOW=$INIT
FUTURE=`/bin/date -d "$1" +%s`
[ $FUTURE -a $FUTURE -eq $FUTURE ] || exit
DIFF=`echo "$FUTURE - $INIT"|bc -l`

while [ $INIT -le $FUTURE -a $NOW -lt $FUTURE ]; do
    NOW=`/bin/date +%s`
    STEP=`echo "$NOW - $INIT"|bc -l`
    SLEFT=`echo "$FUTURE - $NOW"|bc -l`
    MLEFT=`echo "scale=2;$SLEFT/60"|bc -l`
    TEXT="$SLEFT seconds left ($MLEFT minutes)";
    TITLE="Waiting $1: $2"
    sleep 1s
    PTG=`echo "scale=0;$STEP * 100 / $DIFF"|bc -l`
    echo $PTG| dialog --title "$TITLE" --gauge "$TEXT" 7 72
done

if [ "$2" == "" ]; then msg="Espera terminada: $1";audio="Listo";
else msg=$2;audio=$2;fi 

/usr/bin/notify-send --icon=stock_appointment-reminder-excl "$msg"
espeak -v spanish "$audio"

So I can put:

Wait "34 min" "warm up the oven"

or

Wait "dec 31" "happy new year"

Answer 6

I did a pure shell version for an embedded system taking advantage of:

• /usr/bin/dd's SIGUSR1 signal handling feature.

  Basically, if you send a 'kill SIGUSR1 $(pid_of_running_dd_process)', it'll output a summary of throughput speed and amount transferred.

• backgrounding dd and then querying it regularly for updates, and generating hash ticks like old-school ftp clients used to.

• Using /dev/stdout as the destination for non-stdout friendly programs like scp

The end result allows you to take any file transfer operation and get progress update that looks like old-school FTP 'hash' output where you'd just get a hash mark for every X bytes.

This is hardly production quality code, but you get the idea. I think it's cute.

For what it's worth, the actual byte-count might not be reflected correctly in the number of hashes - you may have one more or less depending on rounding issues. Don't use this as part of a test script, it's just eye-candy. And, yes, I'm aware this is terribly inefficient - it's a shell script and I make no apologies for it.

Examples with wget, scp and tftp provided at the end. It should work with anything that has emits data. Make sure to use /dev/stdout for programs that aren't stdout friendly.

#!/bin/sh
#
# Copyright (C) Nathan Ramella (nar+progress-script@remix.net) 2010 
# LGPLv2 license
# If you use this, send me an email to say thanks and let me know what your product
# is so I can tell all my friends I'm a big man on the internet!

progress_filter() {

        local START=$(date +"%s")
        local SIZE=1
        local DURATION=1
        local BLKSZ=51200
        local TMPFILE=/tmp/tmpfile
        local PROGRESS=/tmp/tftp.progress
        local BYTES_LAST_CYCLE=0
        local BYTES_THIS_CYCLE=0

        rm -f ${PROGRESS}

        dd bs=$BLKSZ of=${TMPFILE} 2>&1 \
                | grep --line-buffered -E '[[:digit:]]* bytes' \
                | awk '{ print $1 }' >> ${PROGRESS} &

        # Loop while the 'dd' exists. It would be 'more better' if we
        # actually looked for the specific child ID of the running 
        # process by identifying which child process it was. If someone
        # else is running dd, it will mess things up.

        # My PID handling is dumb, it assumes you only have one running dd on
        # the system, this should be fixed to just get the PID of the child
        # process from the shell.

        while [ $(pidof dd) -gt 1 ]; do

                # PROTIP: You can sleep partial seconds (at least on linux)
                sleep .5    

                # Force dd to update us on it's progress (which gets
                # redirected to $PROGRESS file.
                # 
                # dumb pid handling again
                pkill -USR1 dd

                local BYTES_THIS_CYCLE=$(tail -1 $PROGRESS)
                local XFER_BLKS=$(((BYTES_THIS_CYCLE-BYTES_LAST_CYCLE)/BLKSZ))

                # Don't print anything unless we've got 1 block or more.
                # This allows for stdin/stderr interactions to occur
                # without printing a hash erroneously.

                # Also makes it possible for you to background 'scp',
                # but still use the /dev/stdout trick _even_ if scp
                # (inevitably) asks for a password. 
                #
                # Fancy!

                if [ $XFER_BLKS -gt 0 ]; then
                        printf "#%0.s" $(seq 0 $XFER_BLKS)
                        BYTES_LAST_CYCLE=$BYTES_THIS_CYCLE
                fi
        done

        local SIZE=$(stat -c"%s" $TMPFILE)
        local NOW=$(date +"%s")

        if [ $NOW -eq 0 ]; then
                NOW=1
        fi

        local DURATION=$(($NOW-$START))
        local BYTES_PER_SECOND=$(( SIZE / DURATION ))
        local KBPS=$((SIZE/DURATION/1024))
        local MD5=$(md5sum $TMPFILE | awk '{ print $1 }')

        # This function prints out ugly stuff suitable for eval() 
        # rather than a pretty string. This makes it a bit more 
        # flexible if you have a custom format (or dare I say, locale?)

        printf "\nDURATION=%d\nBYTES=%d\nKBPS=%f\nMD5=%s\n" \
            $DURATION \
            $SIZE \
            $KBPS \
            $MD5
}

Examples:

echo "wget"
wget -q -O /dev/stdout http://www.blah.com/somefile.zip | progress_filter

echo "tftp"
tftp -l /dev/stdout -g -r something/firmware.bin 192.168.1.1 | progress_filter

echo "scp"
scp user@192.168.1.1:~/myfile.tar /dev/stdout | progress_filter