Using getopts to process long and short command line options

Question

I wish to have long and short forms of command line options invoked using my shell script.

I know that getopts can be used, but like in Perl, I have not

Answer 1

I only write shell scripts now and then and fall out of practice, so any feedback is appreciated.

Using the strategy proposed by @Arvid Requate, we noticed some user errors. A user who forgets to include a value will accidentally have the next option's name treated as a value:

./getopts_test.sh --loglevel= --toc=TRUE

will cause the value of "loglevel" to be seen as "--toc=TRUE". This can be avoided.

I adapted some ideas about checking user error for CLI from http://mwiki.wooledge.org/BashFAQ/035 discussion of manual parsing. I inserted error checking into handling both "-" and "--" arguments.

Then I started fiddling around with the syntax, so any errors in here are strictly my fault, not the original authors.

My approach helps users who prefer to enter long with or without the equal sign. That is, it should have same response to "--loglevel 9" as "--loglevel=9". In the --/space method, it is not possible to know for sure if the user forgets an argument, so some guessing is needed.

1. if the user has the long/equal sign format (--opt=), then a space after = triggers an error because an argument was not supplied.
2. if user has long/space arguments (--opt ), this script causes a fail if no argument follows (end of command) or if argument begins with dash)

In case you are starting out on this, there is an interesting difference between "--opt=value" and "--opt value" formats. With the equal sign, the command line argument is seen as "opt=value" and the work to handle that is string parsing, to separate at the "=". In contrast, with "--opt value", the name of the argument is "opt" and we have the challenge of getting the next value supplied in the command line. That's where @Arvid Requate used ${!OPTIND}, the indirect reference. I still don't understand that, well, at all, and comments in BashFAQ seem to warn against that style (http://mywiki.wooledge.org/BashFAQ/006). BTW, I don't think previous poster's comments about importance of OPTIND=$(( $OPTIND + 1 )) are correct. I mean to say, I see no harm from omitting it.

In newest version of this script, flag -v means VERBOSE printout.

Save it in a file called "cli-5.sh", make executable, and any of these will work, or fail in the desired way

./cli-5.sh  -v --loglevel=44 --toc  TRUE
./cli-5.sh  -v --loglevel=44 --toc=TRUE
./cli-5.sh --loglevel 7
./cli-5.sh --loglevel=8
./cli-5.sh -l9

./cli-5.sh  --toc FALSE --loglevel=77
./cli-5.sh  --toc=FALSE --loglevel=77

./cli-5.sh   -l99 -t yyy
./cli-5.sh   -l 99 -t yyy

Here is example output of the error-checking on user intpu

$ ./cli-5.sh  --toc --loglevel=77
ERROR: toc value must not have dash at beginning
$ ./cli-5.sh  --toc= --loglevel=77
ERROR: value for toc undefined

You should consider turning on -v, because it prints out internals of OPTIND and OPTARG

#/usr/bin/env bash

## Paul Johnson
## 20171016
##

## Combines ideas from
## https://stackoverflow.com/questions/402377/using-getopts-in-bash-shell-script-to-get-long-and-short-command-line-options
## by @Arvid Requate, and http://mwiki.wooledge.org/BashFAQ/035

# What I don't understand yet: 
# In @Arvid REquate's answer, we have 
# val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
# this works, but I don't understand it!


die() {
    printf '%s\n' "$1" >&2
    exit 1
}

printparse(){
    if [ ${VERBOSE} -gt 0 ]; then
        printf 'Parse: %s%s%s\n' "$1" "$2" "$3" >&2;
    fi
}

showme(){
    if [ ${VERBOSE} -gt 0 ]; then
        printf 'VERBOSE: %s\n' "$1" >&2;
    fi
}


VERBOSE=0
loglevel=0
toc="TRUE"

optspec=":vhl:t:-:"
while getopts "$optspec" OPTCHAR; do

    showme "OPTARG:  ${OPTARG[*]}"
    showme "OPTIND:  ${OPTIND[*]}"
    case "${OPTCHAR}" in
        -)
            case "${OPTARG}" in
                loglevel) #argument has no equal sign
                    opt=${OPTARG}
                    val="${!OPTIND}"
                    ## check value. If negative, assume user forgot value
                    showme "OPTIND is {$OPTIND} {!OPTIND} has value \"${!OPTIND}\""
                    if [[ "$val" == -* ]]; then
                        die "ERROR: $opt value must not have dash at beginning"
                    fi
                    ## OPTIND=$(( $OPTIND + 1 )) # CAUTION! no effect?
                    printparse "--${OPTARG}" "  " "${val}"
                    loglevel="${val}"
                    shift
                    ;;
                loglevel=*) #argument has equal sign
                    opt=${OPTARG%=*}
                    val=${OPTARG#*=}
                    if [ "${OPTARG#*=}" ]; then
                        printparse "--${opt}" "=" "${val}"
                        loglevel="${val}"
                        ## shift CAUTION don't shift this, fails othewise
                    else
                        die "ERROR: $opt value must be supplied"
                    fi
                    ;;
                toc) #argument has no equal sign
                    opt=${OPTARG}
                    val="${!OPTIND}"
                    ## check value. If negative, assume user forgot value
                    showme "OPTIND is {$OPTIND} {!OPTIND} has value \"${!OPTIND}\""
                    if [[ "$val" == -* ]]; then
                        die "ERROR: $opt value must not have dash at beginning"
                    fi
                    ## OPTIND=$(( $OPTIND + 1 )) #??
                    printparse "--${opt}" " " "${val}"
                    toc="${val}"
                    shift
                    ;;
                toc=*) #argument has equal sign
                    opt=${OPTARG%=*}
                    val=${OPTARG#*=}
                    if [ "${OPTARG#*=}" ]; then
                        toc=${val}
                        printparse "--$opt" " -> " "$toc"
                        ##shift ## NO! dont shift this
                    else
                        die "ERROR: value for $opt undefined"
                    fi
                    ;;

                help)
                    echo "usage: $0 [-v] [--loglevel[=]<value>] [--toc[=]<TRUE,FALSE>]" >&2
                    exit 2
                    ;;
                *)
                    if [ "$OPTERR" = 1 ] && [ "${optspec:0:1}" != ":" ]; then
                        echo "Unknown option --${OPTARG}" >&2
                    fi
                    ;;
            esac;;
        h|-\?|--help)
            ## must rewrite this for all of the arguments
            echo "usage: $0 [-v] [--loglevel[=]<value>] [--toc[=]<TRUE,FALSE>]" >&2
            exit 2
            ;;
        l)
            loglevel=${OPTARG}
            printparse "-l" " "  "${loglevel}"
            ;;
        t)
            toc=${OPTARG}
            ;;
        v)
            VERBOSE=1
            ;;

        *)
            if [ "$OPTERR" != 1 ] || [ "${optspec:0:1}" = ":" ]; then
                echo "Non-option argument: '-${OPTARG}'" >&2
            fi
            ;;
    esac
done



echo "
After Parsing values
"
echo "loglevel  $loglevel" 
echo "toc  $toc"

Answer 2

The Bash builtin getopts function can be used to parse long options by putting a dash character followed by a colon into the optspec:

#!/usr/bin/env bash 
optspec=":hv-:"
while getopts "$optspec" optchar; do
    case "${optchar}" in
        -)
            case "${OPTARG}" in
                loglevel)
                    val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
                    echo "Parsing option: '--${OPTARG}', value: '${val}'" >&2;
                    ;;
                loglevel=*)
                    val=${OPTARG#*=}
                    opt=${OPTARG%=$val}
                    echo "Parsing option: '--${opt}', value: '${val}'" >&2
                    ;;
                *)
                    if [ "$OPTERR" = 1 ] && [ "${optspec:0:1}" != ":" ]; then
                        echo "Unknown option --${OPTARG}" >&2
                    fi
                    ;;
            esac;;
        h)
            echo "usage: $0 [-v] [--loglevel[=]<value>]" >&2
            exit 2
            ;;
        v)
            echo "Parsing option: '-${optchar}'" >&2
            ;;
        *)
            if [ "$OPTERR" != 1 ] || [ "${optspec:0:1}" = ":" ]; then
                echo "Non-option argument: '-${OPTARG}'" >&2
            fi
            ;;
    esac
done

After copying to executable file name=getopts_test.sh in the current working directory, one can produce output like

$ ./getopts_test.sh
$ ./getopts_test.sh -f
Non-option argument: '-f'
$ ./getopts_test.sh -h
usage: code/getopts_test.sh [-v] [--loglevel[=]<value>]
$ ./getopts_test.sh --help
$ ./getopts_test.sh -v
Parsing option: '-v'
$ ./getopts_test.sh --very-bad
$ ./getopts_test.sh --loglevel
Parsing option: '--loglevel', value: ''
$ ./getopts_test.sh --loglevel 11
Parsing option: '--loglevel', value: '11'
$ ./getopts_test.sh --loglevel=11
Parsing option: '--loglevel', value: '11'

Obviously getopts neither performs OPTERR checking nor option-argument parsing for the long options. The script fragment above shows how this may be done manually. The basic principle also works in the Debian Almquist shell ("dash"). Note the special case:

getopts -- "-:"  ## without the option terminator "-- " bash complains about "-:"
getopts "-:"     ## this works in the Debian Almquist shell ("dash")

Note that, as GreyCat from over at http://mywiki.wooledge.org/BashFAQ points out, this trick exploits a non-standard behaviour of the shell which permits the option-argument (i.e. the filename in "-f filename") to be concatenated to the option (as in "-ffilename"). The POSIX standard says there must be a space between them, which in the case of "-- longoption" would terminate the option-parsing and turn all longoptions into non-option arguments.

Answer 3

Inventing yet another version of the wheel...

This function is a (hopefully) POSIX-compatible plain bourne shell replacement for GNU getopt. It supports short/long options which can accept mandatory/optional/no arguments, and the way in which options are specified is almost identical to GNU getopt, so conversion is trivial.

Of course this is still a sizeable chunk of code to drop into a script, but it's about half the lines of the well-known getopt_long shell function, and might be preferable in cases where you just want to replace existing GNU getopt uses.

This is pretty new code, so YMMV (and definitely please let me know if this isn't actually POSIX-compatible for any reason -- portability was the intention from the outset, but I don't have a useful POSIX test environment).

Code and example usage follows:

#!/bin/sh
# posix_getopt shell function
# Author: Phil S.
# Version: 1.0
# Created: 2016-07-05
# URL: http://stackoverflow.com/a/37087374/324105

# POSIX-compatible argument quoting and parameter save/restore
# http://www.etalabs.net/sh_tricks.html
# Usage:
# parameters=$(save "$@") # save the original parameters.
# eval "set -- ${parameters}" # restore the saved parameters.
save () {
    local param
    for param; do
        printf %s\\n "$param" \
            | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/"
    done
    printf %s\\n " "
}

# Exit with status $1 after displaying error message $2.
exiterr () {
    printf %s\\n "$2" >&2
    exit $1
}

# POSIX-compatible command line option parsing.
# This function supports long options and optional arguments, and is
# a (largely-compatible) drop-in replacement for GNU getopt.
#
# Instead of:
# opts=$(getopt -o "$shortopts" -l "$longopts" -- "$@")
# eval set -- ${opts}
#
# We instead use:
# opts=$(posix_getopt "$shortopts" "$longopts" "$@")
# eval "set -- ${opts}"
posix_getopt () { # args: "$shortopts" "$longopts" "$@"
    local shortopts longopts \
          arg argtype getopt nonopt opt optchar optword suffix

    shortopts="$1"
    longopts="$2"
    shift 2

    getopt=
    nonopt=
    while [ $# -gt 0 ]; do
        opt=
        arg=
        argtype=
        case "$1" in
            # '--' means don't parse the remaining options
            ( -- ) {
                getopt="${getopt}$(save "$@")"
                shift $#
                break
            };;
            # process short option
            ( -[!-]* ) {         # -x[foo]
                suffix=${1#-?}   # foo
                opt=${1%$suffix} # -x
                optchar=${opt#-} # x
                case "${shortopts}" in
                    ( *${optchar}::* ) { # optional argument
                        argtype=optional
                        arg="${suffix}"
                        shift
                    };;
                    ( *${optchar}:* ) { # required argument
                        argtype=required
                        if [ -n "${suffix}" ]; then
                            arg="${suffix}"
                            shift
                        else
                            case "$2" in
                                ( -* ) exiterr 1 "$1 requires an argument";;
                                ( ?* ) arg="$2"; shift 2;;
                                (  * ) exiterr 1 "$1 requires an argument";;
                            esac
                        fi
                    };;
                    ( *${optchar}* ) { # no argument
                        argtype=none
                        arg=
                        shift
                        # Handle multiple no-argument parameters combined as
                        # -xyz instead of -x -y -z. If we have just shifted
                        # parameter -xyz, we now replace it with -yz (which
                        # will be processed in the next iteration).
                        if [ -n "${suffix}" ]; then
                            eval "set -- $(save "-${suffix}")$(save "$@")"
                        fi
                    };;
                    ( * ) exiterr 1 "Unknown option $1";;
                esac
            };;
            # process long option
            ( --?* ) {            # --xarg[=foo]
                suffix=${1#*=}    # foo (unless there was no =)
                if [ "${suffix}" = "$1" ]; then
                    suffix=
                fi
                opt=${1%=$suffix} # --xarg
                optword=${opt#--} # xarg
                case ",${longopts}," in
                    ( *,${optword}::,* ) { # optional argument
                        argtype=optional
                        arg="${suffix}"
                        shift
                    };;
                    ( *,${optword}:,* ) { # required argument
                        argtype=required
                        if [ -n "${suffix}" ]; then
                            arg="${suffix}"
                            shift
                        else
                            case "$2" in
                                ( -* ) exiterr 1 \
                                       "--${optword} requires an argument";;
                                ( ?* ) arg="$2"; shift 2;;
                                (  * ) exiterr 1 \
                                       "--${optword} requires an argument";;
                            esac
                        fi
                    };;
                    ( *,${optword},* ) { # no argument
                        if [ -n "${suffix}" ]; then
                            exiterr 1 "--${optword} does not take an argument"
                        fi
                        argtype=none
                        arg=
                        shift
                    };;
                    ( * ) exiterr 1 "Unknown option $1";;
                esac
            };;
            # any other parameters starting with -
            ( -* ) exiterr 1 "Unknown option $1";;
            # remember non-option parameters
            ( * ) nonopt="${nonopt}$(save "$1")"; shift;;
        esac

        if [ -n "${opt}" ]; then
            getopt="${getopt}$(save "$opt")"
            case "${argtype}" in
                ( optional|required ) {
                    getopt="${getopt}$(save "$arg")"
                };;
            esac
        fi
    done

    # Generate function output, suitable for:
    # eval "set -- $(posix_getopt ...)"
    printf %s "${getopt}"
    if [ -n "${nonopt}" ]; then
        printf %s "$(save "--")${nonopt}"
    fi
}

Example usage:

# Process command line options
shortopts="hvd:c::s::L:D"
longopts="help,version,directory:,client::,server::,load:,delete"
#opts=$(getopt -o "$shortopts" -l "$longopts" -n "$(basename $0)" -- "$@")
opts=$(posix_getopt "$shortopts" "$longopts" "$@")
if [ $? -eq 0 ]; then
    #eval set -- ${opts}
    eval "set -- ${opts}"
    while [ $# -gt 0 ]; do
        case "$1" in
            ( --                ) shift; break;;
            ( -h|--help         ) help=1; shift; break;;
            ( -v|--version      ) version_help=1; shift; break;;
            ( -d|--directory    ) dir=$2; shift 2;;
            ( -c|--client       ) useclient=1; client=$2; shift 2;;
            ( -s|--server       ) startserver=1; server_name=$2; shift 2;;
            ( -L|--load         ) load=$2; shift 2;;
            ( -D|--delete       ) delete=1; shift;;
        esac
    done
else
    shorthelp=1 # getopt returned (and reported) an error.
fi

Answer 4

I don't have enough rep yet to comment or vote his solution up, but sme's answer worked extremely well for me. The only issue I ran into was that the arguments end up wrapped in single-quotes (so I have an strip them out).

I also added some example usages and HELP text. I'll included my slightly extended version here:

#!/bin/bash

# getopt example
# from: https://stackoverflow.com/questions/402377/using-getopts-in-bash-shell-script-to-get-long-and-short-command-line-options
HELP_TEXT=\
"   USAGE:\n
    Accepts - and -- flags, can specify options that require a value, and can be in any order. A double-hyphen (--) will stop processing options.\n\n

    Accepts the following forms:\n\n

    getopt-example.sh -a -b -c value-for-c some-arg\n
    getopt-example.sh -c value-for-c -a -b some-arg\n
    getopt-example.sh -abc some-arg\n
    getopt-example.sh --along --blong --clong value-for-c -a -b -c some-arg\n
    getopt-example.sh some-arg --clong value-for-c\n
    getopt-example.sh
"

aflag=false
bflag=false
cargument=""

# options may be followed by one colon to indicate they have a required argument
if ! options=$(getopt -o abc:h\? -l along,blong,help,clong: -- "$@")
then
    # something went wrong, getopt will put out an error message for us
    exit 1
fi

set -- $options

while [ $# -gt 0 ]
do
    case $1 in
    -a|--along) aflag=true ;;
    -b|--blong) bflag=true ;;
    # for options with required arguments, an additional shift is required
    -c|--clong) cargument="$2" ; shift;;
    -h|--help|-\?) echo -e $HELP_TEXT; exit;;
    (--) shift; break;;
    (-*) echo "$0: error - unrecognized option $1" 1>&2; exit 1;;
    (*) break;;
    esac
    shift
done

# to remove the single quotes around arguments, pipe the output into:
# | sed -e "s/^'\\|'$//g"  (just leading/trailing) or | sed -e "s/'//g"  (all)

echo aflag=${aflag}
echo bflag=${bflag}
echo cargument=${cargument}

while [ $# -gt 0 ]
do
    echo arg=$1
    shift

    if [[ $aflag == true ]]; then
        echo a is true
    fi

done

Answer 5

I have been working on that subject for quite a long time... and made my own library which you will need to source in your main script. See libopt4shell and cd2mpc for an example. Hope it helps !

Answer 6

if simply this is how you want to call the script

myscript.sh --input1 "ABC" --input2 "PQR" --input2 "XYZ"

then you can follow this simplest way to achieve it with the help of getopt and --longoptions

try this , hope this is useful

# Read command line options
ARGUMENT_LIST=(
    "input1"
    "input2"
    "input3"
)



# read arguments
opts=$(getopt \
    --longoptions "$(printf "%s:," "${ARGUMENT_LIST[@]}")" \
    --name "$(basename "$0")" \
    --options "" \
    -- "$@"
)


echo $opts

eval set --$opts

while true; do
    case "$1" in
    --input1)  
        shift
        empId=$1
        ;;
    --input2)  
        shift
        fromDate=$1
        ;;
    --input3)  
        shift
        toDate=$1
        ;;
      --)
        shift
        break
        ;;
    esac
    shift
done