Using getopts to process long and short command line options

Question

I wish to have long and short forms of command line options invoked using my shell script.

I know that getopts can be used, but like in Perl, I have not

Answer 1

I wanted something without external dependencies, with strict bash support (-u), and I needed it to work on even the older bash versions. This handles various types of params:

• short bools (-h)
• short options (-i "image.jpg")
• long bools (--help)
• equals options (--file="filename.ext")
• space options (--file "filename.ext")
• concatinated bools (-hvm)

Just insert the following at the top of your script:

# Check if a list of params contains a specific param
# usage: if _param_variant "h|?|help p|path f|file long-thing t|test-thing" "file" ; then ...
# the global variable $key is updated to the long notation (last entry in the pipe delineated list, if applicable)
_param_variant() {
  for param in $1 ; do
    local variants=${param//\|/ }
    for variant in $variants ; do
      if [[ "$variant" = "$2" ]] ; then
        # Update the key to match the long version
        local arr=(${param//\|/ })
        let last=${#arr[@]}-1
        key="${arr[$last]}"
        return 0
      fi
    done
  done
  return 1
}

# Get input parameters in short or long notation, with no dependencies beyond bash
# usage:
#     # First, set your defaults
#     param_help=false
#     param_path="."
#     param_file=false
#     param_image=false
#     param_image_lossy=true
#     # Define allowed parameters
#     allowed_params="h|?|help p|path f|file i|image image-lossy"
#     # Get parameters from the arguments provided
#     _get_params $*
#
# Parameters will be converted into safe variable names like:
#     param_help,
#     param_path,
#     param_file,
#     param_image,
#     param_image_lossy
#
# Parameters without a value like "-h" or "--help" will be treated as
# boolean, and will be set as param_help=true
#
# Parameters can accept values in the various typical ways:
#     -i "path/goes/here"
#     --image "path/goes/here"
#     --image="path/goes/here"
#     --image=path/goes/here
# These would all result in effectively the same thing:
#     param_image="path/goes/here"
#
# Concatinated short parameters (boolean) are also supported
#     -vhm is the same as -v -h -m
_get_params(){

  local param_pair
  local key
  local value
  local shift_count

  while : ; do
    # Ensure we have a valid param. Allows this to work even in -u mode.
    if [[ $# == 0 || -z $1 ]] ; then
      break
    fi

    # Split the argument if it contains "="
    param_pair=(${1//=/ })
    # Remove preceeding dashes
    key="${param_pair[0]#--}"

    # Check for concatinated boolean short parameters.
    local nodash="${key#-}"
    local breakout=false
    if [[ "$nodash" != "$key" && ${#nodash} -gt 1 ]]; then
      # Extrapolate multiple boolean keys in single dash notation. ie. "-vmh" should translate to: "-v -m -h"
      local short_param_count=${#nodash}
      let new_arg_count=$#+$short_param_count-1
      local new_args=""
      # $str_pos is the current position in the short param string $nodash
      for (( str_pos=0; str_pos<new_arg_count; str_pos++ )); do
        # The first character becomes the current key
        if [ $str_pos -eq 0 ] ; then
          key="${nodash:$str_pos:1}"
          breakout=true
        fi
        # $arg_pos is the current position in the constructed arguments list
        let arg_pos=$str_pos+1
        if [ $arg_pos -gt $short_param_count ] ; then
          # handle other arguments
          let orignal_arg_number=$arg_pos-$short_param_count+1
          local new_arg="${!orignal_arg_number}"
        else
          # break out our one argument into new ones
          local new_arg="-${nodash:$str_pos:1}"
        fi
        new_args="$new_args \"$new_arg\""
      done
      # remove the preceding space and set the new arguments
      eval set -- "${new_args# }"
    fi
    if ! $breakout ; then
      key="$nodash"
    fi

    # By default we expect to shift one argument at a time
    shift_count=1
    if [ "${#param_pair[@]}" -gt "1" ] ; then
      # This is a param with equals notation
      value="${param_pair[1]}"
    else
      # This is either a boolean param and there is no value,
      # or the value is the next command line argument
      # Assume the value is a boolean true, unless the next argument is found to be a value.
      value=true
      if [[ $# -gt 1 && -n "$2" ]]; then
        local nodash="${2#-}"
        if [ "$nodash" = "$2" ]; then
          # The next argument has NO preceding dash so it is a value
          value="$2"
          shift_count=2
        fi
      fi
    fi

    # Check that the param being passed is one of the allowed params
    if _param_variant "$allowed_params" "$key" ; then
      # --key-name will now become param_key_name
      eval param_${key//-/_}="$value"
    else
      printf 'WARNING: Unknown option (ignored): %s\n' "$1" >&2
    fi
    shift $shift_count
  done
}

And use it like so:

# Assign defaults for parameters
param_help=false
param_path=$(pwd)
param_file=false
param_image=true
param_image_lossy=true
param_image_lossy_quality=85

# Define the params we will allow
allowed_params="h|?|help p|path f|file i|image image-lossy image-lossy-quality"

# Get the params from arguments provided
_get_params $*

Answer 2

The accepted answer does a very nice job of pointing out all the shortcomings of bash built-in getopts. The answer ends with:

So while it is possible to write more code to work around the lack of support for long options, this is a lot more work and partially defeats the purpose of using a getopt parser to simplify your code.

And even though I agree in principle with that statement, I feel that the number of times we all implemented this feature in various scripts justifies putting a bit of effort into creating a "standardised", well tested solution.

As such, I've "upgraded" bash built in getopts by implementing getopts_long in pure bash, with no external dependencies. The usage of the function is 100% compatible with the built-in getopts.

By including getopts_long (which is hosted on GitHub) in a script, the answer to the original question can be implemented as simply as:

source "${PATH_TO}/getopts_long.bash"

while getopts_long ':c: copyfile:' OPTKEY; do
    case ${OPTKEY} in
        'c'|'copyfile')
            echo 'file supplied -- ${OPTARG}'
            ;;
        '?')
            echo "INVALID OPTION -- ${OPTARG}" >&2
            exit 1
            ;;
        ':')
            echo "MISSING ARGUMENT for option -- ${OPTARG}" >&2
            exit 1
            ;;
        *)
            echo "Misconfigured OPTSPEC or uncaught option -- ${OPTKEY}" >&2
            exit 1
            ;;
    esac
done

shift $(( OPTIND - 1 ))
[[ "${1}" == "--" ]] && shift

Answer 3

I kind of solved this way:

# A string with command options
options=$@

# An array with all the arguments
arguments=($options)

# Loop index
index=0

for argument in $options
  do
    # Incrementing index
    index=`expr $index + 1`

    # The conditions
    case $argument in
      -a) echo "key $argument value ${arguments[index]}" ;;
      -abc) echo "key $argument value ${arguments[index]}" ;;
    esac
  done

exit;

Am I being dumb or something? getopt and getopts are so confusing.

Answer 4

In order to stay cross-platform compatible, and avoid the reliance on external executables, I ported some code from another language.

I find it very easy to use, here is an example:

ArgParser::addArg "[h]elp"    false    "This list"
ArgParser::addArg "[q]uiet"   false    "Supress output"
ArgParser::addArg "[s]leep"   1        "Seconds to sleep"
ArgParser::addArg "v"         1        "Verbose mode"

ArgParser::parse "$@"

ArgParser::isset help && ArgParser::showArgs

ArgParser::isset "quiet" \
   && echo "Quiet!" \
   || echo "Noisy!"

local __sleep
ArgParser::tryAndGetArg sleep into __sleep \
   && echo "Sleep for $__sleep seconds" \
   || echo "No value passed for sleep"

# This way is often more convienient, but is a little slower
echo "Sleep set to: $( ArgParser::getArg sleep )"

The required BASH is a little longer than it could be, but I wanted to avoid reliance on BASH 4's associative arrays. You can also download this directly from http://nt4.com/bash/argparser.inc.sh

#!/usr/bin/env bash

# Updates to this script may be found at
# http://nt4.com/bash/argparser.inc.sh

# Example of runtime usage:
# mnc.sh --nc -q Caprica.S0*mkv *.avi *.mp3 --more-options here --host centos8.host.com

# Example of use in script (see bottom)
# Just include this file in yours, or use
# source argparser.inc.sh

unset EXPLODED
declare -a EXPLODED
function explode 
{
    local c=$# 
    (( c < 2 )) && 
    {
        echo function "$0" is missing parameters 
        return 1
    }

    local delimiter="$1"
    local string="$2"
    local limit=${3-99}

    local tmp_delim=$'\x07'
    local delin=${string//$delimiter/$tmp_delim}
    local oldifs="$IFS"

    IFS="$tmp_delim"
    EXPLODED=($delin)
    IFS="$oldifs"
}


# See: http://fvue.nl/wiki/Bash:_Passing_variables_by_reference
# Usage: local "$1" && upvar $1 "value(s)"
upvar() {
    if unset -v "$1"; then           # Unset & validate varname
        if (( $# == 2 )); then
            eval $1=\"\$2\"          # Return single value
        else
            eval $1=\(\"\${@:2}\"\)  # Return array
        fi
    fi
}

function decho
{
    :
}

function ArgParser::check
{
    __args=${#__argparser__arglist[@]}
    for (( i=0; i<__args; i++ ))
    do
        matched=0
        explode "|" "${__argparser__arglist[$i]}"
        if [ "${#1}" -eq 1 ]
        then
            if [ "${1}" == "${EXPLODED[0]}" ]
            then
                decho "Matched $1 with ${EXPLODED[0]}"
                matched=1

                break
            fi
        else
            if [ "${1}" == "${EXPLODED[1]}" ]
            then
                decho "Matched $1 with ${EXPLODED[1]}"
                matched=1

                break
            fi
        fi
    done
    (( matched == 0 )) && return 2
    # decho "Key $key has default argument of ${EXPLODED[3]}"
    if [ "${EXPLODED[3]}" == "false" ]
    then
        return 0
    else
        return 1
    fi
}

function ArgParser::set
{
    key=$3
    value="${1:-true}"
    declare -g __argpassed__$key="$value"
}

function ArgParser::parse
{

    unset __argparser__argv
    __argparser__argv=()
    # echo parsing: "$@"

    while [ -n "$1" ]
    do
        # echo "Processing $1"
        if [ "${1:0:2}" == '--' ]
        then
            key=${1:2}
            value=$2
        elif [ "${1:0:1}" == '-' ]
        then
            key=${1:1}               # Strip off leading -
            value=$2
        else
            decho "Not argument or option: '$1'" >& 2
            __argparser__argv+=( "$1" )
            shift
            continue
        fi
        # parameter=${tmp%%=*}     # Extract name.
        # value=${tmp##*=}         # Extract value.
        decho "Key: '$key', value: '$value'"
        # eval $parameter=$value
        ArgParser::check $key
        el=$?
        # echo "Check returned $el for $key"
        [ $el -eq  2 ] && decho "No match for option '$1'" >&2 # && __argparser__argv+=( "$1" )
        [ $el -eq  0 ] && decho "Matched option '${EXPLODED[2]}' with no arguments"        >&2 && ArgParser::set true "${EXPLODED[@]}"
        [ $el -eq  1 ] && decho "Matched option '${EXPLODED[2]}' with an argument of '$2'"   >&2 && ArgParser::set "$2" "${EXPLODED[@]}" && shift
        shift
    done
}

function ArgParser::isset
{
    declare -p "__argpassed__$1" > /dev/null 2>&1 && return 0
    return 1
}

function ArgParser::getArg
{
    # This one would be a bit silly, since we can only return non-integer arguments ineffeciently
    varname="__argpassed__$1"
    echo "${!varname}"
}

##
# usage: tryAndGetArg <argname> into <varname>
# returns: 0 on success, 1 on failure
function ArgParser::tryAndGetArg
{
    local __varname="__argpassed__$1"
    local __value="${!__varname}"
    test -z "$__value" && return 1
    local "$3" && upvar $3 "$__value"
    return 0
}

function ArgParser::__construct
{
    unset __argparser__arglist
    # declare -a __argparser__arglist
}

##
# @brief add command line argument
# @param 1 short and/or long, eg: [s]hort
# @param 2 default value
# @param 3 description
##
function ArgParser::addArg
{
    # check for short arg within long arg
    if [[ "$1" =~ \[(.)\] ]]
    then
        short=${BASH_REMATCH[1]}
        long=${1/\[$short\]/$short}
    else
        long=$1
    fi
    if [ "${#long}" -eq 1 ]
    then
        short=$long
        long=''
    fi
    decho short: "$short"
    decho long: "$long"
    __argparser__arglist+=("$short|$long|$1|$2|$3")
}

## 
# @brief show available command line arguments
##
function ArgParser::showArgs
{
    # declare -p | grep argparser
    printf "Usage: %s [OPTION...]\n\n" "$( basename "${BASH_SOURCE[0]}" )"
    printf "Defaults for the options are specified in brackets.\n\n";

    __args=${#__argparser__arglist[@]}
    for (( i=0; i<__args; i++ ))
    do
        local shortname=
        local fullname=
        local default=
        local description=
        local comma=

        explode "|" "${__argparser__arglist[$i]}"

        shortname="${EXPLODED[0]:+-${EXPLODED[0]}}" # String Substitution Guide: 
        fullname="${EXPLODED[1]:+--${EXPLODED[1]}}" # http://tldp.org/LDP/abs/html/parameter-substitution.html
        test -n "$shortname" \
            && test -n "$fullname" \
            && comma=","

        default="${EXPLODED[3]}"
        case $default in
            false )
                default=
                ;;
            "" )
                default=
                ;;
            * )
                default="[$default]"
        esac

        description="${EXPLODED[4]}"

        printf "  %2s%1s %-19s %s %s\n" "$shortname" "$comma" "$fullname" "$description" "$default"
    done
}

function ArgParser::test
{
    # Arguments with a default of 'false' do not take paramaters (note: default
    # values are not applied in this release)

    ArgParser::addArg "[h]elp"      false       "This list"
    ArgParser::addArg "[q]uiet" false       "Supress output"
    ArgParser::addArg "[s]leep" 1           "Seconds to sleep"
    ArgParser::addArg "v"           1           "Verbose mode"

    ArgParser::parse "$@"

    ArgParser::isset help && ArgParser::showArgs

    ArgParser::isset "quiet" \
        && echo "Quiet!" \
        || echo "Noisy!"

    local __sleep
    ArgParser::tryAndGetArg sleep into __sleep \
        && echo "Sleep for $__sleep seconds" \
        || echo "No value passed for sleep"

    # This way is often more convienient, but is a little slower
    echo "Sleep set to: $( ArgParser::getArg sleep )"

    echo "Remaining command line: ${__argparser__argv[@]}"

}

if [ "$( basename "$0" )" == "argparser.inc.sh" ]
then
    ArgParser::test "$@"
fi

Answer 5

Here's an example that actually uses getopt with long options:

aflag=no
bflag=no
cargument=none

# options may be followed by one colon to indicate they have a required argument
if ! options=$(getopt -o abc: -l along,blong,clong: -- "$@")
then
    # something went wrong, getopt will put out an error message for us
    exit 1
fi

set -- $options

while [ $# -gt 0 ]
do
    case $1 in
    -a|--along) aflag="yes" ;;
    -b|--blong) bflag="yes" ;;
    # for options with required arguments, an additional shift is required
    -c|--clong) cargument="$2" ; shift;;
    (--) shift; break;;
    (-*) echo "$0: error - unrecognized option $1" 1>&2; exit 1;;
    (*) break;;
    esac
    shift
done

Answer 6

Long options can be parsed by the standard getopts builtin as “arguments” to the - “option”

This is portable and native POSIX shell – no external programs or bashisms are needed.

This guide implements long options as arguments to the - option, so --alpha is seen by getopts as - with argument alpha and --bravo=foo is seen as - with argument bravo=foo. The true argument can be harvested with a simple replacement: ${OPTARG#*=}.

In this example, -b and -c (and their long forms, --bravo and --charlie) have mandatory arguments. Arguments to long options come after equals signs, e.g. --bravo=foo (space delimiters for long options would be hard to implement, see below).

Because this uses the getopts builtin, this solution supports usage like cmd --bravo=foo -ac FILE (which has combined options -a and -c and interleaves long options with standard options) while most other answers here either struggle or fail to do that.

die() { echo "$*" >&2; exit 2; }  # complain to STDERR and exit with error
needs_arg() { if [ -z "$OPTARG" ]; then die "No arg for --$OPT option"; fi; }

while getopts ab:c:-: OPT; do
  # support long options: https://stackoverflow.com/a/28466267/519360
  if [ "$OPT" = "-" ]; then   # long option: reformulate OPT and OPTARG
    OPT="${OPTARG%%=*}"       # extract long option name
    OPTARG="${OPTARG#$OPT}"   # extract long option argument (may be empty)
    OPTARG="${OPTARG#=}"      # if long option argument, remove assigning `=`
  fi
  case "$OPT" in
    a | alpha )    alpha=true ;;
    b | bravo )    needs_arg; bravo="$OPTARG" ;;
    c | charlie )  needs_arg; charlie="$OPTARG" ;;
    ??* )          die "Illegal option --$OPT" ;;  # bad long option
    ? )            exit 2 ;;  # bad short option (error reported via getopts)
  esac
done
shift $((OPTIND-1)) # remove parsed options and args from $@ list

When the option is a dash (-), it is a long option. getopts will have parsed the actual long option into $OPTARG, e.g. --bravo=foo originally sets OPT='-' and OPTARG='bravo=foo'. The if stanza sets $OPT to the contents of $OPTARG before the first equals sign (bravo in our example) and then removes that from the beginning of $OPTARG (yielding =foo in this step, or an empty string if there is no =). Finally, we strip the argument's leading =. At this point, $OPT is either a short option (one character) or a long option (2+ characters).

The case then matches either short or long options. For short options, getopts automatically complains about options and missing arguments, so we have to replicate those manually using the needs_arg function, which fatally exits when $OPTARG is empty. The ??* condition will match any remaining long option (? matches a single character and * matches zero or more, so ??* matches 2+ characters), allowing us to issue the "Illegal option" error before exiting.

Minor bug: if somebody gives an invalid single-character long option (and it's not also a short option), this will exit with an error but without a message. This is because this implementation assumes it was a short option. You could track that with an extra variable in the long option stanza preceding the case and then test for it in the final case condition, but I consider that too much of a corner case to bother.

(A note about all-uppercase variable names: Generally, the advice is to reserve all-uppercase variables for system use. I'm keeping $OPT as all-uppercase to keep it in line with $OPTARG, but this does break that convention. I think it fits because this is something the system should have done, and it should be safe because there are no standards (afaik) that use such a variable.)

---

To complain about unexpected arguments to long options, mimic what we did for mandatory arguments: use a helper function. Just flip the test around to complain about an argument when one isn't expected:

no_arg() { if [ -n "$OPTARG" ]; then die "No arg allowed for --$OPT option"; fi; }

---

An older version of this answer had an attempt at accepting long options with space-delimited arguments, but it was not reliable; getopts could prematurely terminate on the assumption that the argument was beyond its scope and manually incrementing $OPTIND doesn't work in all shells.

This would be accomplished using one of these techniques depending on your shell:

• POSIX eval: eval "bravo=\"\$$OPTIND\""
• Bash indirect expansion: bravo="${!OPTIND}"
• Zsh parameter expansion P flag: bravo="${(P)OPTIND}"

and then concluded with something like [ $# -gt $OPTIND ] && OPTIND=$((OPTIND+1))