Can Python's argparse permute argument order like gnu getopt?

Question

GNU getopt, and command line tools that use it, allow options and arguments to be interleaved, known as permuting options (see http://www.gnu.org/software/libc/manual/html_node/Using-Getopt.html#Using-Getopt). Perl's Getopt::Long module also supports this (with qw(:config gnu_getopt)). argparse seems to not support (or even mention) permuting options.

There are many SO questions related to arg/opt order, but none seem answer this question: Can argparse be made to permute argument order like getopt?

The use case is a prototypical command line signature like GNU sort:

sort [opts] [files]

in which 1) options and files are permuted, and 2) the file list may contain zero or more arguments.

For example:

import argparse
p = argparse.ArgumentParser();
p.add_argument('files',nargs='*',default=['-']);
p.add_argument('-z',action='store_true')

p.parse_args(['-z','bar','foo']) # ok
p.parse_args(['bar','foo','-z']) # ok
p.parse_args(['bar','-z','foo']) # not okay
usage: ipython [-h] [-z] [files [files ...]]

I've tried:

• p.parse_known_args -- doesn't complain, but doesn't actually permute either and it doesn't balk about arguments that look like invalid options (e.g., --bogus or -b above).
• p.add_argument('files',nargs=argparse.REMAINDER) -- option -z is included in files unless before positional args
• p.add_argument('files',nargs='*',action='append');

I want to implement something close to the GNU sort prototype above. I am not interested in a flag that can be specified for each file (e.g., -f file1 -f file2).

Answer 1

Here's a quick solution which decodes the argument list one (options, positional arguments) pair at a time.

import argparse

class ExtendAction(argparse.Action):
    def __call__(self, parser, namespace, values, option_string=None):
        items = getattr(namespace, self.dest, None)
        if items is None:
            items = []
        items.extend(values)
        setattr(namespace, self.dest, items)

parser = argparse.ArgumentParser()
parser.add_argument('files', nargs='*', action=ExtendAction)
parser.add_argument('-z', action='store_true')
parser.add_argument('-v', action='count')
parser.add_argument('args_tail', nargs=argparse.REMAINDER)

def interleaved_parse(argv=None):
    opts = parser.parse_args(argv)
    optargs = opts.args_tail
    while optargs:
        opts = parser.parse_args(optargs, opts)
        optargs = opts.args_tail
    return opts

print(interleaved_parse('-z bar foo'.split()))
print(interleaved_parse('bar foo -z'.split()))
print(interleaved_parse('bar -z foo'.split()))
print(interleaved_parse('-v a -zv b -z c -vz d -v'.split()))

Output:

Namespace(args_tail=[], files=['bar', 'foo'], v=None, z=True)
Namespace(args_tail=[], files=['bar', 'foo'], v=None, z=True)
Namespace(args_tail=[], files=['bar', 'foo'], v=None, z=True)
Namespace(args_tail=[], files=['a', 'b', 'c', 'd'], v=4, z=True)

Note: Don't try to use this with other non-flag arguments (besides a single nargs='*' argument and the args_tail argument). The parser won't know about previous invocations of parse_args so it will store the wrong value for these non-flag arguments. As a workaround, you can parse the nargs='*' argument manually after using interleaved_parse.

Answer 2

I've seen nothing definitive in the argparse documentation stating that it can or cannot permute. Based on your own observations, where permutation failed, and the following doc quotes, I'm going to conclude it cannot be done.

1. There's already a module explicitly named 'getopt':

   Note The getopt module is a parser for command line options whose API is designed to be familiar to users of the C getopt() function. Users who are unfamiliar with the C getopt() function or who would like to write less code and get better help and error messages should consider using the argparse module instead.

2. Even the default for getopt does not permute, there's a more explicitly defined method named gnu_getopt():

   This function works like getopt(), except that GNU style scanning mode is used by default. This means that option and non-option arguments may be intermixed.

3. In the getopt docs, the above reference to argparse is further exaggerated by the inclusion of the following:

   Note that an equivalent command line interface could be produced with less code and more informative help and error messages by using the argparse module:

Again, nothing definitive, but, to me, a very sharp divide is being drawn between getopt and argparse with the documentation favoring/advocating argparse.

Here's an example using gnu_getop() which satifies your -z [file [file]] test:

>>> args = 'file1 -z file2'.split()
>>> args
['file1', '-z', 'file2']
>>> opts, args = getopt.gnu_getopt(args, 'z')
>>> opts
[('-z', '')]
>>> args
['file1', 'file2']

Edit 1: Go Permute Yourself, with argparse

Inspired by the definition of "permute" in the 'Using Getopt' page you linked to,

The default is to permute the contents of argv while scanning it so that eventually all the non-options are at the end.

how about permuting the arg string before passing it to parse_args()?

import argparse

p = argparse.ArgumentParser();
p.add_argument('files',nargs='*',default=['-']);
p.add_argument('-z',action='store_true')

Rolling your own:

import re

def permute(s, opts_ptn='-[abc]'):
    """Returns a permuted form of arg string s using a regular expression."""
    opts = re.findall(opts_ptn, s)
    args = re.sub(opts_ptn, '', s)
    return '{} {}'.format(' '.join(opts), args).strip()

>>> p.parse_args(permute('bar -z foo', '-[z]').split())
Namespace(files=['bar', 'foo'], z=True)

Leveraging getopt:

import getopt

def permute(s, opts_ptn='abc'):
    """Returns a permuted form of arg string s using `gnu_getop()'."""
    opts, args = getopt.gnu_getopt(s.split(), opts_ptn)
    opts = ' '.join([''.join(x) for x in opts])
    args = ' '.join(args)
    return '{} {}'.format(opts, args).strip()

>>> p.parse_args(permute('bar -z foo', 'z').split())
Namespace(files=['bar', 'foo'], z=True)