testing command line utilities

Question

I\'m looking for a way to run tests on command-line utilities written in bash, or any other language.

I\'d like to find a testing framework that would have statements li

Answer 1

Check out ScriptTest :

from scripttest import TestFileEnvironment

env = TestFileEnvironment('./scratch')

def test_script():
    env.reset()
    result = env.run('do_awesome_thing testfile --with extra_win --file %s' % filename)
    # or use a list like ['do_awesome_thing', 'testfile', ...]
    assert result.stdout.startswith('Creating awesome file')
    assert filename in result.files_created

It's reasonably doctest-usable as well.

Answer 2

I know that this question is old, but since I was looking for an answer, I figured I would add my own for anyone else who happens along.

Full disclaimer: The project I am mentioning is my own, but it is completely free and open source.

I ran into a very similar problem, and ended up rolling my own solution. The test code will look like this:

from CLITest import CLITest, TestSuite
from subprocess import CalledProcessError


class TestEchoPrintsToScreen(CLITest):
    '''Tests whether the string passed in is the string
    passed out'''

    def test_output_contains_input(self):
        self.assertNotIsInstance(self.output, CalledProcessError)
        self.assertIn("test", self.output)

    def test_ouput_equals_input(self):
        self.assertNotIsInstance(self.output, CalledProcessError)
        self.assertEqual("test", self.output)

suite = TestSuite()

suite.add_test(TestEchoPrintsToScreen("echo test"))

suite.run_tests()

This worked well enough to get me through my issues, but I know it could use some more work to make it as robust as possible (test discovery springs to mind). It may help, and I always love a good pull request.

Answer 3

outside of any prepackaged testing framework that may exist but I am unaware of, I would just point out that expect is an awesome and so underutilized tool for this kind of automation, especially if you want to support multistage interaction, which is to say not just send a command and check output but respond to output with more input. If you wind up building your own system, it's worth looking into.

There is also python reimplementation of expect called pexpect.There may be some direct interfaces to the expect library available as well. I'm not a python guy so I couldn't tell you much about them.

Answer 4

Well... What we usually do (and one of the wonders of O.O. languages) is to write all the components of an application before actually make the application. Every component might have an standalone way to be executed, for testing purpose (command line, usually), that also allows you to think in them as complete programs each by each, and use them in future projects. If what you want is to test the integrity of an existing program... well, I think the best way is to learn in deep how it work, or even deeper: read the source. Or even deeper: develop a bot to force-test it :3

Sorry that's what I have .-.