How to skip the rest of tests in the class if one has failed?

Question

I\'m creating the test cases for web-tests using Jenkins, Python, Selenium2(webdriver) and Py.test frameworks.

So far I\'m organizing my tests in the following str

Answer 1

• If you'd like to stop the test execution after N failures anywhere (not in a particular test class) the command line option pytest --maxfail=N is the way to go: https://docs.pytest.org/en/latest/usage.html#stopping-after-the-first-or-n-failures

• if you instead want to stop a test that is comprised of multiple steps if any of them fails, (and continue executing the other tests) you should put all your steps in a class, and use the @pytest.mark.incremental decorator on that class and edit your conftest.py to include the code shown here https://docs.pytest.org/en/latest/example/simple.html#incremental-testing-test-steps.

Answer 2

It's generally bad practice to do what are you doing. Each test should be as independent as possible from the others, while you completely depend on the results of the other tests.

Anyway, reading the docs it seems like a feature like the one you want is not implemented.(Probably because it wasn't considered useful).

A work-around could be to "fail" your tests calling a custom method which sets some condition on the class, and mark each test with the "skipIf" decorator:

class MyTestCase(unittest.TestCase):
    skip_all = False

   @pytest.mark.skipIf("MyTestCase.skip_all")
   def test_A(self):
        ...
        if failed:
            MyTestCase.skip_all = True
  @pytest.mark.skipIf("MyTestCase.skip_all")
  def test_B(self):
      ...
      if failed:
          MyTestCase.skip_all = True

Or you can do this control before running each test and eventually call pytest.skip().

edit: Marking as xfail can be done in the same way, but using the corresponding function calls.

Probably, instead of rewriting the boiler-plate code for each test, you could write a decorator(this would probably require that your methods return a "flag" stating if they failed or not).

Anyway, I'd like to point out that,as you state, if one of these tests fails then other failing tests in the same test case should be considered false positive... but you can do this "by hand". Just check the output and spot the false positives. Even though this might be boring./error prone.

Answer 3

The pytest -x option will stop test after first failure: pytest -vs -x test_sample.py