JUnit @Ignore usefulness?

Question

In JUnit you can use @Ignore before methods to tell the test runner to automatically skip those tests. From what I can gather, this is really only a convenient way

Answer 1

Using @Ignore instead of commenting out @Test has the advantage that Hudson/Jenkins recognizes this as an ignored test and displays it in the test results accordingly. So you get reminded about that ignored Test which is a big benefit!

Answer 2

@Ignored tests are more maintainable (and maintained) than commented-out ones. They are subject to compiler errors as things evolve, structured searching via IDE's, and IDE-assisted refactoring.

Answer 3

Am I correct in saying then, that at runtime there is no difference between an @Ignore test, a method with no annotation, and a commented out method?

An @Ignored method can be found via reflection. A method with no annotation can't (or, to be precise, it can't be identified with certainty as an ignored test method), and a commented out method does not even get into the bytecode.

Albeit I don't think there would be much practical value in finding @Ignored methods runtime, it may be useful to generate statistics / reports.

how much usefulness does the @Ignore tag really have

One thing I can think of is searchability. You can easily identify all @Ignore annotations in the source code, while unannotated or commented out tests are not so simple to find.

it might be more useful to fail the test so it's not overlooked?

If you want (and can) fix it right away, it is fine to have it fail. There are cases when you can't, but you still want to method to be around, precisely so that it does not get forgotten. Then @Ignore makes sense.

Answer 4

• A method with no annotation most likely is a utility method, likely used by other methods in the test.

• A commented-out method is one that is completely hidden.

• But you might use @Ignore to indicate that you are temporarily disabling the test. If you have a large and complicated piece of code that is in transition, you might want to temporarily disable some tests while that code is failing, especially if the mechanism running that code is run as part of some larger regression suite. The @Ignore attribute would be a reminder to you (or your QA team) that there is still more to do.

  Remember that @Ignore'd tests are still live code, and can be seen and even executed via reflection, which commented-out code cannot be. And it also means when you refactor other code, they are also updated, and syntax breakage in an API will show there. So even tests with @Ignore set can actively contribute to quality of code.

• JUnit 4 TestRunners will show @Ignore'd tests as "skipped" so you can see in the end how many tests passed/failed/skipped.

Although your copy of Netbeans may not take full advantage of @Ignore, I can assure you that it is not the only consumer of JUnit tests.

Answer 5

FYI, TestNG supports this slightly differently:

• You can include a description of why the test is being ignored:

  @Test(enabled = false, description = "Disabled until BUG-1234 gets fixed")
  

  • The Eclipse TestNG plug-in and the HTML reports will give you a list of all the tests that have been disabled, along with the reason why:

Answer 6

IMHO, @Ignore's usefullness would be to know in the reports that there are x skipped tests, which you don't get if the test isn't annotated as a test or is commented. Failing the tests isn't a good option as we know that the tests we want to ignore aren't relevant at some point, but will have to be reenabled later. So we want to see if all the other tests will succeed/fail, knowing that we will skip the irrelevant ones (the fact that we've skipped them will be visible in reports).