Solid Java unit test automation? (JUnit/Hamcrest/…)

Question

Intent

I am looking for the following:

• A solid unit testing methodology
  1. What am I missing from my approach?
  2. What am I doing wro

Answer 1

Ok, here is my view on your questions:

Is there a list of techniques to use to build a unit test?

Short answer, no. Your problem is that to generate a test for a method, you need to analyse what it does and put a test in for each possible value in each place. There are/were test generators, but IIRC, they didn't generate maintainable code (see Resources for Test Driven Development).

You've already got a fairly good list of things to check, to which I would add:

• Make sure all paths through your methods are covered.
• Make sure all important functionality is covered by more than one test, I use Parameterized a lot for this.

One thing I find really useful to do is to ask what should this method be doing, as opposed to what does this method do. This way, you write the tests with a more open mind.

Another thing I find useful is to cut down on the boilerplate associated with the tests, so I can read the tests more easily. The easier it is to add tests, the better. I find Parameterized very good for this. For me, readability of tests is key.

So, taking your example above, if we drop the requirement 'test only one thing in a method' we get

public static class Root {
  @Test
  public void testROOT() {
    assertThat("hasComponents", MyPath.ROOT.getComponents(), is(not(empty())));
    assertThat("hasExactlyOneComponent", MyPath.ROOT.getComponents(), hasSize(1));
    assertThat("hasExactlyOneInboxComponent", MyPath.ROOT.getComponents(), contains("ROOT"));
    assertThat("isNotNull", MyPath.ROOT, is(notNullValue()));
    assertThat("toStringIsSlashSeparatedAbsolutePathToInbox", MyPath.ROOT.toString(), is(equalTo("/ROOT")));
  }
}

I've done two things, I've added the description into the assert, and I've merged all of the tests into one. Now, we can read the test and see that we've actually got duplicate tests. We probably don't need to test is(not(empty()) && is(notNullValue()), etc. This violates the one assert per method rule, but I think it's justified because you've removed lots of boilerplate without cutting down on coverage.

Can I perform checks automatically?

Yes. But I wouldn't use annotations to do it. Let's say we have a method like:

public boolean validate(Foobar foobar) {
  return !foobar.getBar().length > 40;
} 

So I have a test method which says something like:

private Foobar getFoobar(int length) {
  Foobar foobar = new Foobar();
  foobar.setBar(StringUtils.rightPad("", length, "x")); // make string of length characters
  return foobar;
}

@Test
public void testFoobar() {
  assertEquals(true, getFoobar(39));
  assertEquals(true, getFoobar(40));
  assertEquals(false, getFoobar(41));
}

The above method is easy enough to factor out depending upon the length, into a Parameterized test of course. Moral of the story, you can factorize your tests just as you can with non-test code.

So to answer your question, in my experience, I've come to the conclusion that you can do a lot to help with all of the combinations by cutting down on boilerplate within your tests, by using a judicious combination of Parameterized and factorization of your tests. As a final example, this is how I would implement your test with Parameterized:

@RunWith(Parameterized.class) public static class OfComponents { @Parameters public static Collection data() { return Arrays.asList(new Object[][] { { new String[] {"Test1", "Test2", "Test3"}, null }, { new String[] {"Test1"}, null }, { null, NullPointerException.class }, { new String[] {"Test1", "", "Test2"}, IllegalArgumentException }, }); }

private String[] components;

@Rule
public TestRule expectedExceptionRule = ExpectedException.none();

public OfComponents(String[] components, Exception expectedException) {
   this.components = components;
   if (expectedException != null) {
     expectedExceptionRule.expect(expectedException);
   }
}

@Test
public void test() {
  MyPath.ofComponents(components);
}

Please note that the above isn't tested and probably doesn't compile. From the above, you can analyse the data as input and add (or at least think about adding) all of the combinations of everything. For instance, you haven't got a test for {"Test1", null, "Test2"} ...