Dependencies analysis tool - updating regression test cases

Question

Problem

Its quite a common problem I would like to think. Adding new code translates into regression - existing test cases become obsolete. Dependencie

Answer 1

If I read the question correctly, you want a tool that analyses source code and determines which tests don't need to be rerun.

First thing to consider is what's the problem with always running all the tests? That's what CI means. If you can't run all your unit tests in 15 minutes and all your integration/stress tests overnight, resolve whatever issue causes that to be the case, probably by buying some hardware.

Failing that, the only thing that can track potential test regression is coverage analysis (e.g. http://www.eclemma.org/). Given even basic OO design, let alone dependency injection, static package or class dependencies are at best meaningless, and probably actively misleading, in terms of what changes can cause what tests to fail. And that's ignoring the case where the problem is A should be calling B, and isn't. However, cross-reference changed files with called files and if there is no intersection, maybe not rerunning the test is arguably safeish - the remaining failures will be things like some code adding an event handler where there wasn't one before.

I don't think there is a free tool to do this, but it should be scriptable from Jenkins/Emma/Git. Or use one of the integrated solutions like Kalistick. But I'd be sceptical that it could efficiently beat human judgement based on communication with the development team as to what they are trying to do.

On a vaguely related note, the simple tool the Java world actually needs, but doesn't afaik exist, is an extension to junit that runs a set of tests in dependency order, and stop on the first error. That would provide a little bit of extra help to let a developer focus on the most likely source of a regression.