mutation-testing

While trying to run a PIT mutation test I get the following error: mutationCoverage failed: All tests did not pass without mutation when calculating line coverage. Mutation testing requires a green suite. The tests run just fine when I do a normal test build but while running the mutation tests phase they supposedly fail but no details are provided as to why. I have gone through the reasons listed on the PIT Testing FAQ but I still have no clue what could be wrong. I tried: adding the -Dthreads=1 option to rule of any multi threading issue could not find any system properties unique the couple

I used to use Heckle, but it is incompatible with ruby 1.9 because of issues with ParseTree. I've looked for alternatives, but the only thing that looked promising was Chaser, and that did not have any clear documentation that I could use to see if I could make it work with RSpec. It seems to have Test::Unit dependencies. So - is anyone out there using any cool tools to really check the quality of your tests? Alternatively - are there any coverage tools that provide better than c0 coverage? This would kind of help solve the same problem. I'm using cover_me at the moment, but it is c0, like

Mutation testing has been out there for a while now, and it seems there are at least one or two commercial mutation testing frameworks for C/C++. Have you used them? What are your experiences? Are there any open source alternatives? A brief search resulted in: PlexTest : http://www.itregister.com.au/products/plextest_detail.htm Insure++ : http://www.parasoft.com/jsp/products/insure.jsp;jsessionid=baacpvbaDywLID?itemId=63 MILU (may be only for C): http://www.dcs.kcl.ac.uk/pg/jiayue/milu/ With that said, you need to realize that mutation testing isn't particularly useful (at least from some

Do you have any examples of real life applications of mutation testing? Does it work better than simple test coverage tools? Or is it useless? What are the advantages/disadvantages of mutation testing in the real world? Nicolas De Nayer The usefulness of unit tests is no longer discussed. They are essential in conception of a quality application. But, how can we assess their relevance? A code coverage indicator up to 100% doesn’t mean the code is 100% tested. This is just a view of executed code during unit tests execution. Mutation testing will allow you to have more confidence in your tests.

In another question I asked if mutation-testing is useful in practice. As I didn't get any answers that satisfy me, I want to check the current tools myself. So I need an overview about current existing mutation-test-frameworks. Which of them are most usable and why? I program in Java, so I would prefer Java-tools, but I would risk a look at interesting frameworks for different languages. I want to integrate in an automatic build-process, so I would prefer tools that can be executed through command-line. There is also PIT which can be hooked into your build via a maven plugin or command line