What can be alternative metrics to code coverage?

Question

Code coverage is propably the most controversial code metric. Some say, you have to reach 80% code coverage, other say, it\'s superficial and does not say anything about your te

Answer 1

SQLite is an extremely well-tested library, and you can extract all kinds of metrics from it.

As of version 3.6.14 (all statistics in the report are against that release of SQLite), the SQLite library consists of approximately 63.2 KSLOC of C code. (KSLOC means thousands of "Source Lines Of Code" or, in other words, lines of code excluding blank lines and comments.) By comparison, the project has 715 times as much test code and test scripts - 45261.5 KSLOC.

In the end, what always strikes me as the most significant is none of those possible metrics seem to be as important as the simple statement, "it meets all the requirements." (So don't lose sight of that goal in the process of achieving it.)

If you want something to judge a team's progress, then you have to lay down individual requirements. This gives you something to point to and say "this one's done, this one isn't". It's not linear (solving each requirement will require varying work), and the only way you can linearize it is if the problem has already been solved elsewhere (and thus you can quantize work per requirement).

Answer 2

This hasn't been mentioned, but the amount of change in a given file of code or method (by looking at version control history) is interesting particularly when you're building up a test suite for poorly tested code. Focus your testing on the parts of the code you change a lot. Leave the ones you don't for later.

Watch out for a reversal of cause and effect. You might avoid changing untested code and you might tend to change tested code more.

Answer 3

If you are looking for some useful metrics that tell you about the quality (or lack there of) of your code, you should look into the following metrics:

1. Cyclomatic Complexity
   • This is a measure of how complex a method is.
   • Usually 10 and lower is good, 11-25 is poor, higher is terrible.
2. Nesting Depth
   • This is a measure of how many nested scopes are in a method.
   • Usually 4 and lower is good, 5-8 is poor, higher is terrible.
3. Relational Cohesion
   • This is a measure of how well related the types in a package or assembly are.
   • Relational cohesion is somewhat of a relative metric, but useful none the less.
   • Acceptable levels depends on the formula. Given the following:
     • R: number of relationships in package/assembly
     • N: number of types in package/assembly
     • H: Cohesion of relationship between types
   • Formula: H = (R+1)/N
   • Given the above formula, acceptable range is 1.5 - 4.0
4. Lack of Cohesion of Methods (LCOM)
   • This is a measure of how cohesive a class is.
   • Cohesion of a class is a measure of how many fields each method references.
   • Good indication of whether your class meets the Principal of Single Responsibility.
   • Formula: LCOM = 1 - (sum(MF)/M*F)
     • M: number of methods in class
     • F: number of instance fields in class
     • MF: number of methods in class accessing a particular instance field
     • sum(MF): the sum of MF over all instance fields
   • A class that is totally cohesive will have an LCOM of 0.
   • A class that is completely non-cohesive will have an LCOM of 1.
   • The closer to 0 you approach, the more cohesive, and maintainable, your class.

These are just some of the key metrics that NDepend, a .NET metrics and dependency mapping utility, can provide for you. I recently did a lot of work with code metrics, and these 4 metrics are the core key metrics that we have found to be most useful. NDepend offers several other useful metrics, however, including Efferent & Afferent coupling and Abstractness & Instability, which combined provide a good measure of how maintainable your code will be (and whether or not your in what NDepend calls the Zone of Pain or the Zone of Uselessness.)

Even if you are not working with the .NET platform, I recommend taking a look at the NDepend metrics page. There is a lot of useful information there that you might be able to use to calculate these metrics on whatever platform you develop on.