When is a Test not a Unit-test?

Question

I am looking for rules like:

A test is not a unit-test if:

• it communicates with a database
• it cannot run in parallel with other tests

Answer 1

See Michael Feathers' definition

A test is not a unit test if:

• It talks to the database
• It communicates across the network
• It touches the file system
• It can't run at the same time as any of your other unit tests
• You have to do special things to your environment (such as editing config files) to run it.

Answer 2

A test is not a unit test if it is not testing a unit.

Seriously, that's all there is to it.

The concept of "unit" in unit testing is not well-defined, in fact, the best definition I have found so far, isn't actually a definition because it is circular: a unit in a unit test is the smallest possible thing that can be tested in isolation.

This gives you two checkpoints: is it tested in isolation? And is it the smallest possible thing?

Please note that both of these are context-dependent. What might be the smallest possible thing in one situation (say, an entire object) might in another situation just one small piece of one single method. And what counts as isolation in one situation might be in another (e.g. in a memory-managed language, you never run in isolation from the garbage collector, and most of the time that is irrelevant, but sometimes it might not be).

Answer 3

It has no asserts, and is not expecting an exception to be thrown.

Answer 4

Difficult one...

For me a unit test verifies one specific piece of logic in isolation. Meaning, I take some logic, extract it from the rest (if necessary by mocking dependencies) and test just that logic - a unit (of the whole) - by exploring different kind of possible control flows.

But on the other side...can we always 100% say correct or incorrect?? Not to become philosophical, but - as also Michael says in his post:

Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

So why shouldn't I write a unit test that verifies the logic of parsing for instance an xls file by accessing some dummy file from the file system in my test folder (like MS tests allow with the DeploymentItem)?

Of course - as mentioned - we should separate these kind of tests from the others (maybe in a separate test suite in JUnit). But I think one should also write those tests if he feels comfortable in having them there...clearly then always again remembering that a unit test should just test a piece in isolation.

What is most important in my eyes is that these tests run fast and don't take too long s.t. they can be run repeatedly and very often.

Answer 5

After whether a test is a unit test or not is settled the next question is, is it a good unit test?

Answer 6

Implementing a test across multiple possibly failing units would not be a unit test.