Do you put unit tests in same project or another project?

Question

Do you put unit tests in the same project for convenience or do you put them in a separate assembly?

If you put them in a separate assembly like we do, we end up wit

Answer 1

I do not understand the frequent objection to deploying tests with production code. I led a team at a small microcap (grew from 14 to 130 people). We had a half-dozen or so Java apps and we found it EXTREMELY valueable to deploy tests into the field to execute them on a specific machine that was exhibiting unusual behavior. Random problems occur in the field and being able to throw a few thousand unit tests at the mystery with zero cost was invaluable and often diagnosed problems in minutes...including installation problems, flaky RAM problems, machine-specific problems, flaky network problems, etc, etc. I think it is incredibly valuable to put tests into the field. Also, random problems pop up at random times and it is nice to have the unit tests sitting there already waiting to be executed at a moments notice. Hard-drive space is cheap. Just like we try to keep data and functions together (OO design), I think there is something fundamentally valuable in keeping code and tests together (function + tests that validate the functions).

I would like to put my tests in the same project in C#/.NET/Visual Studio 2008, but I still haven't investigated this enought to achieve it.

One big benefit of keeping Foo.cs in the same project as FooTest.cs is that developers are constantly reminded when a class is missing a sibling test! This encourages better test-driven coding practices...holes are more apparent.

Answer 2

I fluctuate between same project and different projects.

If you're releasing a library releasing the test code with the production code is a problem, otherwise I find it usually isn't (although there's a strong psychological barrier before you try).

When putting tests in the same project I find it easier to switch between tests and the code they test, and easier to refactor/move them around.

Answer 3

I am really inspired by the unit testing framework of the Flood NN library by Robert Lopez. It uses a different project for every single unit tested class, and has one solution holding all these projects, as well as a main project that compiles and runs all the tests.

The neat thing is also the layout of the project. The source files are in a folder, but then the folder for the VS project is below. This allows you to make different subfolders for different compilers. All the VS projects are shipped with the code, so it is very easy for anyone to run any or all of the unit tests.

Answer 4

As others have answered - put tests in separate projects

One thing that hasn't been mentioned is the fact that you can't actually run nunit3-console.exe against anything but .dll files.

If you're planning on running your tests via TeamCity, this will pose a problem.

Let's say you have a console application project. When you compile, it returns an executable .exe to the bin folder.

From the nunit3-console.exe you wouldn't be able to run any tests defined in that console application.

In other words, a console application returns an exe file and a class library returns dll files.

I just got bit by this today and it sucks :(

Answer 5

Separate projects, although I debate with myself whether they should share the same svn. At the moment, I'm giving them separate svn repositories, one called

"MyProject" - for the project itself

and one called

"MyProjectTests" - for the tests associated with MyProject.

This is fairly clean and has the advantage that commits to the project and commits to the tests are quite separate. It also means you can hand over the project's svn if needed, without having to release your tests. It also means you can have branch/trunk/tag directories for your tests and for your project.

But I'm increasingly inclined to have something like the following, in a single svn repository, for each project.

MyProject
|\Trunk
| |\Code
|  \Tests
|\Tags
| |\0.1
| | |\Code
| |  \Tests
|  \0.2
|   |\Code
|    \Tests
\Branches
  \MyFork
   |\Code
    \Test

I'd be interested to know what other people think of this solution.

Answer 6

I know this is a very old question, but I would like to add my experience there I recently change unit testing habit from separate projects to same one.

Why?

First I am very tend to keep main project folder structure same with test project. So, if I have a file under Providers > DataProvider > SqlDataProvider.cs then I am creating same structure in my unit test projects like Providers > DataProvider > SqlDataProvider.Tests.cs

But after project is getting bigger and bigger, once you move files around from one folder to another, or from one project to another, then it is getting very cumbersome work to sync those up with unit test projects.

Second, it is not always very easy to navigate from class to be tested to unit test class. This is even harder for JavaScript and Python.

Recently, I started to practice that, every single file I created (for example SqlDataProvider.cs) I am creating another file with Test suffix, like SqlDataProvider.Tests.cs

At the beginning it seems it will bloat up files and library references, but at long term, you will eliminate moving file syndrome at first glance, and also you will make sure, every single file those are candidates of being tested will have a pair file with .Tests suffix. It gives you easy of jumping into test file (because it is side by side) instead of looking through separate project.

You can even write business rules to scan through project and identify class which does not have .Tests file, and report them to the owner. Also you can tell your test runner easily to target .Tests classes.

Especially for Js and Python you will not need to import your references from different path, you can simply use same path of target file being tested.

I am using this practice for a while, and I think it is very reasonable trade-off between project size vs maintainability and learning curve for new comers to the project.