How to write Unit Tests for functions that rely on dynamic data?

Question

Lets say you have a website, that uses a function to retrieve data from the database and returns the result to be displayed/parsed/etc...

Since the data that is retrieve

Answer 1

If your function does something interesting beyond pulling the data out of the database you should extract the retrieval into a different function and mock it, so you can test the rest.

This still leaves you with the task of testing the database access. You can't really do a unit test for that, because that would by definition not access any db and you could just test if it sends the sql statement you think it should, but not if the sql statement actually works.

So you need a database

You have various optiones:

1) create a fixed database for such tests which doesn't get changed by the tests.

Pro: Conceptually easy Con: hard to maintain. Tests become interdependent, because they rely on the same data. No way to test stuff that does updates, inserts or deletes (let alone DDL)

2) create a database during your test. Now you have two problems: setting up the database for the test and filling it with data.

Setting up:

1) have a database server running, with a user/schema/database for erveryone who needs to run tests (at least devs + ci-server). Schema can get created using stuff like hibernate or the scripts you use for deployment.

Works great, but drives oldfashioned DBAs crazy. The application must not depend on the schema name. You will also run into problems when you have more then one schema used by the app. This setup is fairly slow. It can help to put in on fast discs. Like RAM discs

2) Have an in memory database. Easy to start from code and fast. But in most cases it will behave the same as your production database. This is of less concern if you use something that tries to hide the difference. I often use an in memory database for the first build stage and the real thing in a second stage.

Loading the testdata

1) people tell me to use dbunit. I'm not convinced it seems to be lots of XML and hard to maintain when columns or constraints change.

2) I prefer normal application code. (Java + Hibernate) in my case, but the code that writes you data into the database in production should in many cases be suitable to write test data for your test. It helps to have a little special API that hides the details of satisfying all the foreign key and stuff: http://blog.schauderhaft.de/2011/03/13/testing-databases-with-junit-and-hibernate-part-1-one-to-rule-them/

Answer 2

Ignoring the fact that you are talking about the DB I think you may be looking for your unit tests to cover every eventuallity where as that can lead to set of diminishing returns. If I were you I'd cover a standard path and then a couple of edge cases. The fact is you cannot pragmatically test everything.

Here's some further reading

http://37signals.com/svn/posts/3159-testing-like-the-tsa
How deep are your unit tests?
http://johnnosnose.blogspot.co.uk/2012/04/re-over-testing.html
http://martinfowler.com/bliki/TestCoverage.html

Looking at your specifc db related problem, to test this function you probably need to create a seam to pre poulate the data so you can cover those cases.

Answer 3

I would build the data in the test itself. That way you can even test complicated scenarios for the ever changing data. The key point is that you can control the data changes in your tests by having a dedicated test db,

Step 1: Insert the data you need into a db that is only used by the test Step 2: The db is now in a stable predictable state, so you can run your queries and test the output

Answer 4

You can't, really. You'll need a guaranteed static set of data to create reliable unit tests. Perhaps a database snapshot will work for you.

Dynamic data can be useful in other ways, such as to perform regression testing...

Answer 5

Most tests focus on the logic paths that are involved in things like obtaining data. Not on the validity of the data itself. Validity of data is only meaningful if your application is somehow calculating or aggregating it or whatever, in which case you should be able to control the inputs and verify that the results are correct.

That said, sometimes you do want to hit the same database your app is using to verify returns. For example, if you're testing a function that returns a filtered dataset, your unit test could perform the same query and then do a row-by-row comparison of, say, each records primary key, and verify that your function returned the same set of data you were expecting.

I don't know if this is your specific question, but there's nothing wrong with hitting the database to perform asserts in unit tests, on the contrary. At least I do it all the time and no one has tried to have me arrested :)

Answer 6

Unit tests, in their ideal form, should only test one thing. In this case, you're testing two things:

1. the logic of your function
2. the database retrieval

So I would suggest the following refactor:

1. Move the database retrieval logic into a separate function
2. Have the function you want to test call that other function
3. Mock out the function that returns data so you can unit test just the logic of your app
4. If it makes sense (if you're just relying on another library to do this, then hopefully that lib already has tests), write a unit test for the dynamic retrieval function, where you can't test specifics, but can can test the structure and reasonableness of the data returned (e.g. it has all the fields set, and is a time within 5 seconds of now).

Also, typically it's a good idea to run unit tests in a test environment where you have complete control over what's stored in the database. You don't want to run these against production data.