File access, unit testing, dependency injection

Question

I've recently asked a question regarding best ways of separating business logic from data access to make application testable (here). Thanks to Jeff Sternal, I've created interface for the data access and passing its concrete implementation from the application top level to BL. But now I'm trying to figure out how can I separate file access methods from business logic.

Let's say I have a function that loads data from a database into dataset, formats loaded data (formats are stored in some external xml file) and finally serializes dataset to a file. So, to support testability I need to move all the functions that access file system to some interface. But first - I find it quite handy to just call dataset.WriteXml(file), but for the testability I have to create interface and move dataset.WriteXml() to its implementation, which looks to me as unnecessary layer and makes code less obvious. And second - if I move all methods that access file system to a single interface, it will violate SRP principle, because serializing\deserializing datasets and reading data formats from a file seems to be different responsibilities, correct?

Answer 1

Go with the extra classes. When testing it its easier to:

• For the dataset exporter/writer - check the modified data is passed to the writer
• For the formatter - check the formatting logic

This doesn't mean you don't:

• Use dataSet.WriteXml, you just do so in the v. simple class ... as a one liner, you don't need to do add a focused integration test for it ... but if u l8r on store say in an external system, u might choose to do so in that other implementation, without affecting the other code
• Use separate classes for a dataset exporter/writter than for a simple file writer.

As long as you are using v. simple dependency injection, you are really just keeping everything simple ... which will get you better results both in the short and in the long run.

Ps. not hitting the file system during tests is important, as when your number of tests grow you want them to run v. fast, so don't be misguided to what appears at first "quick" accesses for a test

Answer 2

I think you need to split up your code a little bit more...

You say: Let's say I have a function that

1. loads data from a database into dataset,
2. formats loaded data (formats are stored in some external xml file) and
3. finally serializes dataset to a file.

That sounds like 3-4 jobs at least...

If you split your code some more, then you can test each of those functions without all the other goo around them.

If you just want to do Dataset.WriteXML, Then you dont have to test that. Thats a Framework function thats working quite nicely. Try getting some mocks in there to fake it out. How exactly depends on your solution...

Answer to comment:

Creating all those small classes with their own tests will make testing easy, and it will also make your functions small and compact (-> easy to test) You would test if the content of the Dataset is exactly what you require, not if the dataset gets serialized correctly into an xml file. You would also test if your formater would perform its function correctly, without any dependencies to any other logic. You would also test the Data access, but without accessing the DB(Stubs/Mocks again)

After you know that all this works like it should, you "just" verify that the propper method on the dataset will get called, and that should satisfy you, since you tested the other parts in isolation already.

The tricky part about unittesting is getting meaningfull tests. They should be -fast- and -simple-

To make the tests fast, you should not touch stuff you have no control over:

• Webservices
• Filesystems
• Databases
• Com Objects

To make them simple, you keep you clases focused on a single task, that where the SRP comes in, which you already mentioned. Take a look at this answer... It will also point out the other principles of "SOLID" development

https://stackoverflow.com/questions/1423597/solid-principles/1423627#1423627

Answer 3

For a quick summary, if you really want to make this testable, I recommend:

1. Extracting the data formatting code into a new class (which implements an interface you can use to mock).
2. Passing this class your DataSet.
3. Make the new class return the DataSet as an IXmlSerializable that you can also mock.

Without seeing the current code, I have to make some assumptions (hopefully not too many!) - so perhaps the current code looks something like this:

public class EmployeeService {

    private IEmployeeRepository _Repository;

    public EmployeeService(IRepository repository) {
        this._Repository = repository;
    }

    public void ExportEmployeeData(int employeeId, string path) {

        DataSet dataSet = this._Repository.Get(employeeId);
        // ... Format data in the dataset here ...
       dataSet.WriteXml(path);
    }
}

This code is simple and effective, but not testable (without side-effects). Additionally, having all that logic in one place is a violation of the single-responsibility principal.

Depending on your needs, that's probably fine! Fulfilling the SRP is always just a goal, and we always need to balance testability with other factors that influence our designs.

However, if you want to segregate responsibilities a bit more and make it testable, you could do so by moving the formatting logic into its own class (which will implement IEmployeeDataSetFormatter), then inject an IEmployeeDataSetFormatter into this method call (alternately we could inject it into the service's constructor, just like the IEmployeeRepository). The method that formats the data will return an IXmlSerializable so we can mock it for safe, isolated testing:

public interface IEmployeeDataSetFormatter {
    IXmlSerializable FormatForExport(DataSet dataSet);
}

public class EmployeeDataSetFormatter: IEmployeeDataSetFormatter {
    public IXmlSerializable FormatForExport(DataSet dataSet) {
        // ... Format data in the dataset here ...
        return (IXmlSerializable) dataSet;
    }
}

public void ExportEmployeeData2(int employeeId, string path, IEmployeeDataSetFormatter formatter) {

    DataSet dataSet = this._Repository.Get(employeeId);

    IXmlSerializable xmlSerializable = formatter.FormatForExport(dataSet);

    // This is still an intermediary step - it's probably worth
    // moving this logic into its own class so you don't have to deal
    // with the concrete FileStream underlying the XmlWriter here
    using (XmlWriter writer = XmlWriter.Create(path)) {
        xmlSerializable.WriteXml(writer);
    }
}

This has definite costs. It adds an additional interface, an additional class, and a bit more complexity. But it's testable and more modular (in a good way, I think).

Answer 4

You could use the versions of WriteXml() that takes a Stream or TextWriter and set up your code such that this object is passed into your code, then for testing pass in a mock object.

Answer 5

It's not completely clear from your description what your code is doing. What is the most important thing your code is doing? Is it the fomatting? I would say don't bother testing the writing of the xml. How would you verify that anyway?

If you must read from and write to files, you might be better altering your code so that you pass in a streamreader/streamwriter or textreader/textwriter, with the calling code injecting an instance of something like memorystream for testing and filestream for real i/o in production.