Why should Mocha test cases be stateless?

问题

It is a common recommendation, that Mocha test cases should not share state. In light of Mochas strongly sequential nature of execution of test cases, I really do not understand this recommentation. And more - I see it dubious.

If the test cases, even asynchronic ones, are executed strictly one after another, there is no risk of time-race problems or other unpredicted execution sequence.

Let's take this Mocha structure:

describe
   describe
     it1
     it2    // async
     it3
   describe
     it4
     it5
describe
   describe
     it6    // async
     it7    // async
     it8
   describe
     it9
     it10

Test cases will ALWAYS be executed in clear sequential order, it1, it2,... it10, independently on whether they are sync or async, independently even on hierarchy of describes. The following structure will produce perfectly the same output:

describe
   it1
   it2    // async
   it3
   it4
   it5
   it6    // async
   it7    // async
   it8
   it9
   it10

Shared data between test cases in a single describe() can make it much easier and comfortable to communicate between test cases in order to:

reuse testing code and avoid repeating the same logic in each (or some of them) test case. Hooks can be handy here, but sometimes is more control expected than they offer
to reduce the testing execution time - a lots of time can be saved avoiding too many hooks that do nothing but satisfying this recommendation about statelessness.

Another fact in favor of using shared data between test cases is this simple and nice structure of the Mocha code. Different level of describe()s permit an easy and clear scope management for this eventual shared data.

The only negative side I see is making the code harder to understand, follow and maintain, when the "globals" are being "abused". Anyhow, nothing that cannot be avoided by disciplined coding.

Is there something I'm not aware of here?

回答1:

It is possible to use Mocha to run tests that are not stateless and thus depend on one another. It's just not what Mocha was designed for. At the end of the day, if you want to impose dependencies between your tests, well, sure, you can. It comes with some caveats though.

Optimizing for the Wrong Thing

You cite this advantage to sharing state among tests:

to reduce the testing execution time

Sure. All else being equal, we'd rather have a test suite that takes less time to run. Not resetting state between tests could amount to quite a bit of time saved on large test suites when running the entire suite. However, the fact is that "all else" is not equal:

Running an entire suite should be done by an automated process that reports what passed and what failed. In other words, there should be no one sitting there waiting for the whole suite to complete.
When there is a failure or when a new test is being implemented, the developer will want to run only that test which fails or the new test, not the whole suite. If the suite is designed so that tests 1 to N-1 must be run before test N is run, then this is that much more time the developer is waiting to get the result of the test he actually cares about. So he can sit there and twiddle his thumbs at a rate of $X/minute. "Multitask" is not the answer, as it has been proved that there is a cognitive cost to switching tasks. ("Ok the test is done... wait, what was the issue again?")

Test Selection

You can use --grep to select specific tests. This is extremely useful on large test suites to avoid wasting time to run tests you do not care about. So let's suppose that the titles of your tests are it1, ... it10 and you use --grep it7. Since Mocha takes all the tests to be independent from one another, it will just run whatever before and beforeEach hooks are applicable to it7 and run the test (and then run whatever after and afterEach hooks apply to it). It won't run it1 to it6 before running it7. For it to run these tests, you'd have to craft a --grep that covers all the necessary tests, which is certainly always possible to do but not pleasant.

In the HTML interface that you get when running Mocha in a browser, you can click on a test if you want Mocha to just run that test. Again, this is extremely useful if you have a single failing test that you want to fix. There's simply no equivalent to this simple click if it happens that the failing test is dependent on a bunch of tests that should run before it.

Test Order

In general, if your tests must run in a specific order, you have to be careful to ensure that tests split among multiple files are going to load in the order necessary for your tests to run.

For instance, it would be possible to share state among tests stored in different files using a global structure. However, when tests are in different files (where one file does not require the other), the order in which the files are processed by Mocha is completely dependent on the order in which the filesystem lists the files when the directory that contains them is read. There is a --sort option for those who want a predictable order based on a lexicographic sort. If you want to impose your own arbitrary order, then I guess you'll have to name your files 01foo.js, 02bar.js, etc.

This is true whether using Node.js to run Mocha or running Mocha in a browser. I have suites where the test files are loaded with RequireJS. Requesting modules A, B, C won't guaranteed that they'll be loaded in the order A, B, C unless C is declared to depend on B (and maybe A) and B to depend on A.

来源：https://stackoverflow.com/questions/29508444/why-should-mocha-test-cases-be-stateless

标签

testing

mocha