how does MSTest determine the order in which to run test methods?

Question

edit: note, question 288805 is similar, however, I specifically am asking how does MSTest choose the default test order. Please

Answer 1

As for VSTest execution order. Here is how it's organized in your TestProject:

1. Sort cs-files in your project by their CREATION Time ASC
2. Method Position in each file

For example, you have 3 cs files in project.

• UnitTest1.cs - created 01/01/1970 with methods TestMethod05 and TestMethod03
• UnitTest2.cs - created 05/01/1970 with method TestMethod02.
• UnitTest3.cs - created 03/01/1970 with method TestMethod01.

Then order of executing test is this:

    TestProject1.UnitTest1.TestMethod05
    TestProject1.UnitTest1.TestMethod03
    TestProject1.UnitTest3.TestMethod01
    TestProject1.UnitTest2.TestMethod02

You can see the 'default order' using command:

vstest.console.exe TestProject1.dll /ListTests

Answer 2

The MSDN says ;-)

How to: Create an Ordered Test

http://msdn.microsoft.com/en-us/library/ms182631.aspx

MSTest.exe command-line options

http://msdn.microsoft.com/en-us/library/ms182489(v=vs.120).aspx

Answer 3

The tests created in MSTest are ordered by execution time, in ascending order, after the tests that have not failed, for this reason the random execution. Unless the method names are in alphabetical order.

Answer 4

I believe that MSTest executes test methods ordering them by their 'ID' (seems to be their full namespace).

I created a TestProject1 wich contains 4 unt tests (UnitTest1, ...2, ...A, ...B). Each unit test contains 5 test methods (TestMethodA, ...B, ...1, ...2, ...3). They were declared with random order inside their test classes. Now, every time I run MSTest, the tests are executed with the same order:

TestProject1.UnitTest1.TestMethod1
TestProject1.UnitTest1.TestMethod2
TestProject1.UnitTest1.TestMethod3
TestProject1.UnitTest1.TestMethodA
TestProject1.UnitTest1.TestMethodB
TestProject1.UnitTest2.TestMethod1
TestProject1.UnitTest2.TestMethod2
TestProject1.UnitTest2.TestMethod3
TestProject1.UnitTest2.TestMethodA
TestProject1.UnitTest2.TestMethodB
TestProject1.UnitTestA.TestMethod1
TestProject1.UnitTestA.TestMethod2
TestProject1.UnitTestA.TestMethod3
TestProject1.UnitTestA.TestMethodA
TestProject1.UnitTestA.TestMethodB
TestProject1.UnitTestB.TestMethod1
TestProject1.UnitTestB.TestMethod2
TestProject1.UnitTestB.TestMethod3
TestProject1.UnitTestB.TestMethodA
TestProject1.UnitTestB.TestMethodB

The only way to change that order is to rename one TestClass or a TestMethod. If for example I rename the TestMethodB, of the UnitTest1, to TestMethod4 it will be executed before TestMethodA.

To see the IDs of your test methods open the 'Test View' window from VS and then right click on a column header (e.g. Test Name) --> "Add/Remove Columns..." and add 'ID' column.