MSTest - Run “LoadTests” and Write Results to SQL Server Database

Question

If you want to configure your VS "Load Tests" to write the results to a database server, you use the following instructions.

If you want to run your "Load Tests" through powershell on a separate machine(think TFS 2018 release step), you use the following instructions.

I would like to do both, on multiple machines, in a automated manner, but there's not a great deal of documentation on this, I can run my tests like this:

.\mstest /testcontainer:"C:\XXX\ABC.loadtest"

But the results are kicked out to a "TRX" file rather than being placed into a database(there is some discussion on this). How do I put the results into a external database like when I run it locally(per instructions above)?

Note: @AdrianHHH points out that the "TRX" file is only a summary and that most of the info is stored locally(MDF/LDF file) in the user folder of current user running the load tests.

Update 1

Hmm I wonder where this is persisted:

(Curious, also click on the "?" icon in the "Manage Test Controller" box, nothing...)

It's not in the saved XML:

<RunConfigurations>
    <RunConfiguration Name="Run Settings1" Description="" ResultsStoreType="Database" TimingDetailsStorage="AllIndividualDetails" SaveTestLogsOnError="true" SaveTestLogsFrequency="0" MaxErrorDetails="200" MaxErrorsPerType="1000" MaxThresholdViolations="1000" MaxRequestUrlsReported="1000" UseTestIterations="false" RunDuration="10" WarmupTime="0" CoolDownTime="0" TestIterations="100" WebTestConnectionModel="ConnectionPerUser" WebTestConnectionPoolSize="50" SampleRate="5" ValidationLevel="High" SqlTracingConnectString="" SqlTracingConnectStringDisplayValue="" SqlTracingDirectory="" SqlTracingEnabled="false" SqlTracingFileCount="2" SqlTracingRolloverEnabled="true" SqlTracingMinimumDuration="500" RunUnitTestsInAppDomain="true" CoreCount="0" ResourcesRetentionTimeInMinutes="0" AgentDiagnosticsLevel="Warning">
      <CounterSetMappings>
        <CounterSetMapping ComputerName="[CONTROLLER MACHINE]">
          <CounterSetReferences>
            <CounterSetReference CounterSetName="LoadTest" />
            <CounterSetReference CounterSetName="Controller" />
          </CounterSetReferences>
        </CounterSetMapping>
        <CounterSetMapping ComputerName="[AGENT MACHINES]">
          <CounterSetReferences>
            <CounterSetReference CounterSetName="Agent" />
          </CounterSetReferences>
        </CounterSetMapping>
      </CounterSetMappings>
      <LoadGeneratorLocations>
        <GeoLocation Location="Default" Percentage="100" />
      </LoadGeneratorLocations>
    </RunConfiguration>
  </RunConfigurations>

They're not persisted in my default "testsettings" file either:

<?xml version="1.0" encoding="UTF-8"?>
<TestSettings name="Local" id="02cad612-043b-447d-993e-a9b9b0547c9d" 
     xmlns="http://microsoft.com/schemas/VisualStudio/TeamTest/2010">
  <Description>These are default test settings for a local test run.</Description>
  <Deployment enabled="false" />
  <Execution hostProcessPlatform="MSIL">
    <TestTypeSpecific />
    <AgentRule name="Execution Agents">
    </AgentRule>
  </Execution>
  <Properties>
    <Property name="TestSettingsUIType" value="UnitTest" />
  </Properties>
</TestSettings>

So I need to find where ever this configuration information is being persisted, then maybe I can find a way to feed it to MSTest. Does anyone else understand how this works?

Update 2

My TRX file does contains a "connection string" but I don't think it's to my database, my database is empty, on running via powershell it completes, but all I see is the "TRX" file.

Update 3

This one is tricky, I keep trying various ways to determine where this "Manage Test Configuration" data/credentials is being stored. One of the ways I did this was to use Microsoft's Process Monitor. You can actually see where it initially is being populated from:

It's from a Application Hive, of course that's begs the question where did the "Application Hive" get populated from, that's where things get a bit murky, there's allot of different calls to many files. A common trend is that the "Temp\Local" folder is often referenced.

I deleted the entire "Temp" folder for my user account(in the process losing all my VS configuration) and upon reopening my solution it appears as though this had an effect. When I pull up my "LoadTest" file, the "Load test results store" line is now empty. In fact the entire "Manage Test Controller" window has been restored back to default(empty).

I know believe that the configuration for this "Manage Test Controller" window is persisted in the temp folder. However, I've yet to locate where it is and/or how to change/automatically populate that information with a powershell script.

Answer 1

Finally figured this out. Basically I used several tools to check what files were being modified when I changed the connection string, the results made it obvious:

privateregistry.bin

Once I found this it was pretty obvious that VS was maintaining it's own little registry hive. It's clearly stated in this post, so I opened it in the way described in the article and found the connection string:

This indicated that:

"The SQL Connection String is NOT stored in the loadtest files. The setting seems to be PC specific so I had to change it on the build server - in one loadtest file (address.loadtest) as shown, then all the other loadtests adopt the same connection string."

So that's basically what I did, I logged into each build server and configured them so that they write all there results to my database rather than locally.

Load tests are clearly not designed to make this process easy, I don't think many people have attempted to do what I've done. All the articles just tell you to use their cloud service. I'm pretty sure that only covers web tests. If your using load testing to test unit tests you pretty much out of luck(without this work around). I really hope this gets official support in the future, it would be really nice to both run/view all types of load tests from TFS. For now though I'm going to have to keep using this work around.