Why does scalatest mix up the output?

Question

I run my scalatest from sbt, and the output gets mixed up - scalatest prints all the test run, and comments to them, and somewhere in the middle it prints the statistics:

Answer 1

It looks to me the reason for that is that SBT by default executes the tests in parallel, the same way maven-surefire-plugin does.

As it is explained in ScalaTest wiki, this can be solved by:

Disable Parallel Execution of Tests By default, sbt runs all tasks in parallel. Because each test is mapped to a task, tests are also run in parallel by default. To disable parallel execution of tests:

parallelExecution in Test := false

Answer 2

Have the same issue and solved it by saving logs into separate files. Not ideal, but good for CI, especially in case of long-running integration tests. How I did it:

1. Create a special directory for logs (for example, at the end of build.sbt):

   lazy val logDirectory = taskKey[File]("A directory for logs")
   logDirectory := {
     val r = target.value / "logs"
     IO.createDirectory(r)
     r
   }
   

2. [Optionally] Specify a file for a summary information in a project:

   testOptions in test += Tests.Argument(
     TestFrameworks.ScalaTest, "-fW", (logDirectory.value / "summary.log").toString
   )
   

   W disables colors

3. Create a group for each test. Each group must run a test in a forked JVM with special arguments:

   testGrouping in test := testGrouping.value.flatMap { group =>
     group.tests.map { suite =>
       val fileName = {
         // foo.bar.baz.TestSuite -> f.b.b.TestSuite
         val parts = suite.name.split('.')
         (parts.init.map(_.substring(0, 1)) :+ parts.last).mkString(".")
       }
   
       val forkOptions = ForkOptions(
         bootJars = Nil,
         javaHome = javaHome.value,
         connectInput = connectInput.value,
         outputStrategy = outputStrategy.value,
         runJVMOptions = javaOptions.value ++ Seq(
           "-Dour.logging.appender=FILE",
           s"-Dour.logging.dir=${logDirectory.value / fileName}"
         ),
         workingDirectory = Some(baseDirectory.value),
         envVars = envVars.value
       )
   
       group.copy(
         name = suite.name,
         runPolicy = Tests.SubProcess(forkOptions),
         tests = Seq(suite)
       )
     }
   }
   

   Note, we can tell the logback, where we save our logs through our.logging.appender and our.logging.dir

4. Create a configuration for logging (test/resources/logback-test.xml):

   <?xml version="1.0" encoding="UTF-8"?>
   <configuration>
       <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
           <target>System.out</target>
           <encoder>
               <pattern>%date{yyyy-MM-dd HH:mm:ss} %-5level [%thread] %logger{26} - %msg%n</pattern>
           </encoder>
       </appender>
   
       <appender name="FILE" class="ch.qos.logback.core.FileAppender">
           <file>${our.logging.dir}/test.log</file>
           <append>false</append>
           <encoder>
               <pattern>%date{yyyy-MM-dd HH:mm:ss.SSS} %-5level [%thread] %logger{26} - %msg%n</pattern>
           </encoder>
       </appender>
   
       <root level="TRACE">
           <appender-ref ref="${our.logging.appender}"/>
       </root>
   </configuration>
   

That's all.