Play tests with database: “Too many connections”

Question

To have a database available in scalatest with evolutions I use this extension of the default PlaySpec inspired by this SO question:

trait ResetDbSpec extends PlaySpec with BeforeAndAfterAll {
  lazy val appBuilder = new GuiceApplicationBuilder()
  lazy val injector = appBuilder.injector()
  lazy val databaseApi = injector.instanceOf[DBApi]

  override def beforeAll() = {
    Evolutions.applyEvolutions(databaseApi.database("default"))
  }

  override def afterAll() = {
    Evolutions.cleanupEvolutions(databaseApi.database("default"))
    databaseApi.database("default").shutdown()
  }
}

It applies database evolutions when the suite starts, and reverts them when the suite ends. A test then looks like

class ProjectsSpec extends ResetDbSpec with OneAppPerSuite { ...

After adding more tests like this, I hit a point where some tests that succeed when I run them alone, fail with this error:

com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections"

As can be see in the code above, I tried to add the line

databaseApi.database("default").shutdown()

in afterAll() to mitigate that, but it had no effect. I tried to not run tests in parallel, but no effect either. Where is it that I open db connections without closing them, and where should I call shutdown()?

N.B. I use Play 2.5.10 and Slick 3.1.

Answer 1

I have a lot of tests (about 500) and I don't get this error, the only difference I have with your code is that I add

databaseApi.database("default").getConnection().close()

and

Play.stop(fakeApplication)

for the integration tests.

Let me know if it changes anything.

Answer 2

Although it does not answer to what is happening with the connections leakage, I finally managed to hack around this:

1. Add jdbc to you libraryDependencies, even if the Play-Slick FAQ tells you not to do it:

   # build.sbt
   libraryDependencies += jdbc
   

   Restart sbt to take changes into account. In IntelliJ, you will want to refresh the project, too.

2. Disable the jdbc module that is conflicting with play-slick (credits: this SO answer):

   # application.conf
   play.modules.disabled += "play.api.db.DBModule"
   

   At the same place you should have already configured something like

   slick {
     dbs {
       default {
         driver = "slick.driver.MySQLDriver$"
         db.driver = "com.mysql.jdbc.Driver"
         db.url = "jdbc:mysql://localhost/test"
         db.user = "sa"
         db.password = ""
       }
     }
   }
   

3. Now you can use play.api.db.Databases from jdbc and its method withDatabase to run the evolutions.

   import org.scalatest.BeforeAndAfterAll
   import org.scalatestplus.play.PlaySpec
   import play.api.db.{Database, Databases}
   import play.api.db.evolutions.Evolutions
   
   
   trait ResetDbSpec extends PlaySpec with BeforeAndAfterAll {
   
     /**
      * Here we use Databases.withDatabase to run evolutions without leaking connections.
      * It slows down the tests considerably, though.
      */
   
     private def withTestDatabase[T](block: Database => T) = {
       Databases.withDatabase(
         driver = "com.mysql.jdbc.Driver",
         url = "jdbc:mysql://localhost/test",
         name = "default",
         config = Map(
           "username" -> "sa",
           "password" -> ""
         )
       )(block)
     }
   
     override def beforeAll() = {
       withTestDatabase { database =>
         Evolutions.applyEvolutions(database)
       }
     }
   
     override def afterAll() = {
       withTestDatabase { database =>
         Evolutions.cleanupEvolutions(database)
       }
     }
   
   }
   

4. Finally, call tests requiring a db reset like this:

   class MySpec extends ResetDbSpec {...}
   

Of course it sucks repeating this config both in "application.test.conf" and in withDatabase(), plus it mixes two different APIs, not talking about performance. Also it adds this before and after each suite, which is annoying:

[info] application - Creating Pool for datasource 'default'
[info] application - Shutting down connection pool.

If somebody has a better suggestion, please improve on this answer! I have been struggling for months.