C3p0 APPARENT DEADLOCK exception

Question

I keep getting this exception in my Tomcat log:

com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
WARNING: com.mchange.v2.async.ThreadPoolAs         


        

Answer 1

In my very particular case the problem had to do with the configuration of the server where I deployed my application.

It was only after printing out the stack trace of my application that it was hung because the oracle driver was waiting for a secure random number being generated:

Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
  java.io.FileInputStream.readBytes(Native Method)
  java.io.FileInputStream.read(FileInputStream.java:255)
  sun.security.provider.SeedGenerator$URLSeedGenerator.getSeedBytes(SeedGenerator.java:539)
  sun.security.provider.SeedGenerator.generateSeed(SeedGenerator.java:144)
  sun.security.provider.SecureRandom$SeederHolder.(SecureRandom.java:203)
  sun.security.provider.SecureRandom.engineNextBytes(SecureRandom.java:221)
  java.security.SecureRandom.nextBytes(SecureRandom.java:468)
  oracle.security.o5logon.O5Logon.a(Unknown Source)
  oracle.security.o5logon.O5Logon.(Unknown Source)
  oracle.jdbc.driver.T4CTTIoauthenticate.(T4CTTIoauthenticate.java:582)
  oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:401)
  oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:553)
  oracle.jdbc.driver.T4CConnection.(T4CConnection.java:254)
  oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
  oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:528)
  com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:134)
  com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182)
  com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171)
  com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137)
  com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014)
  com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
  com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810)
  com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)

Then I had to learn and understand a bit about entropy noise and could realize, with a little help from my Unix friends, that one of the servers had no installed a package that generates the "noise".

As mentioned in the following post I ran the next commands: cat /proc/sys/kernel/random/entropy_avail 23 cat /proc/sys/kernel/random/poolsize 4096

So there where too few entropy available to generate a secure random number.

After installing the package cat /proc/sys/kernel/random/entropy_avail 4096 Afterwards I had no more apparent deadlocks, my application could get connections to the DB.

http://www.usn-it.de/index.php/2009/02/20/oracle-11g-jdbc-driver-hangs-blocked-by-devrandom-entropy-pool-empty/

Answer 2

The tasks that are dealocking are Connection acquisition tasks. That is, c3p0 is trying to acquire new Connections from your database, and those Connection acquisition attempts are taking a long time.

The first thing I would do is upgrade to 0.9.2.1, which has a much improved means of performing a round of Connection acquisitions in situations where acquisition attempts sometimes fail.

If that doesn't solve your problem, then you'll need to figure out why c3p0's attempts to acquire a Connection are hanging for long periods of time: neither succeeding nor failing with an Exception.

Answer 3

Got a similar error like this before.

It could also be caused by database server blocking your IP. Make sure your servers' IP are not blocked by the database server/cloud provider.

In my case, was trying to insert data to the database in the cloud with multithreading, and it only works on the local, but not on a server (on a different network).