System not scaling to support concurrent users

Question

I'm having a problem with scaling the number of concurrent users on my system. From my tests, scaling the number of concurrent users seems to directly increase the duration of a request in a linear relationship.

I am running a Java Web Application deployed on a (virtual) Ubuntu Quad Core machine with 16Gb RAM. I am using Apache Tomcat 7 and a MySQl 5.5 Database. Tomcat and MySQL are using the default settings - I have not configured them in any way.

I am using Apache Benchmark to run a number of tests which ultimately create a SQL query to return one row of data, where the response size is very small.

I use Spring's JDBCTemplate, and a Apache Commons BasicDataSource. The spring bean's configuration is shown below.

<!-- READ ONLY CONNECTION TO DATABASE -->
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
  <property name="driverClassName">
    <value>com.mysql.jdbc.Driver</value>
  </property>
  <property name="username">
    <value>${database.username}</value>
  </property>
  <property name="password">
    <value>${database.password}</value>
  </property>
  <property name="url">
    <value>${database.url}/${database.schema}</value>
  </property>
  <property name="timeBetweenEvictionRunsMillis" value="7200000" />
  <property name="minEvictableIdleTimeMillis" value="3600000" />
  <property name="maxActive" value="100" />
  <property name="maxIdle" value="5" />
  <property name="defaultAutoCommit" value="false" />
  <property name="defaultReadOnly" value="true" />
</bean>

<bean id="myDao" class="...">
   <property name="jdbcTemplate" ref="jdbcTemplate"></property>
   <property name="dataSource" ref="dataSource"></property>
</bean>

<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
   <property name="dataSource" ref="dataSource" />
</bean>

My Java method that creates a couple of queries is annotated with @Transactional.

These are the results from my tests:

• 1 request takes 0.2 seconds.
• 10 requests (being performed concurrently) takes 0.9 seconds.

Thus you can see my application is not scaling. I'm not sure what the cause of the problem could be. Can anyone see what I am doing wrong or suggest ways in which I can investigate this further?

Thanks in advance,

Phil

Update

Further metrics:

  1 Request,  Concurrency   1 = 0.22s
 10 Requests, Concurrency  10 = 0.6s (mean), 0.5(min)
100 Requests, Concurrency 100 = 7 (mean), 3.7(min)
300 Requests, Concurrency 300 = 12s (mean), 4.3(min)
300 Requests, Concurrency 300 = 18s (mean), 6.4(min)

Response size is 1kb.

Trying same requests & changing the concurrency:

300 Requests, Concurrency   8 = total time: 14.9s
300 Requests, Concurrency  20 = total time: 15.3s
300 Requests, Concurrency 300 = total time: 24s

So reducing the concurrency to 8 completes 10s quicker then a concurrency of 300. Increasing from 8 slows down transactions. 8 seems to be the most optimum concurrency.

Answer 1

There are a few things to consider when trying to make an application concurrent.

Firstly, just because your server has four cores, does not mean they are all available to your JVM, you need to interrogate the runtime to see how many are available and it is also technically possible for this to change during the lifetime of the JVM, although rare.

Next, you need to consider the physical topology of your environment. Is the DB running on the same server as the application? If so you have additional contention for resources in terms of processing and IO, not just what your application is doing.

Once you have got an understanding of those points, you need to consider the IO vs. Processing profile of your application. An application which is finding prime numbers and outputting them to the system log for example is virtually 100% processing vs. 0% IO. In this type of application there is no point in having more threads in your application than there are available cores as the cores will be continuously busy with what they're doing and the overhead of task switching will actually slow down your application.

An application heavily tied to a DB would generally have a relatively high IO to Processing profile, although that profile becomes less IO bound if you are only reading and those reads are relatively small with a well defined database where the data being queried is logically laid out. The size of the DB will also impact IO, based on whether the entire DB set can be kept in memory or whether disk paging is occurring.

I'd highly recommend reading Java Concurrency in Practice by Brian Goetz if you are new to concurrency. That being said, you are taking a sensible approach by profiling your application as you are.