ThreadPoolExecutor : Tasks are getting queued up and not submitted

Question

We have a scenario where tasks submitted to ThreadPoolExecutor are long running. When the thread pool is started we start it with core pool size = 5, max pool size = 20 and queu

Answer 1

I think another approach would be to set the corePoolSize based on the the number of tasks waiting in the queue. One can control the corePoolSize by using setCorePoolSize. A sample monitor thread can control you threadPoolExecutor. You can also improve this monitor to adjust the degree of parallelism.

    public class ExecutorMonitor extends Thread{

            ThreadPoolExecutor executor = null;
            int initialCorePoolSize;
            public ExecutorMonitor(ThreadPoolExecutor executor)
            {
                this.executor = executor;
                this.initialCorePoolSize = executor.getCorePoolSize();
            }
            @Override
            public void run()
            {
                while (true)
                {   
                    try {
                        Thread.sleep(5000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                    if (executor.getQueue().size() > 0)
                    {
                        if(executor.getActiveCount() < executor.getMaximumPoolSize())
                            executor.setCorePoolSize(executor.getCorePoolSize() + 1);
                    }
                    if (executor.getQueue().size() == 0)
                    {
                        if(executor.getCorePoolSize() > initialCorePoolSize)
                            executor.setCorePoolSize(executor.getCorePoolSize() -1);
                    }
                }
            }
        }

Answer 2

The Javadocs for ThreadPoolExecutor states:

Any BlockingQueue may be used to transfer and hold submitted tasks. The use of this queue interacts with pool sizing:

• If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
• If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
• If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.

Unless you exceed your queue size after 5 threads are "hanging", you're not going to get more threads.

The real answer is: fix the problem that's causing your threads to hang. Otherwise you're going to have to implement some scheme that uses the Futures returned by submit() to cancel threads if they are running too long.

Answer 3

The overall situation is like this:

core pool size = 5,
max pool size = 20 and 
queue size of 10

10 tasks are submitted. Out of which

1. 5 Tasks hanged on I/O => all threads of core pool size are occupied. And hence there is no idle thread.
2. 5 Tasks are remained . These 5 threads are enqueued to queue since there is no idle thread and the queue can accommodate 10 tasks. These enqueued tasks will not execute until either the queue is full or any of the threads in core pool is free.

Hence, Your Program is hanged .

To know more about dynamics of ThreadPoolExecutor watch here . The notable points of this doc is as follows:

• If fewer than corePoolSize threads are running, the Executor always prefers adding a new thread rather than queuing.
• If corePoolSize or more threads are running, the Executor always prefers queuing a request rather than adding a new thread.
• If a request cannot be queued, a new thread is created unless this would exceed maximumPoolSize, in which case, the task will be rejected.

EDIT
If you wish to increase core pool size then you can use setCorePoolSize(int corePoolSize) . If you increase the corepoolsize then new threads will, if needed, be started to execute any queued tasks.

Answer 4

i guess the best solution would be override the execute method of ThreadPoolExecutor and introduce the changes in that.

Answer 5

The "variable" sized thread pool feature in the jdk is a bit tricky. Basically, the setup you are using won't work very well for the reasons you outlined. When i want a variable size thread pool, i typically use this setup:

   ThreadPoolExecutor executor = new ThreadPoolExecutor(maxPoolSize, maxPoolSize,
                                                         DEFAULT_THREAD_TIMEOUT, DEFAULT_THREAD_TIMEOUT_UNIT,
                                                         new LinkedBlockingQueue<Runnable>(),
                                                         threadFactory);
    executor.allowCoreThreadTimeOut(true);

this will create a variable sized thread pool which will vary between 1 and maxPoolSize threads. since the core size is the same as the max size, the pool will always prefer adding threads to queuing, thus you will never backup your queue until the number of threads is maxed out.

UPDATE: As for your hanging tasks problem, if there is an upper limit on the length of a task, you could have a separate manager track the outstanding Future's and cancel them if they are over the max running time. this, of course, assumes your tasks are interruptible.

Answer 6

The fundamental problem is that the user is really getting CORE_POOL_SIZE threads with unlimited Queue scenario. As such if there are 5 threads in core pool that's all he can ever use, max size does nothing to help. While reducing the time of thread execution is advisable in all cases, in production scenario we cannot often control how third party services will behave and as such the solution would be to increase core pool size to be equal to max pool size or limit the queue size.