ExecutorService that executes tasks sequentially but takes threads from a pool

Question

I am trying to build an implementation of the ExecutorService, let\'s call it SequentialPooledExecutor, with the following properties.

Answer 1

Recently I encountered the same problem. There is no built-in class for this, but a queue is close enough. My simple implementation looks like this (maybe it's helpful for others looking for examples on the same issue)

public class SerializedAsyncRunnerSimple implements Runnable {
private final ExecutorService pool;
protected final LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue<>(); //thread safe queue
protected final AtomicBoolean active = new AtomicBoolean(false);


public SerializedAsyncRunnerSimple(ExecutorService threadPool) {this.pool = threadPool;}


public void addWork(Runnable r){        
    queue.add(r);
    startExecutionIfInactive();
}

private void startExecutionIfInactive() {
    if(active.compareAndSet(false, true)) {
        pool.execute(this);
    }
}

@Override
public synchronized void run() {
    while(!queue.isEmpty()){
        queue.poll().run();
    }
    active.set(false); //all future adds will not be executed on this thread anymore
    if(!queue.isEmpty()) { //if some items were added to the queue after the last queue.poll
        startExecutionIfInactive();// trigger an execution on next thread
    }
}

Answer 2

If you want to execute your task sequentially simply create a ExecutorService with only one thread thanks to Executors.newSingleThreadExecutor().

If you have different type of tasks and you want to execute only the tasks of the same type sequentially, you can use the same single threaded ExecutorService for the same type of tasks, no need to reinvent the wheel.

So let's say that you have 1 000 different type of tasks, you could use 200 single threaded ExecutorService, the only thing that you need to implement yourself is the fact that you always need to use the same single threaded ExecutorService for a given type of task.

Answer 3

private Map<Integer, CompletableFuture<Void>> sessionTasks = new HashMap<>();
private ExecutorService pool = Executors.newFixedThreadPool(200);

public void submit(int sessionId, Runnable task) {  
    if (sessionTasks.containsKey(sessionId)) {
        sessionTasks.compute(sessionId, (i, c) -> c.thenRunAsync(task, pool));
    } else {
        sessionTasks.put(sessionId, CompletableFuture.runAsync(task, pool));
    }
}

If a session has no task, a new task is created and run in the provided pool. If a session already has a tasks when a new task is added, the latter is chained (with thenRun) to the previous one, ensuring order.

Answer 4

If you want to configure bounded queue, use ThreadPoolExecutor

ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, 
TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
RejectedExecutionHandler handler)

For your use case, use ThreadPoolExecutor as

ThreadPoolExecutor executor =    
ThreadPoolExecutor(1,1,60,TimeUnit.SECONDS,new ArrayBlockingQueue<Runnable>(1000));

Above code caps size of queue is ThreadPoolExecutor as 1000. If you want to use custom rejected execution handler, you can configure RejectedExeutionHandler.

Related SE question:

How to properly use Java Executor?

Answer 5

@Nicolas's answer is probably your best bet as it is simple, well tested, and efficient.

If however it can not meet your requirement, I would do it like so :

1. Do not make "SequentialPooledExecutor" an executor service, make it a facade for a "pool" of single thread executor services
2. Make your "SequentialPooledExecutor" implement a submit method (takes a Runnable / Callable, and a String that represents a "queue name"), returns a Future, like an executor service
3. On call of this method, make your "SequentialPooledExecutor" dispatch to one of its internal, single thread, executor service, by taking the hash of the queue name, and dispatching it to the corresponding internal executor.

The hashing part that takes place at step 3 allows you to have the tasks for each "queue name" always go to the same (single thread) executor service inside of your "SequentialPooledExecutor".

Another possible route is the use of CompletionStage and CompletableFutures. These are, in effect, listenable futures (that have a completion handler). With these, the first time you have a "session", you create a CompletableFuture with your first task, and hold on to it. At each new task, you combine the previous future with the new task, calling thenAcceptAsync (or any of the like). What you get is a linear chain of execution tasks.

Answer 6

If you have thousands of keys which must be processed sequentially, but you don't have thousands of cores you can use a hashing strategy to distribute the work like this

ExecutorService[] es = // many single threaded executors

public <T> Future<T> submit(String key, Callable<T> calls) {
    int h = Math.abs(key.hashCode() % es.length);
    return es[h].submit(calls);
}

In general you only need 2 * N threads to keep N cores busy, if your task is CPU bound, more than that just adds overhead.