“Closing” a blocking queue

Question

I’m using java.util.concurrent.BlockingQueue in a very simple producer-consumer scenario. E.g. this pseudo code depicts the consumer part:

class QueueCo         


        

Answer 1

Today I solved this problem using a wrapper object. Since the ComplexObject is too complex to subclass I wrapped the ComplexObject into ComplexObjectWrapper object. Then used ComplexObjectWrapper as the generic type.

public class ComplexObjectWrapper {
ComplexObject obj;
}

public class EndOfQueue extends ComplexObjectWrapper{}

Now instead of BlockingQueue<ComplexObject> I did BlockingQueue<ComplexObjectWrapper>

Since I had control of both the Consumer and Producer this solution worked for me.

Answer 2

An alternative would be to wrap the processing you're doing with an ExecutorService, and let the ExecutorService itself control whether or not jobs get added to the queue.

Basically, you take advantage of ExecutorService.shutdown(), which when called disallows any more tasks from being processed by the executor.

I'm not sure how you're currently submitting tasks to the QueueConsumer in your example. I've made the assumption that you have some sort of submit() method, and used a similar method in the example.

import java.util.concurrent.*;

class QueueConsumer {
    private final ExecutorService executor = Executors.newSingleThreadExecutor();

    public void shutdown() {
        executor.shutdown(); // gracefully shuts down the executor
    }

    // 'Result' is a class you'll have to write yourself, if you want.
    // If you don't need to provide a result, you can just Runnable
    // instead of Callable.
    public Future<Result> submit(final ComplexObject complexObject) {
        if(executor.isShutdown()) {
            // handle submitted tasks after the executor has been told to shutdown
        }

        return executor.submit(new Callable<Result>() {
            @Override
            public Result call() {
                return process(complexObject);
            }
        });
    }

    private Result process(final ComplexObject complexObject) {
        // Do something with the complex object.
    }
}

This example is just an off-the-cuff illustration of what the java.util.concurrent package offers; there are probably some optimizations that could be made to it (e.g., QueueConsumer as its own class probably isn't even necessary; you could just provide the ExecutorService to whatever producers are submitting the tasks).

Dig through the java.util.concurrent package (starting at some of the links above). You might find that it gives you a lot of great options for what you're trying to do, and you don't even have to worry about regulating the work queue.

Answer 3

Is it possible to extend ComplexObject and mock out the non-trivial creation functionality? Essentially you're ending up with a shell object but you can do then do instance of to see if is the end of queue object.

Answer 4

You can wrap your generic object into a dataobject. On this dataobject you can add additional data like the poison object status. The dataobject is a class with 2 fields. T complexObject; and boolean poison;.

Your consumer takes the data objects from the queue. If a poison object is returned, you close the consumer, else you unwrap the generic and call 'process(complexObject)'.

I'm using a java.util.concurrent.LinkedBlockingDeque<E> so that you can add object at the end of the queue and take them from the front. That way your object will be handled in order, but more important it's safe to close the queue after you run into the poison object.

To support multiple consumers, I add the poison object back onto the queue when I run into it.

public final class Data<T> {
    private boolean poison = false;
    private T complexObject;

    public Data() {
        this.poison = true;
    }

    public Data(T complexObject) {
        this.complexObject = complexObject;
    }

    public boolean isPoison() {
        return poison;
    }

    public T getComplexObject() {
        return complexObject;
    }
}

public class Consumer <T> implements Runnable {

    @Override
    public final void run() {
        Data<T> data;
        try {
            while (!(data = queue.takeFirst()).isPoison()) {
                process(data.getComplexObject());
            }
        } catch (final InterruptedException e) {
            Thread.currentThread().interrupt();
            return;
        }
        // add the poison object back so other consumers can stop too.
        queue.addLast(line);
    }
}