Java concurrency - improving a copy-on-read collection

Question

I have a multithreaded application, where a shared list has write-often, read-occasionally behaviour.

Specifically, many threads will dump data into the list, and th

Answer 1

synchronized (~20 ns) is pretty fast and even though other operations can allow concurrency, they can be slower.

private final Lock lock = new ReentrantLock();
private List<T> items = new ArrayList<T>();

public void add(T item) {
    lock.lock();
    // trivial lock time.
    try {
        // Add item while holding the lock.
        items.add(item);
    } finally {
        lock.unlock();
    }
}

public List<T> makeSnapshot() {
    List<T> copy = new ArrayList<T>(), ret;
    lock.lock();
    // trivial lock time.
    try {
        ret = items;
        items = copy;
    } finally {
        lock.unlock();
    }
    return ret;
}

public static void main(String... args) {
    long start = System.nanoTime();
    Main<Integer> ints = new Main<>();
    for (int j = 0; j < 100 * 1000; j++) {
        for (int i = 0; i < 1000; i++)
            ints.add(i);
        ints.makeSnapshot();
    }
    long time = System.nanoTime() - start;
    System.out.printf("The average time to add was %,d ns%n", time / 100 / 1000 / 1000);
}

prints

The average time to add was 28 ns

This means if you are creating 30 million entries per second, you will have one thread accessing the list on average. If you are creating 60 million per second, you will have concurrency issues, however you are likely to be having many more resourcing issue at this point.

Using Lock.lock() and Lock.unlock() can be faster when there is a high contention ratio. However, I suspect your threads will be spending most of the time building the objects to be created rather than waiting to add the objects.

Answer 2

You can use a ReadWriteLock to allow multiple threads to perform add operations on the backing list in parallel, but only one thread to make the snapshot. While the snapshot is being prepared all other add and snapshot request are put on hold.

A ReadWriteLock maintains a pair of associated locks, one for read-only operations and one for writing. The read lock may be held simultaneously by multiple reader threads, so long as there are no writers. The write lock is exclusive.

class CopyOnReadList<T> {

    // free to use any concurrent data structure, ConcurrentLinkedQueue used as an example
    private final ConcurrentLinkedQueue<T> items = new ConcurrentLinkedQueue<T>();
    private final ReadWriteLock rwLock = new ReentrantReadWriteLock();
    private final Lock shared = rwLock.readLock();
    private final Lock exclusive = rwLock.writeLock(); 

    public void add(T item) {
        shared.lock(); // multiple threads can attain the read lock
        // try-finally is overkill if items.add() never throws exceptions
        try {
          // Add item while holding the lock.
          items.add(item);
        } finally {
          shared.unlock();
        }
    }

    public List<T> makeSnapshot() {
        List<T> copy = new ArrayList<T>(); // probably better idea to use a LinkedList or the ArrayList constructor with initial size
        exclusive.lock(); // only one thread can attain write lock, all read locks are also blocked
        // try-finally is overkill if for loop never throws exceptions
        try {
          // Make a copy while holding the lock.
          for (T t : items) {
            copy.add(t);
          }
        } finally {
          exclusive.unlock();
        }
        return copy;
    }

}

Edit:

The read-write lock is so named because it is based on the readers-writers problem not on how it is used. Using the read-write lock we can have multiple threads achieve read locks but only one thread achieve the write lock exclusively. In this case the problem is reversed - we want multiple threads to write (add) and only thread to read (make the snapshot). So, we want multiple threads to use the read lock even though they are actually mutating. Only thread is exclusively making the snapshot using the write lock even though snapshot only reads. Exclusive means that during making the snapshot no other add or snapshot requests can be serviced by other threads at the same time.

As @PeterLawrey pointed out, the Concurrent queue will serialize the writes aqlthough the locks will be used for as minimal a duration as possible. We are free to use any other concurrent data structure, e.g. ConcurrentDoublyLinkedList. The queue is used only as an example. The main idea is the use of read-write locks.

Answer 3

Yes, there is a way. It is similar to the way ConcurrentHashMap made, if you know.

You should make your own data structure not from one list for all writing threads, but use several independent lists. Each of such lists should be guarded by it's own lock. .add() method should choose list for append current item based on Thread.currentThread.id (for example, just id % listsCount). This will gives you good concurrency properties for .add() -- at best, listsCount threads will be able to write without contention.

On makeSnapshot() you should just iterate over all lists, and for each list you grab it's lock and copy content.

This is just an idea -- there are many places to improve it.

Answer 4

You could use a ConcurrentDoublyLinkedList. There is an excellent implementation here ConcurrentDoublyLinkedList.

So long as you iterate forward through the list when you make your snapshot all should be well. This implementation preserves the forward chain at all times. The backward chain is sometimes inaccurate.

Answer 5

First of all, you should investigate if this really is too slow. Adds to ArrayLists are O(1) in the happy case, so if the list has an appropriate initial size, CopyOnReadList.add is basically just a bounds check and an assignment to an array slot, which is pretty fast. (And please, do remember that CopyOnReadList was written to be understandable, not performant.)

If you need a non-locking operation, you can have something like this:

class ConcurrentStack<T> {
    private final AtomicReference<Node<T>> stack = new AtomicReference<>();

    public void add(T value){
        Node<T> tail, head;
        do {
            tail = stack.get();
            head = new Node<>(value, tail);
        } while (!stack.compareAndSet(tail, head));
    }
    public Node<T> drain(){
        // Get all elements from the stack and reset it
        return stack.getAndSet(null);
    }
}
class Node<T> {
    // getters, setters, constructors omitted
    private final T value;
    private final Node<T> tail;
}

Note that while adds to this structure should deal pretty well with high contention, it comes with several drawbacks. The output from drain is quite slow to iterate over, it uses quite a lot of memory (like all linked lists), and you also get things in the opposite insertion order. (Also, it's not really tested or verified, and may actually suck in your application. But that's always the risk with using code from some random dude on the intertubes.)