How to implement a most-recently-used cache

Question

What would be the best way to implement a most-recently-used cache of objects?

Here are the requirements and restrictions...

• Objects are stored as key/

Answer 1

Why implement something already implemented? Use Ehcache.

However, if third-party libraries are totally out of the question, I guess you are looking to implement a data structure which looks something like this:

• Basically is a HashMap (extends HashMap<Object, Object> if you will)
• Each value in the Map points to an object in a sorted list, based on which is most used.
• Objects recently used are added to the head of the list - O(1)
• Purging least-recently used means truncating the end of the list - O(1)
• Still gives you Map lookups, but still keeps recently-used items first...

Answer 2

A ConcurrentLinkedHashMap is difficult to construct, due to the locking requirements. A LinkedHashMap with a lock is straightforward, but not always performant. A concurrent version would attempt to reduce the amount of locking, either by lock splitting or ideally making CAS operations to make locking very cheap. If the CAS operations ever do become expensive, then similarly bucket splitting can be helpful. As an LRU requires writes for every access operation, and uses a doubly-linked list, this is very tricky to implement with pure CAS operations. I've attempted it, but I need to continue to mature my algorithm. If you search for ConcurrentLinkedHashMap, you'll see my project page...

If Java ME doesn't support CAS operations, which I'd expect to be true, then basic synchronization is all you can do. This is probably good enough with a LHM, given that I've only seen performance problems at high thread count on the server-side. So +1 to the answers above.

Answer 3

Another approach may be to take a look at section 5.6 in Java Concurrency in Practice by Brian Goetz: "Building an efficient, scalable result cache". Take a look at the Memoizer, although you may have to customize it for your purposes.

As an aside, what I cannot figure out is why Java does not have a ConcurrentLinkedHashMap out of the box. This data structure would be very helpful for building a cache.

Answer 4

Java Collections provide LinkedHashMap out of the box, which is well-suited to building caches. You probably don't have this in Java ME, but you can grab the source code here:

http://kickjava.com/src/java/util/LinkedHashMap.java.htm

If you can't just copy-paste it, looking at it should get you started implementing one for inclusion in your mobile app. The basic idea is just to include a linked list through the map elements. If you keep this updated whenever someone does put or get, you can efficiently track access order and use order.

The docs contain instructions for building an MRU Cache by overriding the removeEldestEntry(Map.Entry) method. All you really have to do is make a class that extends LinkedHashMap and override the method like so:

private static final int MAX_ENTRIES = 100;

protected boolean removeEldestEntry(Map.Entry eldest) {
   return size() > MAX_ENTRIES;
}

There's also a constructor that lets you specify whether you want the class to store things in order by insertion or by use, so you've got a little flexibility for your eviction policy, too:

public LinkedHashMap(int initialCapacity,
                     float loadFactor,
                     boolean accessOrder)

Pass true for use-order and false for insertion order.