Avoiding multiple repopulations of the same cache region (due to concurrency)

Question

I have a high traffic website and I use hibernate. I also use ehcache to cache some entities and queries which are required to generate the pages.

The problem is \"paral

Answer 1

I'm not quite sure, but:

It allows concurrent read access to elements already in the cache. If the element is null, other reads will block until an element with the same key is put into the cache.

Doesn't it means that Hibernate would wait until some other thread places the object into cache? That's what you observe, right?

Hib and cache works like this:

1. Hib gets a request for an object
2. Hib checks if the object is in cache -- cache.get()
3. No? Hib loads the object from DB and puts into cache -- cache.put()

So if the object is not in cache (not placed there by some previous update operation), Hib would wait on 1) forever.

I think you need a cache variant where the thread only waits for an object for a short time. E.g. 100ms. If the object is not arrived, the thread should get null (and thus Hibernate will load the object from DB and place into the cache).

Actually, a better logic would be:

1. Check that another thread is requesting the same object
2. If true, wait for long (500ms) for the object to arrive
3. If not true, return null immediately

(We cannot wait on 2 forever, as the thread may fail to put the object into cache -- due to exception).

If BlockingCache doesn't support this behaviour, you need to implement a cache yourself. I did it in past, it's not hard -- main methods are get() and put() (though API apparently has grown since that).

UPDATE

Actually, I just read the sources of BlockingCache. It does exactly what I said -- lock and wait for timeout. Thus you don't need to do anything, just use it...

public Element get(final Object key) throws RuntimeException, LockTimeoutException {
    Sync lock = getLockForKey(key);
    Element element;
        acquiredLockForKey(key, lock, LockType.WRITE);
        element = cache.get(key);
        if (element != null) {
            lock.unlock(LockType.WRITE);
        }
    return element;
}

public void put(Element element) {
    if (element == null) {
        return;
    }
    Object key = element.getObjectKey();
    Object value = element.getObjectValue();

    getLockForKey(key).lock(LockType.WRITE);
    try {
        if (value != null) {
            cache.put(element);
        } else {
            cache.remove(key);
        }
    } finally {
        getLockForKey(key).unlock(LockType.WRITE);
    }
}

So it's kind of strange it doesn't work for you. Tell me something: in your code this spot:

Ehcache cache = manager.getEhcache("foo");

is it synchronized? If multiple requests come at the same time, will there be only one instance of cache?

Answer 2

The biggest improvement on this issue is that ehcache now (since 2.1) supports the transactional hibernate cache policy. This vastly mitigates the problems described in this issue.

In order to go a step further (lock threads while accessing the same query cache region) one would need to implement a QueryTranslatorFactory to return custom (extended) QueryTranslatorImpl instances which would inspect query and parameters and block as necessary in the list method. This of course regards the specific use case of query cache using hql which fetch many entities.