Stop Reentrancy on MemoryCache Calls

Question

The app needs to load data and cache it for a period of time. I would expect that if multiple parts of the app want to access the same cache key at the same time, the cache

Answer 1

MemoryCache leaves it to you to decide how to handle races to populate a cache key. In your case you don't want multiple threads to compete to populate a key presumably because it's expensive to do that.

To coordinate the work of multiple threads like that you need a lock, but using a C# lock statement in asynchronous code can lead to thread pool starvation. Fortunately, SemaphoreSlim provides a way to do async locking so it becomes a matter of creating a guarded memory cache that wraps an underlying IMemoryCache.

My first solution only had a single semaphore for the entire cache putting all cache population tasks in a single line which isn't very smart so instead here is more elaborate solution with a semaphore for each cache key. Another solution could be to have a fixed number of semaphores picked by a hash of the key.

sealed class GuardedMemoryCache : IDisposable
{
    readonly IMemoryCache cache;
    readonly ConcurrentDictionary semaphores = new();

    public GuardedMemoryCache(IMemoryCache cache) => this.cache = cache;

    public async Task GetOrCreateAsync(object key, Func> factory)
    {
        var semaphore = GetSemaphore(key);
        await semaphore.WaitAsync();
        try
        {
            return await cache.GetOrCreateAsync(key, factory);
        }
        finally
        {
            semaphore.Release();
            RemoveSemaphore(key);
        }
    }

    public object Get(object key) => cache.Get(key);

    public void Dispose()
    {
        foreach (var semaphore in semaphores.Values)
            semaphore.Release();
    }

    SemaphoreSlim GetSemaphore(object key) => semaphores.GetOrAdd(key, _ => new SemaphoreSlim(1));

    void RemoveSemaphore(object key)
    {
        if (semaphores.TryRemove(key, out var semaphore))
            semaphore.Dispose();
    }
}

If multiple threads try to populate the same cache key only a single thread will actually do it. The other threads will instead return the value that was created.

Assuming that you use dependency injection, you can let GuardedMemoryCache implement IMemoryCache by adding a few more methods that forward to the underlying cache to modify the caching behavior throughout your application with very few code changes.