Stop Reentrancy on MemoryCache Calls

Question

The app needs to load data and cache it for a period of time. I would expect that if multiple parts of the app want to access the same cache key at the same time, the cache

Answer 1

Here is a custom extension method GetOrCreateExclusiveAsync, similar to the native IMemoryCache.GetOrCreateAsync, that prevents concurrent invocations of the supplied asynchronous lambda under normal conditions. The intention is to enhance the efficiency of the caching mechanism under heavy usage. There is still the possibility for concurrency to occur, so this is not a substitute for thread synchronization (if needed).

This implementation also evicts faulted tasks from the cache, so that the failed asynchronous operations are subsequently retried.

using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Primitives;

/// 

/// Returns an entry from the cache, or creates a new cache entry using the
/// specified asynchronous factory method. Concurrent invocations are prevented,
/// unless the entry is evicted before the completion of the delegate. The errors
/// of failed invocations are not cached.
/// 
public static Task GetOrCreateExclusiveAsync(this IMemoryCache cache, object key,
    Func> factory, MemoryCacheEntryOptions options = null)
{
    if (!cache.TryGetValue(key, out Task task))
    {
        var entry = cache.CreateEntry(key);
        if (options != null) entry.SetOptions(options);
        var cts = new CancellationTokenSource();
        var newTaskTask = new Task>(async () =>
        {
            try { return await factory().ConfigureAwait(false); }
            catch { cts.Cancel(); throw; }
            finally { cts.Dispose(); }
        });
        var newTask = newTaskTask.Unwrap();
        entry.ExpirationTokens.Add(new CancellationChangeToken(cts.Token));
        entry.Value = newTask;
        entry.Dispose(); // The Dispose actually inserts the entry in the cache
        if (!cache.TryGetValue(key, out task)) task = newTask;
        if (task == newTask)
            newTaskTask.RunSynchronously(TaskScheduler.Default);
        else
            cts.Dispose();
    }
    return task;
}

Usage example:

var cache = new MemoryCache(new MemoryCacheOptions());
string html = await cache.GetOrCreateExclusiveAsync(url, async () =>
{
    return await httpClient.GetStringAsync(url);
}, new MemoryCacheEntryOptions().SetAbsoluteExpiration(TimeSpan.FromMinutes(10)));

This implementation uses nested tasks (Task>) instead of lazy tasks (Lazy>) internally as wrappers, because the later construct is susceptible to deadlocks under some conditions.
Reference: Lazy with asynchronous initialization, VSTHRD011 Use AsyncLazy.

---

Related API suggestion on GitHub: GetOrCreateExclusive() and GetOrCreateExclusiveAsync(): Exclusive versions of GetOrCreate() and GetOrCreateAsync()