Caching Patterns

The problem: a database under read load

In our shop, each product page is read from the database on every request. A product’s price and description change a few times a day, but they are read thousands of times. The result:

1. The database CPU is high.
2. Pages get slow.
3. With every ad campaign, the database reaches its limit.

The idea of a cache is simple: read data that is read a lot and changes rarely once, and keep a copy of it in a faster place.

The Cache-Aside pattern

The most common pattern is that the app manages the cache itself:

1. First, look for the key in the cache. For example, the key for product 42.
2. If it is there, return it.
3. If it is not, read it from the database. Then put it in the cache with an expiration time and return it.

Why is an expiration time (TTL) always needed? Because sooner or later, a bug somewhere means the cache is not cleared. The expiration time makes sure stale data stays only a few minutes, not forever.

Where should the cache be?

The problem with a local cache shows up when you have several servers. An admin changes a price. Only the server that got the edit request clears its own cache:

The common fix is to combine both: a local cache with a short expiration, in front of a Redis cache with a longer expiration. In .NET, the HybridCache class does exactly this.

Code with HybridCache

The HybridCache class comes from the Microsoft.Extensions.Caching.Hybrid package. If you also register a distributed cache like Redis, it uses it as the second level:

builder.Services.AddStackExchangeRedisCache(o =>
    o.Configuration = builder.Configuration.GetConnectionString("Redis"));

builder.Services.AddHybridCache(o =>
{
    o.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromMinutes(10),          // Redis
        LocalCacheExpiration = TimeSpan.FromSeconds(30) // memory of each server
    };
});

Reading and removing a product:

public sealed class ProductService(ShopDb db, HybridCache cache)
{
    // Note: a missing product (null) is cached too, until it expires.
    public async Task<ProductDto?> GetAsync(int id, CancellationToken ct) =>
        await cache.GetOrCreateAsync(
            $"product:{id}",
            async token => await db.Products
                .Where(p => p.Id == id)
                .Select(p => new ProductDto(p.Id, p.Name, p.Price))
                .FirstOrDefaultAsync(token),
            cancellationToken: ct);

    public async Task ChangePriceAsync(int id, decimal price, CancellationToken ct)
    {
        await db.Products
            .Where(p => p.Id == id)
            .ExecuteUpdateAsync(s => s.SetProperty(p => p.Price, price), ct);

        await cache.RemoveAsync($"product:{id}", ct); // first the database, then the cache
    }
}

The GetOrCreateAsync method does the same three Cache-Aside steps. It also has one more benefit that we will see below.

Invalidating the cache: remove, do not update

When the price changes, we have two ways: write the new value into the cache, or remove the key. Removing is safer. Imagine two admins change the price at the same time:

The right order is:

1. First, change the database.
2. Then remove the cache key.
3. The next first read brings the new value from the database and caches it again.

Cache Stampede

The “home page settings” key is read on every request, about 2000 times a second. Its expiration time is 10 minutes. Every 10 minutes, the database CPU hits 100 percent for a few seconds. Why?

1. While the key exists, all requests get the answer from the cache.
2. At the moment of expiry, all requests see the cache as empty.
3. All of them send the same query to the database together.
4. This goes on until one of them fills the cache again. If the query is slow, thousands of concurrent queries arrive.

Click the buttons and compare the database load:

The fixes:

1. One builder per key. Only one request builds the data, and the rest wait for its result. The GetOrCreateAsync method in HybridCache does this inside each server. So with 5 servers, at most 5 queries.
2. Refresh before expiry. For a few very important keys, a background job fills the cache again a little before it expires.
3. A little randomness in the expiration time (Jitter). If thousands of keys were created together, they also expire together. A little randomness spreads them out.

If Redis goes down

The cache must be optional. This means the site gets slower without the cache, but does not go down:

1. Catch cache errors and read directly from the database.
2. Set a short timeout for Redis. Otherwise every request waits a few seconds for a dead Redis, and the whole site gets slow.
3. Know your database capacity. If the database cannot handle the load at all without the cache, the cache is no longer “optional”. Have a plan for this case.

What not to cache?

1. Data that must always be exact. Like a wallet balance, or the stock at the moment of purchase.
2. Data that changes sooner than the expiration time, when stale data matters.
3. Data that is rarely read. The cache only uses memory and gives no benefit.
4. A “not found” answer, without thinking. If a product does not exist and you cache that, when it is created it is not seen until expiry. If you need it, set a short expiration time.

Common mistakes