async/await and the Thread Pool

The problem: a sale day and a slow server

Our online shop has a product page. We get the price of each product from another service. Someone wrote this code:

app.MapGet("/products/{id}/price", (int id, PriceClient prices) =>
{
    var price = prices.GetPriceAsync(id).Result; // waits here
    return Results.Ok(price);
});

On normal days everything is fine. But on a sale day these things happen:

• The response time goes from a few tens of milliseconds to a few seconds.
• The server CPU usage is low. The server looks “idle”.
• The number of threads in the process goes up slowly and steadily.

To understand why, we first need to know what a thread is and what async/await really does.

Thread and Thread Pool in simple words

A Thread is a worker that runs code. The operating system creates it. Each thread has its own stack memory, and creating one has a cost.

That is why .NET does not create a new thread for each request. It has a Thread Pool: a few ready threads that take jobs from a queue. ASP.NET Core itself runs each request on a thread from this pool.

What does await really do?

Most of the work of an API is waiting: waiting for the database, for another service, for a file. During this time the CPU has nothing to do. The question is: is the thread also stuck during this time?

When we reach an await, these steps happen:

1. The code runs until the first await. For example, the HTTP request is built and sent.
2. The waiting is handed to the operating system. No thread is needed to wait for the network.
3. The method returns. It gives an unfinished Task to the caller, and the thread goes back to the pool.
4. The network answer arrives. The operating system tells us.
5. The rest of the method runs on a free thread. It may not be the same thread as before.

Why does Result make the server slow?

Now let us go back to the sale day code. The reason for the slowness in a few steps:

1. Each request takes a thread from the pool.
2. The Result property blocks that thread until the price answer arrives. The thread does nothing else.
3. When there are many requests, all threads in the pool are blocked. New requests stay in the queue.
4. The Thread Pool sees that the queue has grown and adds new threads. But it does this slowly on purpose. That is why the number of threads goes up slowly.
5. These threads only wait and do not compute. So the CPU stays low.

We call this situation Thread Pool Starvation. Its sign is exactly this: low CPU, but a slow system.

Live example

Run it once with Result and once with await. Watch the number of threads, the queue and the max response time:

The right code: async from start to end

The solution is that everything is async, from the endpoint to the lowest layer. We call this rule async all the way.

app.MapGet("/products/{id}/price",
    async (int id, PriceClient prices, CancellationToken ct) =>
    {
        var price = await prices.GetPriceAsync(id, ct);
        return Results.Ok(price);
    });

public sealed class PriceClient(HttpClient http)
{
    public async Task<decimal> GetPriceAsync(int productId, CancellationToken ct)
    {
        var text = await http.GetStringAsync($"prices/{productId}", ct);
        return decimal.Parse(text, CultureInfo.InvariantCulture);
    }
}

Several jobs together

The product page needs the price and the stock. These two do not depend on each other. So we can start both together and then wait for both:

var priceTask = prices.GetPriceAsync(id, ct);
var stockTask = stock.GetStockAsync(id, ct);

await Task.WhenAll(priceTask, stockTask);

var page = new ProductPage(id, await priceTask, await stockTask);

Now the total time is about the time of the slowest job, not the sum of both.

A limit on concurrency

Imagine we must get the price of 1000 products from the price service and put them in the cache. If we send all of them together, the other service gets overloaded. With the ForEachAsync method in the Parallel class we set a limit:

await Parallel.ForEachAsync(productIds,
    new ParallelOptions { MaxDegreeOfParallelism = 10, CancellationToken = ct },
    async (id, token) =>
    {
        var price = await prices.GetPriceAsync(id, token);
        latestPrices[id] = price; // a ConcurrentDictionary<int, decimal>
    });

A Task is different from a Thread

For thousands of requests at the same time, a separate thread for each request is not the right way. But there is one exception: if a background job always works with a blocking library, it is better to give it a dedicated thread. This way it does not take a pool thread forever. For this, we pass the LongRunning option to the StartNew method. This option only makes sense for blocking code. Async code goes back to the pool after the first await.

Canceling work with CancellationToken

A user opens the sales report page. The query takes two minutes. The user does not wait and closes the page. What happens?

1. The ASP.NET Core framework sees that the connection is closed.
2. It fires the cancel token of that request. This token is only a signal.
3. If our code did not give the token to EF Core, nobody tells the database “stop”. The query runs to the end and its result is thrown away.

app.MapGet("/reports/sales", async (ShopDb db, CancellationToken ct) =>
    await db.Orders
        .Where(o => o.CreatedAt >= DateTime.UtcNow.AddDays(-30))
        .GroupBy(o => o.CreatedAt.Date)
        .Select(g => new { Day = g.Key, Total = g.Sum(o => o.Total) })
        .ToListAsync(ct));

Heavy CPU work

If the job is really computing (for example, building a big PDF file), async does not help. The thread is really busy, not waiting.

• In ASP.NET Core, the Run method of the Task class has no benefit. It moves the work from the thread pool to the same thread pool. It only adds extra cost.
• In a desktop app it is useful. It moves heavy work off the UI thread so the screen does not freeze.
• For heavy work on a server, give the work to a queue and a background worker.

Important rules

1. Never use Result or Wait on a Task. In server code it means a blocked thread. In apps that have a SynchronizationContext (like WinForms), it can even cause a deadlock.
2. Do not write an async void method. Nobody catches an error inside it, and it can crash the whole process. The only exception is an event handler in desktop apps.
3. Always pass the cancel token. Every async method that does I/O should take a CancellationToken parameter.
4. Do not leave work running alone (Fire and Forget). If you leave a Task without await, its error is lost. When the request ends, Scoped services like DbContext are also disposed. For background work, use a queue and a BackgroundService.
5. In libraries, use ConfigureAwait with the value false. In ASP.NET Core it is not needed, because it has no SynchronizationContext. But a library does not know in which app it runs.
6. Choose the ValueTask type only when needed. It is for a method that answers without waiting most of the time (for example, from a cache) and is in hot code. It is awaited only once. The default is still Task.

Common mistakes

When to use async?