The Outbox and Inbox patterns

The problem: two writes, no shared transaction

A customer places an order. The order service does two things:

1. It saves the order in its own database.
2. It sends the message “order placed” (OrderPlaced) to Kafka or RabbitMQ. The inventory service uses this message to reserve the items.

The simple code looks like this:

db.Orders.Add(order);
await db.SaveChangesAsync(ct);
await bus.PublishAsync(new OrderPlaced(order.Id, order.Lines), ct);

It looks correct. But the database and the Broker are two separate systems. No transaction covers both of them together. This problem is called Dual Write.

What can go wrong?

1. The app crashes between the two lines. For example, during a new release, the Pod is shut down. The order exists, the message does not. Inventory never reserves the items.
2. The Broker is not available for a few seconds. Sending the message fails. The order is already saved and does not roll back.
3. If we swap the order, the problem flips. If we send the message first and then the save fails, inventory reserves items for an order that does not exist at all.

The Outbox idea: write the message to the database too

The database guarantees only one thing: all writes inside one transaction happen together. So we write the message to the same database too:

1. In one transaction, we save both the order and a row in the OutboxMessages table. This row is the message text.
2. A background job (Relay) reads the unsent rows and sends them to the Broker.
3. After a successful send, it marks the row as “sent”.

Why does this solve the problem?

1. If the transaction succeeded, the message is surely in the table.
2. If the app crashed, after it starts again, the background job finds the same row and sends it.
3. If the Broker was down for a few minutes, the messages wait in the table. No message is lost.

The Inbox idea: recognize duplicate messages

Duplicate messages do not only come from Outbox. Kafka and RabbitMQ may also deliver a message again. For example, the consumer processed the message but crashed before confirming it (an Ack, or a Commit of the Offset).

The Inbox pattern is the mirror of Outbox on the receiver side:

1. Each message has a unique ID (MessageId).
2. The inventory service has a table called InboxMessages. Its primary key is the message ID.
3. In one transaction, it both records the message ID in the Inbox and reserves the items.
4. If the same message arrives again, recording the ID fails with a duplicate key error. The whole transaction rolls back and the items are not reserved again.
5. We confirm (Ack) the message only after a successful save.

Live example

Turn the options on or off and run each scenario. See how many reservations inventory records in each case:

Code

Saving the order and the message in one transaction

The Outbox table is simple: the message ID, the type, the text and the send time.

public sealed class OutboxMessage
{
    public long Id { get; init; }                         // keeps the order of messages
    public Guid MessageId { get; init; } = Guid.NewGuid(); // consumers use it to find duplicates
    public required string Type { get; init; }
    public required string Payload { get; init; }
    public DateTimeOffset? SentAt { get; set; }
}

In EF Core, one call to the SaveChangesAsync method is a transaction by itself. So both rows are saved together:

app.MapPost("/orders", async (PlaceOrder cmd, ShopDbContext db, CancellationToken ct) =>
{
    var order = Order.Place(cmd.CustomerId, cmd.Lines);
    db.Orders.Add(order);
    db.OutboxMessages.Add(new OutboxMessage
    {
        Type = nameof(OrderPlaced),
        Payload = JsonSerializer.Serialize(new OrderPlaced(order.Id, order.Lines))
    });

    await db.SaveChangesAsync(ct); // one transaction: order + message
    return Results.Created($"/orders/{order.Id}", order.Id);
});

The background job that sends the messages

public sealed class OutboxRelay(
    IServiceScopeFactory scopes, IMessagePublisher publisher, ILogger<OutboxRelay> logger)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken ct)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));
        while (await timer.WaitForNextTickAsync(ct))
        {
            try { await SendBatchAsync(ct); }
            catch (Exception ex) when (!ct.IsCancellationRequested)
            {
                logger.LogError(ex, "Outbox batch failed. Will try again.");
            }
        }
    }

    private async Task SendBatchAsync(CancellationToken ct)
    {
        await using var scope = scopes.CreateAsyncScope();
        var db = scope.ServiceProvider.GetRequiredService<ShopDbContext>();

        var batch = await db.OutboxMessages
            .Where(m => m.SentAt == null)
            .OrderBy(m => m.Id)
            .Take(100)
            .ToListAsync(ct);

        foreach (var message in batch)
        {
            // IMessagePublisher is our own small wrapper around Kafka or RabbitMQ.
            await publisher.PublishAsync(message.MessageId, message.Type, message.Payload, ct);
            message.SentAt = DateTimeOffset.UtcNow;
            await db.SaveChangesAsync(ct);
        }
    }
}

If the app crashes between sending and saving the mark, the message is sent again. This is normal. The Inbox on the receiver side catches it.

A consumer with Inbox

public sealed class OrderPlacedHandler(InventoryDbContext db)
{
    public async Task HandleAsync(Guid messageId, OrderPlaced message, CancellationToken ct)
    {
        db.InboxMessages.Add(new InboxMessage(messageId, DateTimeOffset.UtcNow));
        db.Reservations.Add(Reservation.For(message.OrderId, message.Lines));

        try
        {
            await db.SaveChangesAsync(ct); // inbox row + reservation, together
        }
        catch (DbUpdateException ex) when (IsDuplicateKey(ex))
        {
            // Same MessageId was processed before: do nothing.
        }
        // Ack the message only after this method returns.
    }

    private static bool IsDuplicateKey(DbUpdateException ex) =>
        ex.InnerException is PostgresException { SqlState: PostgresErrorCodes.UniqueViolation };
}

How you detect a duplicate key error depends on the database. Here it is PostgreSQL with the Npgsql library.

Ready-made: MassTransit

You do not have to write all of this yourself. The MassTransit library has a ready-made Outbox for EF Core. It builds the same tables and background job, and it also detects duplicate messages on the receiver side:

builder.Services.AddMassTransit(x =>
{
    x.AddEntityFrameworkOutbox<ShopDbContext>(o =>
    {
        o.UsePostgres();
        o.UseBusOutbox(); // published messages go to the outbox table first
    });

    x.UsingRabbitMq((context, cfg) => cfg.ConfigureEndpoints(context));
});

Important rules

1. The message and the data in one transaction. If the Outbox row is saved in a different transaction, the same Dual Write problem comes back.
2. Expect duplicate messages. Outbox gives at-least-once delivery. So every consumer must be Idempotent.
3. Each message has a unique ID. This ID stays the same in all send attempts. The receiver uses it to recognize duplicates.
4. Confirm the message after saving. If you Ack before processing and the app crashes, the message is lost.
5. Be careful with several Pods. If several copies of the background job read the same row at the same time, the message goes out several times. Either run only one copy, or lock the rows (for example with the FOR UPDATE SKIP LOCKED command in PostgreSQL). If the order of messages matters, make this choice carefully.
6. Clean up the tables. Delete old Outbox and Inbox rows after a while. Keep the Inbox table at least as long as the longest possible time for a duplicate message to arrive.
7. Treat work outside the database separately. Inbox makes only changes in the same database happen exactly once. If you send an SMS and crash before saving, the SMS goes out again. For this kind of work, give the outside service a unique ID, if it supports one.

Common mistakes

When to use Outbox and Inbox?