Inconsistent tenant_id values break SingleStream async projection rebuilds

[alekbarszczewski]

Description

SingleStream async projections fail during rebuild/catchup when events in the same stream have inconsistent tenant_id values. Some events are tagged with "DEFAULT" while others are tagged with "Marten", causing the projection's Apply methods to receive documents with default/uninitialized property values.

Reproduction

When running an async SingleStream projection and events are already in the database (e.g., app restart with existing events), the projection produces incorrect results. The Apply method receives a document with all properties set to their default values, as if the Create method was never called.

Example projection

public record BillrunLookup
{
    [Identity]
    public required string BillrunId { get; init; }
    public required bool IsCompleted { get; init; }
    public required YearMonth SettlementMonth { get; init; }
    public required int BillrunSequence { get; init; }

    public static BillrunLookup Create(BillrunStarted started)
    {
        return new BillrunLookup
        {
            BillrunId = started.BillrunId,
            IsCompleted = false,
            SettlementMonth = started.ForMonth,
            BillrunSequence = started.BillrunSequence,
        };
    }

    public static BillrunLookup Apply(BillrunCompleted completed, BillrunLookup lookup)
    {
        // lookup has all properties set to defaults here!
        return lookup with { IsCompleted = true };
    }
}

public class BillrunLookupProjection : SingleStreamProjection { }

// Registration
opts.Projections.Add(ProjectionLifecycle.Async);

Expected result

{"BillrunId": "billrun-1", "IsCompleted": true, "BillrunSequence": 1, "SettlementMonth": "2025-12"}

Actual result

{"BillrunId": "billrun-1", "IsCompleted": true, "BillrunSequence": 0, "SettlementMonth": "0001-01"}

Root Cause

Events appended to the same stream have inconsistent tenant_id values in the database:

seq_id	stream_id	tenant_id
1	billrun-1	DEFAULT
2	billrun-1	Marten

This appears to be caused by the Wolverine.Marten integration when using [AggregateHandler] with IStartStream for stream creation and IEnumerable<object> for subsequent appends.

Handler example that produces inconsistent tenant IDs

public class StartBillrunHandler
{
    [AggregateHandler]
    public static IStartStream Handle(StartBillrun command)
    {
        var started = new BillrunStarted(/* ... */);
        return MartenOps.StartStream(command.BillrunId, started);
    }
}

public class CompleteBillrunHandler
{
    [AggregateHandler]
    public static IEnumerable Handle(CompleteBillrun command, SomeAggregate? aggregate)
    {
        // This event gets a different tenant_id
        return [new BillrunCompleted(command.BillrunId)];
    }
}

Workaround

Manually update the tenant_id column to a consistent value (e.g., "*DEFAULT*") for all events in the affected streams:

UPDATE mt_events SET tenant_id = '*DEFAULT*' WHERE tenant_id != '*DEFAULT*';

After this fix, projections work correctly.

Environment

• Marten: 8.9.0, 8.11.0, 8.16.4, 8.18.0 (confirmed on multiple versions)
• Wolverine.Marten: 5.10.0+
• Multi-tenancy: Not used (single tenant)
• Daemon mode: HotCold (single instance)

Additional observations

• Live aggregation works correctly (not affected)

• Only async projections during rebuild/catchup are affected
• The issue occurs when multiple events are processed in a single batch
• No exceptions are logged; no dead letter events are created

Notes from Jeremy Miller

"It's old. Something a client in Denmark noticed last year"
"Might cheat in Marten and quietly fix the tenant id on the read in"

---

Confirmed by: @alek, @Petteroe

[alekbarszczewski]

One note when I am looking on my streams: all streams created from http endpoints have DEFAULT (correct) tenant_id. The ones that have "Marten" tenant_id were created or appended during some asynchronous processing (marten/pg transport in my case, not local queues).

[Petteroe]

I don't know if this nuance matters, but my non-default tenant_id values are consistently "Main", not Alek's "Marten".

[jeremydmiller]

@Petteroe What, how? Okay, thinks for letting me know that one

[jeremydmiller]

Meh, going to take some coordinated changes in both JasperFx.Events & Marten. Whose idiotic idea was it to separate those libraries for future gains that haven't yet been realized? Oh, wait, it was me.

[jeremydmiller]

Moving this to Marten. Fix is there.