bulkload-with-version-check_v2

[sszakal]

Currently Marten supports 3 bulk insert modes .

// Just say we have an array of documents we want to bulk insert
var data = Target.GenerateRandomData(100).ToArray();

using var store = DocumentStore.For("some connection string");

// Discard any documents that match the identity of an existing document
// in the database
await store.BulkInsertDocumentsAsync(data, BulkInsertMode.IgnoreDuplicates);

// This is the default mode, the bulk insert will fail if any duplicate
// identities with existing data or within the data set being loaded are detected
await store.BulkInsertDocumentsAsync(data, BulkInsertMode.InsertsOnly);

// Overwrite any existing documents with the same identity as the documents
// being loaded
await store.BulkInsertDocumentsAsync(data, BulkInsertMode.OverwriteExisting);

It would be great if Marten also supported an additional mode (or a separate BulkUpdate option) that integrates with IVersion optimistic concurrency.

Desired behavior:
• For each document in the batch:
• insert if it doesn’t exist
• update/overwrite only if IVersion matches the current stored version
• otherwise skip, rather than overwriting or throwing error

This would be very useful for online JSON migrations/backfills without taking the app down.

Example scenario:
• The application performs on-the-fly migrations when a document is loaded/edited after deployment.
• At the same time, a background job migrates all documents in bulk for speed.
• While processing a batch, some documents might already have been migrated/updated by the app due to live traffic.
• In this case, the bulk process should not overwrite those documents — it should skip updates when the stored IVersion has changed since the document was read.

IMPORTANT: I'm not familiar with marten internals . Its my first PR on this project and unfortunately I was not able to make the test pass on my machine even before making any changes to master branch (before the changes from this PR). I could only validate that my new tests are passing and the ones related to bulk loading

UPDATE: this overlaps a bit with #3301

[jeremydmiller]

@sszakal Tests are good when I ran this locally. In between you submitting this and now, we broke up our CI to side step the timeouts.

[jeremydmiller]

This is going to be a cool feature. First new big feature for the document db side of things in ages