You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Preserve Firestore canonical membership consistency while keeping membership operations lightweight.
Read each membership view through one native strong snapshot: a single RunQuery stream for the full table, or one BatchGetDocuments request for the version and requested row. Each membership tuple contains that entry's own document ETag; TableVersion carries the version document's separate ETag.
Apply canonical mutations with one atomic native Commit, enforcing the table-version CAS and row creation/existence precondition. Full-row updates use the supplied table version; the row ETag remains provider metadata, so heartbeat-only changes preserve original-input update usability.
Write the owning silo's heartbeat with one timestamp-only field update. Canonical membership fields and the table version remain stable; the updated row's document ETag can change. Caller cancellation is normalized and native faults propagate.
Select Dead rows once with a native status query, evaluate the strict cutoff against the latest start/heartbeat/suspect timestamp, and reuse conditional delete batches of up to 500 rows. Only a conflicted batch is refreshed under native transaction protection; cleanup preserves the canonical version.
Cover request/document counts, per-entry ETags, streamed snapshot consumption, canonical conflicts, original-input updates after heartbeats, native cancellation/errors, selected-batch cleanup races, and cross-batch error propagation with native SDK mocked regressions and emulator tests.
The read strategy follows Firestore's native snapshot-read guarantees; Commit writes are atomic with an optional transaction token. This removes redundant client transaction round trips while retaining native atomicity and preconditions.
Includes only the shared legacy cleanup-test prerequisite from #11308 (bede893d290a9035f196f4c3ed58213d102efc6c), cherry-picked as a separate commit. The provider changes are carved from the frozen conformance source and adapted to unchanged-version cleanup and sole-owner, canonically noninterfering heartbeats.
This transaction reads the entire collection, so Firestore's server-side transaction tracks/locks the version document and every active row even though only stale Dead rows can be deleted. A heartbeat or membership update on any of those documents can abort the cleanup (or block until the transaction timeout), making periodic cleanup unreliable in a busy cluster. Restrict the transactional query to Dead rows and limit it to the deletion batch while retaining the per-document update-time preconditions.
The newest successful coverage run tested 986e5c7, not current main dcf2925.
Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.
The comparison remains report-only while normal line and branch variance is calibrated.
Cleanup now sends a native Status == Dead equality query before the transactional document reads. Previously the query read the whole collection and only then filtered statuses in memory, unnecessarily adding live heartbeat rows and the version document to its read set. Dead-row compaction preserves the canonical version, so those reads are unnecessary.
Candidate recency still uses the strict cutoff over the latest start, heartbeat, and suspect timestamps. Transactional candidate reads and per-document update-time delete preconditions still protect changes to a candidate's status, heartbeat, or votes. Public ReadAll/ReadRow snapshot transactions are unchanged. The equality filter uses the existing Status field and normal single-field query support; no schema/layout or composite-index changes were added. Native Firestore locking and contention on matching candidates can still occur.
Two native-SDK fake race cases changed a live heartbeat, or live canonical fields plus the version, while cleanup committed. Both failed against the broad query due to an additional retry and pass with the native filter. The tests also assert the exact StructuredQuery predicate and retain all cutoff/candidate-race cases.
Verified on this exact head in CI run 35369437102: both Google Cloud jobs passed. Downloaded TRX results show 102/102 Firestore clustering tests passing on each of net8.0 and net10.0, including 52 native-SDK cases, Dead-row cleanup, and cleanup spanning multiple write batches.
ETag correction (05b5444): Read tuples now return each membership document's own physical ETag, separately from the table-version ETag. Canonical updates ignore the row ETag and use only table-version CAS plus native row existence, so an original-input update remains usable after a heartbeat. This supersedes the logical-row-token encoding/agreement statements in the historical reassessment below; request counts and native snapshot guarantees are unchanged.
Active Firestore reassessment
Implemented in c305598, after auditing the membership provider, its shared storage helpers, and gateway reads against freshly fetched main (0f9537dde1c10107d059fc10f1171f6f04a43f19). Those baseline files are identical to the PR's original d515d75e base.
Native guarantees established first
Firestore's read architecture documents timestamp-based, strongly consistent standalone reads across splits, with no read locks. This is an actual read-boundary guarantee, not an inference from atomic writes.
The RunQuery and BatchGetDocuments protocol default to strong consistency. Their stream contracts keep earlier results valid through subsequent read timestamps. Firestore SDK 4.3.0 Query.GetSnapshotAsync consumes one RunQuery stream; FirestoreDb.GetAllSnapshotsAsync consumes one BatchGet request and restores requested document order. These paths do not paginate with separate requests.
Native Commit applies all writes atomically and in order; its transaction token is optional. Document existence and update-time preconditions remain enforced by that atomic commit.
Request and document-read costs
Nominal successful-attempt counts below exclude authentication/transport traffic and SDK retries. N = membership rows, D = Dead rows, K = eligible Dead rows, K_main = the baseline cleanup's broader set of old non-Active rows, B = ceil(K/500).
Method
main requests
Previous PR requests (690e4854)
Reassessed requests
Document reads / writes
ReadRowAsync
4: Begin + 2 point reads + Commit
4
1 BatchGet
Version + requested row in one strong snapshot; 2 requested documents
0 client document reads; row-exists + version CAS; mismatched logical tokens return contention before a request
UpdateIAmAliveAsync
3: Begin + point read + Commit
1
1 field update
0 client reads; one timestamp field
CleanupDefunctSiloEntriesAsync
1 all-row query + ceil(K_main/500) commits
3(B+1), with repeated Dead scans
1 Dead query + B commits
D documents once; up to K conditional deletes
InitializeMembershipTableAsync
1 group read + optional header creation + optional version creation
Same
Same
Preserves existing shared bootstrap metadata/layout and native initialization failures
DeleteMembershipTableEntriesAsync
1 scoped query + chunked commits; another-cluster path also initializes shared group
Same
Same
Existing terminal deletion scope and bootstrap behavior retained
Gateway GetGateways
1 Active-row query
Same
Same
Existing native strong query already matches its boundary
Cleanup retains main's concurrent batch dispatch through Task.WhenAll. Successful batches get no refresh and no final empty query. A stale/missing candidate or native aborted commit refreshes only that selected batch (at most 500 references) inside the existing native transaction mechanism; it then re-evaluates Dead status and the exclusive max(start, heartbeat, suspect-time) cutoff before conditional deletion. With one uncontended refresh this adds Begin + one BatchGet + Commit for that batch, rather than rescanning the collection. Further commit contention follows the SDK's bounded transaction retry policy. Permission/network/resource failures remain observable, including a different batch's permission failure while another batch handles contention.
Retained boundaries and concrete evidence
FirestoreMembershipTable.cs:126,162: public reads use native strong snapshots, with missing-version errors preserved.
FirestoreMembershipTable.cs:192,234,348: native atomic writes retain canonical version CAS, row creation/existence, and logical-token agreement; version-document ETags remain independent of heartbeat physical timestamps.
FirestoreMembershipTable.cs:73: cleanup retains physical candidate update-time predicates because eligibility must still hold at deletion. Only a conflicted selected batch needs a read/write transaction.
FirestoreMembershipTable.cs:40,57 and FirestoreDataManager.cs:53,88: shared group bootstrap and scoped terminal deletion remain unchanged. The header read preserves the existing startup/storage-layout behavior; it is not a canonical snapshot fence.
FirestoreGatewayListProvider.cs:41: the existing single native query already provides the required gateway read.
FirestoreDataManager.cs:439: the existing generic cancellation adapter is now internal for reuse; its implementation and other providers' behavior are unchanged.
All normal public reads and canonical writes now have zero explicit client transactions. The server still coordinates atomic writes internally, and the canonical version document remains a genuine serialization point. This is not a claim of zero backend locking or contention. GAX's existing Commit retries for Unavailable/ResourceExhausted and BatchGet's transient-read retries remain native policy; no provider heartbeat/CAS loop was added.
Tests which forced raw heartbeat clobbering were removed from canonical-token scenarios: lag/overwrite is permitted, not required. No compatibility flags, schema sidecars, markers, rollout protocol, or public API changes were introduced.
Scan checklist and validation
Bounded scan of the three production files: 0 hits for async-void, sync-over-async, Task.Run, ValueTask misuse candidates, BlockingCollection, substring/string-culture conversions, string replacements, params arrays, temporary byte buffers, stackalloc, string.Format, regex constructors/compiled regex, ContainsKey double-lookup candidates, JSON options/serialization, HttpClient construction, static Dictionary/FrozenDictionary, per-call List/Dictionary allocation, and culture comparers. Nine LINQ candidate lines and three unsealed internal classes (0/3 sealed) were inspected; network-operation costs warranted the changes above, not unrelated micro-optimizations. Existing native query/batch usage in gateway reads and shared clear/delete helpers provided local prior art.
Focused local build/tests: 76/76 passed. Exact-head CI run 35374375696: both Google Cloud jobs passed. Downloaded TRX files confirm 126/126 Firestore clustering tests on each of net8.0 and net10.0, including streamed snapshot changes, out-of-order BatchGet responses, 500+1 cleanup batches, targeted refresh, genuine canonical conflicts, native cancellation, cross-batch failure propagation, and existing emulator concurrent-read/update tests. Latest inline-thread audit is clear; the current-head automated review reports no findings and calls for human review. Remaining overall CI jobs were still running at this report; no failed checks were present.
Assessment
Count
Outcome
Critical findings
0
None identified in the bounded audit
Moderate native-cost findings
3
Removed redundant read transactions, write-only transactions, and repeated cleanup scans
Unmeasured micro-optimization recommendations
0
No unrelated tuning proposed
This is an AI-assisted static/native-operation assessment, not a throughput benchmark. Request counts and correctness checks are verified; production performance and the final design still require human review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preserve Firestore canonical membership consistency while keeping membership operations lightweight.
The read strategy follows Firestore's native snapshot-read guarantees; Commit writes are atomic with an optional transaction token. This removes redundant client transaction round trips while retaining native atomicity and preconditions.
Includes only the shared legacy cleanup-test prerequisite from #11308 (
bede893d290a9035f196f4c3ed58213d102efc6c), cherry-picked as a separate commit. The provider changes are carved from the frozen conformance source and adapted to unchanged-version cleanup and sole-owner, canonically noninterfering heartbeats.Microsoft Reviewers: Open in CodeFlow