Repository navigation
Description
Observed failure
The Google Cloud providers net8.0 CI job failed while FirestorePersistenceGrainTests.Fixture was building its test cluster. All 11 failures in that class share the same fixture-initialization exception; their test bodies did not reach the provider operations.
- PR: feat(messaging): deliver outbox messages #11285
- Head:
7a4492a3291d31f2cc4ddb5441bebd352f1f8508 - Run: https://github.com/dotnet/orleans/actions/runs/35657543374
- Job: https://github.com/dotnet/orleans/actions/runs/35657543374/job/106524796177
- Artifact:
test_output_test-google_net8.0, ID10664878571 - Firestore assembly TRX: 37 total, 26 passed, 11 failed, 0 skipped
- TRX SHA256:
DB50A65B9CAE3FBAF2637FE7052F27D6AFC67111DCBEA01E9613252DEABDF47A
Class fixture type 'Orleans.Persistence.Firestore.Tests.FirestorePersistenceGrainTests+Fixture'
threw in InitializeAsync
---- System.InvalidOperationException: Cannot find enough free ports to spin up a cluster
TestClusterPortAllocator.GetAvailableConsecutiveServerPorts(...): line 158
TestClusterPortAllocator.AllocateConsecutivePortPairs(...): line 35
TestClusterBuilder.ConfigureDefaultPorts(): line 225
TestClusterBuilder.Build(): line 166
BaseTestClusterFixture.InitializeAsync(): line 89
The affected cases include ClearStateAsync_Before_WriteStateAsync, Grain_GrainStorage_Read, the key-type read/write cases, and Grain_GrainStorage_SiloRestart. The downloaded TRX confirms that every failed row has this same fixture exception. Java and Google emulator setup succeeded; the Test step failed. Evidence comes from the completed job's direct log endpoint and its actual test artifact, not an inferred failure from the check name.
Scope and investigation
The failing path is port allocation before cluster deployment. Orleans.TestingHost and the shared fixture infrastructure are unchanged from selected main 0bfd95e8561699b3bb7c619b0b0d2378d84395bc; this occurs before the changed Journaling work-loop or Durable Messaging execution path.
The allocator gives up after 100 candidate attempts. Candidate rejection can involve listener observations, socket-bind failures, or port-reservation mutex acquisition. The current exception does not identify which condition exhausted the attempts, so the evidence does not establish actual system-wide port exhaustion or a particular leaked reservation.
Please capture bounded allocator diagnostics identifying rejected candidates/reasons and investigate fixture/reservation lifetimes under CI concurrency. Preserve the distinction between this pre-deployment allocation failure and a later silo-listener bind conflict.
Existing reports checked
- TestCluster.Deploy: Address already in use. (In Orleans 3.5.0) #7317 and Intermittent SockerException from TestCluster #6845 report later listener-bind failures during silo deployment, rather than this allocator exception.
- Using system given free ports in tests to remove transient test failures in CI #3969 is an older, closed proposal for system-assigned ports.
- test(runtime): allocate heterogeneous silo ports at startup #11201 fixes delayed additional-silo endpoint selection; its merge commit
71d8279cea903ba653d3fce00729908c23c687e0is already an ancestor of this failing head. - test(testing): cover TestingHost failure diagnostics #10883 adds TestingHost diagnostic coverage and does not supply a fix for this failure.
Exact exception, allocator-name, free-port, and Firestore fixture searches found no matching current issue or uninherited fix. No manual CI rerun, unrelated branch rebase, or production workaround was performed.
Activity