Skip to content

test(testing): cluster port allocation fails during Firestore fixture initialization #11345

Closed

Description

@ReubenBond

Observed failure

The Google Cloud providers net8.0 CI job failed while FirestorePersistenceGrainTests.Fixture was building its test cluster. All 11 failures in that class share the same fixture-initialization exception; their test bodies did not reach the provider operations.

Class fixture type 'Orleans.Persistence.Firestore.Tests.FirestorePersistenceGrainTests+Fixture'
threw in InitializeAsync
---- System.InvalidOperationException: Cannot find enough free ports to spin up a cluster

TestClusterPortAllocator.GetAvailableConsecutiveServerPorts(...): line 158
TestClusterPortAllocator.AllocateConsecutivePortPairs(...): line 35
TestClusterBuilder.ConfigureDefaultPorts(): line 225
TestClusterBuilder.Build(): line 166
BaseTestClusterFixture.InitializeAsync(): line 89

The affected cases include ClearStateAsync_Before_WriteStateAsync, Grain_GrainStorage_Read, the key-type read/write cases, and Grain_GrainStorage_SiloRestart. The downloaded TRX confirms that every failed row has this same fixture exception. Java and Google emulator setup succeeded; the Test step failed. Evidence comes from the completed job's direct log endpoint and its actual test artifact, not an inferred failure from the check name.

Scope and investigation

The failing path is port allocation before cluster deployment. Orleans.TestingHost and the shared fixture infrastructure are unchanged from selected main 0bfd95e8561699b3bb7c619b0b0d2378d84395bc; this occurs before the changed Journaling work-loop or Durable Messaging execution path.

The allocator gives up after 100 candidate attempts. Candidate rejection can involve listener observations, socket-bind failures, or port-reservation mutex acquisition. The current exception does not identify which condition exhausted the attempts, so the evidence does not establish actual system-wide port exhaustion or a particular leaked reservation.

Please capture bounded allocator diagnostics identifying rejected candidates/reasons and investigate fixture/reservation lifetimes under CI concurrency. Preserve the distinction between this pre-deployment allocation failure and a later silo-listener bind conflict.

Existing reports checked

Exact exception, allocator-name, free-port, and Firestore fixture searches found no matching current issue or uninherited fix. No manual CI rerun, unrelated branch rebase, or production workaround was performed.

Activity

  1. locked and limited conversation to collaborators on Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions