Skip to content

perf: cache compatible placement silos - #10314

Merged
ReubenBond merged 3 commits into
dotnet:mainfrom
ReubenBond:reubenbond/placement-service-caching
Aug 1, 2026
Merged

ReubenBond merged 3 commits into
dotnet:mainfrom
ReubenBond:reubenbond/placement-service-caching

Conversation

@ReubenBond

@ReubenBond ReubenBond commented Jul 31, 2026 •

Copy link
Copy Markdown
Member

PlacementService repeatedly recomputes manifest-compatible silo candidates for placement requests, even though the inputs change only when cluster membership or the cluster manifest changes.

This change caches the unfiltered compatible silo results by grain type, interface type, and interface version. The cache is invalidated when silo status changes or a new cluster manifest is observed. Placement filter results remain uncached because filters can inspect per-request RequestContext data; filtered requests reuse only the cached intermediate candidate set.

The focused tests cover cache reuse, membership and manifest invalidation, per-request filter execution, and lifecycle shutdown behavior.

Microsoft Reviewers: Open in CodeFlow

Copilot AI review requested due to automatic review settings July 31, 2026 22:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves Orleans runtime placement performance by caching the computed set of manifest-compatible (unfiltered) silo candidates for a placement request, avoiding repeated recomputation when inputs are unchanged. The cache is invalidated on silo status changes and when new cluster manifests are observed, while placement filters still run per-request to preserve RequestContext semantics.

Changes:

  • Add a compatible-silo cache in PlacementService, keyed by grain/interface/version and invalidated by membership and cluster manifest updates.
  • Subscribe PlacementService to silo status events and cluster manifest update stream to trigger cache invalidation.
  • Add focused unit tests validating cache reuse, invalidation behavior, per-request filter execution, and shutdown/lifecycle behavior.
Show a summary per file
File Description
test/Orleans.Core.Tests/Runtime/PlacementServiceTests.cs Adds tests for cached compatible-silo computation and invalidation scenarios, plus supporting test fixture infrastructure.
src/Orleans.Runtime/Placement/PlacementService.cs Introduces caching for unfiltered compatible silos, invalidation hooks (membership + manifest updates), and lifecycle wiring for background update processing.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e37e40a0-1a95-42fa-a9e3-3fb92695a286
Copilot AI review requested due to automatic review settings July 31, 2026 22:33
@ReubenBond
ReubenBond force-pushed the reubenbond/placement-service-caching branch from 335fe1f to 148f8c2 Compare July 31, 2026 22:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Suppressed comments (2)

src/Orleans.Runtime/Placement/PlacementService.cs:148

  • StopAsync waits for _manifestUpdatesTask using WaitAsync(cancellationToken) unconditionally. If the lifecycle stop token is already canceled, WaitAsync will cancel immediately and the manifest update loop may continue running after shutdown (since it is controlled by _shutdownCts, not the lifecycle token). Mirror the worker completion logic so the manifest update task is still awaited (suppressing exceptions) even when the lifecycle stop token is canceled.
            if (_manifestUpdatesTask is Task manifestUpdatesTask)
            {
                await manifestUpdatesTask.WaitAsync(cancellationToken).SuppressThrowing();
            }

src/Orleans.Runtime/Placement/PlacementService.cs:246

  • GetUnfilteredCompatibleSilos includes interfaceType in the cache key even when interfaceVersion == 0, but ComputeUnfilteredCompatibleSilos ignores interfaceType in that case. This can create redundant cache entries for the same grain type depending on which interface initiated placement. Consider normalizing interfaceType when interfaceVersion == 0 to reduce cache fragmentation.
        private SiloAddress[] GetUnfilteredCompatibleSilos(GrainType grainType, GrainInterfaceType interfaceType, ushort interfaceVersion)
        {
            var generation = Volatile.Read(ref _placementCacheGeneration);
            var key = new CompatibleSilosCacheKey(grainType, interfaceType, interfaceVersion, generation);
            return _compatibleSilosCache.GetOrAdd(key, key => ComputeUnfilteredCompatibleSilos(key.GrainType, key.InterfaceType, key.InterfaceVersion));
  • Files reviewed: 2/2 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e37e40a0-1a95-42fa-a9e3-3fb92695a286
Copilot AI review requested due to automatic review settings July 31, 2026 22:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

Suppressed comments (2)

src/Orleans.Runtime/Placement/PlacementService.cs:253

  • GetUnfilteredCompatibleSilos can return a stale compatible-silo set if InvalidatePlacementCaches runs concurrently: the method reads _placementCacheGeneration once, and a concurrent invalidation can increment/clear after that read, allowing an old-generation entry to be added after Clear and (a) be returned to the caller and (b) remain in the dictionary (never used again) until the next invalidation. Consider retrying when the generation changes so callers always observe a stable generation and stale entries are removed.
        private SiloAddress[] GetUnfilteredCompatibleSilos(GrainType grainType, GrainInterfaceType interfaceType, ushort interfaceVersion)
        {
            var generation = Volatile.Read(ref _placementCacheGeneration);
            var key = new CompatibleSilosCacheKey(grainType, interfaceType, interfaceVersion, generation);
            return _compatibleSilosCache.GetOrAdd(key, key => ComputeUnfilteredCompatibleSilos(key.GrainType, key.InterfaceType, key.InterfaceVersion));
        }

src/Orleans.Runtime/Placement/PlacementService.cs:153

  • StopAsync waits for _manifestUpdatesTask using WaitAsync(cancellationToken) even when cancellationToken is already canceled. In that case WaitAsync will throw immediately and be suppressed, so StopAsync may return without actually waiting for the manifest-update loop to observe _shutdownCts cancellation and exit. This differs from the worker-task shutdown path, which still awaits completion even with a canceled token.
            if (_manifestUpdatesTask is Task manifestUpdatesTask)
            {
                await manifestUpdatesTask.WaitAsync(cancellationToken).SuppressThrowing();
            }
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: e37e40a0-1a95-42fa-a9e3-3fb92695a286
Copilot AI review requested due to automatic review settings July 31, 2026 23:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants