Repository navigation
perf: cache compatible placement silos - #10314
Merged
ReubenBond merged 3 commits intoAug 1, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves Orleans runtime placement performance by caching the computed set of manifest-compatible (unfiltered) silo candidates for a placement request, avoiding repeated recomputation when inputs are unchanged. The cache is invalidated on silo status changes and when new cluster manifests are observed, while placement filters still run per-request to preserve RequestContext semantics.
Changes:
- Add a compatible-silo cache in
PlacementService, keyed by grain/interface/version and invalidated by membership and cluster manifest updates. - Subscribe
PlacementServiceto silo status events and cluster manifest update stream to trigger cache invalidation. - Add focused unit tests validating cache reuse, invalidation behavior, per-request filter execution, and shutdown/lifecycle behavior.
Show a summary per file
| File | Description |
|---|---|
| test/Orleans.Core.Tests/Runtime/PlacementServiceTests.cs | Adds tests for cached compatible-silo computation and invalidation scenarios, plus supporting test fixture infrastructure. |
| src/Orleans.Runtime/Placement/PlacementService.cs | Introduces caching for unfiltered compatible silos, invalidation hooks (membership + manifest updates), and lifecycle wiring for background update processing. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 1
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e37e40a0-1a95-42fa-a9e3-3fb92695a286
ReubenBond
force-pushed
the
reubenbond/placement-service-caching
branch
from
July 31, 2026 22:33
335fe1f to
148f8c2
Compare
Contributor
There was a problem hiding this comment.
Copilot's findings
Suppressed comments (2)
src/Orleans.Runtime/Placement/PlacementService.cs:148
- StopAsync waits for _manifestUpdatesTask using WaitAsync(cancellationToken) unconditionally. If the lifecycle stop token is already canceled, WaitAsync will cancel immediately and the manifest update loop may continue running after shutdown (since it is controlled by _shutdownCts, not the lifecycle token). Mirror the worker completion logic so the manifest update task is still awaited (suppressing exceptions) even when the lifecycle stop token is canceled.
if (_manifestUpdatesTask is Task manifestUpdatesTask)
{
await manifestUpdatesTask.WaitAsync(cancellationToken).SuppressThrowing();
}
src/Orleans.Runtime/Placement/PlacementService.cs:246
- GetUnfilteredCompatibleSilos includes interfaceType in the cache key even when interfaceVersion == 0, but ComputeUnfilteredCompatibleSilos ignores interfaceType in that case. This can create redundant cache entries for the same grain type depending on which interface initiated placement. Consider normalizing interfaceType when interfaceVersion == 0 to reduce cache fragmentation.
private SiloAddress[] GetUnfilteredCompatibleSilos(GrainType grainType, GrainInterfaceType interfaceType, ushort interfaceVersion)
{
var generation = Volatile.Read(ref _placementCacheGeneration);
var key = new CompatibleSilosCacheKey(grainType, interfaceType, interfaceVersion, generation);
return _compatibleSilosCache.GetOrAdd(key, key => ComputeUnfilteredCompatibleSilos(key.GrainType, key.InterfaceType, key.InterfaceVersion));
- Files reviewed: 2/2 changed files
- Comments generated: 0 new
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e37e40a0-1a95-42fa-a9e3-3fb92695a286
Contributor
There was a problem hiding this comment.
Copilot's findings
Suppressed comments (2)
src/Orleans.Runtime/Placement/PlacementService.cs:253
- GetUnfilteredCompatibleSilos can return a stale compatible-silo set if InvalidatePlacementCaches runs concurrently: the method reads _placementCacheGeneration once, and a concurrent invalidation can increment/clear after that read, allowing an old-generation entry to be added after Clear and (a) be returned to the caller and (b) remain in the dictionary (never used again) until the next invalidation. Consider retrying when the generation changes so callers always observe a stable generation and stale entries are removed.
private SiloAddress[] GetUnfilteredCompatibleSilos(GrainType grainType, GrainInterfaceType interfaceType, ushort interfaceVersion)
{
var generation = Volatile.Read(ref _placementCacheGeneration);
var key = new CompatibleSilosCacheKey(grainType, interfaceType, interfaceVersion, generation);
return _compatibleSilosCache.GetOrAdd(key, key => ComputeUnfilteredCompatibleSilos(key.GrainType, key.InterfaceType, key.InterfaceVersion));
}
src/Orleans.Runtime/Placement/PlacementService.cs:153
- StopAsync waits for _manifestUpdatesTask using WaitAsync(cancellationToken) even when cancellationToken is already canceled. In that case WaitAsync will throw immediately and be suppressed, so StopAsync may return without actually waiting for the manifest-update loop to observe _shutdownCts cancellation and exit. This differs from the worker-task shutdown path, which still awaits completion even with a canceled token.
if (_manifestUpdatesTask is Task manifestUpdatesTask)
{
await manifestUpdatesTask.WaitAsync(cancellationToken).SuppressThrowing();
}
- Files reviewed: 3/3 changed files
- Comments generated: 0 new
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e37e40a0-1a95-42fa-a9e3-3fb92695a286
This was referenced Aug 28, 2026
Merged
This was referenced Aug 31, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PlacementServicerepeatedly recomputes manifest-compatible silo candidates for placement requests, even though the inputs change only when cluster membership or the cluster manifest changes.This change caches the unfiltered compatible silo results by grain type, interface type, and interface version. The cache is invalidated when silo status changes or a new cluster manifest is observed. Placement filter results remain uncached because filters can inspect per-request
RequestContextdata; filtered requests reuse only the cached intermediate candidate set.The focused tests cover cache reuse, membership and manifest invalidation, per-request filter execution, and lifecycle shutdown behavior.
Microsoft Reviewers: Open in CodeFlow