Repository navigation
Start ActivationRebalancerMonitor without eagerly fetching a report - #9228
Merged
Merged
Conversation
ReubenBond
force-pushed
the
fix/rebalancer-startup
branch
from
February 5, 2025 17:56
5287976 to
d9f782b
Compare
Member
Author
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
This breaks some tests which expect this to run during startup. |
ReubenBond
marked this pull request as draft
March 20, 2025 22:36
ReubenBond
force-pushed
the
fix/rebalancer-startup
branch
from
November 17, 2025 23:54
d9f782b to
caa7887
Compare
ReubenBond
force-pushed
the
fix/rebalancer-startup
branch
from
February 15, 2026 21:50
caa7887 to
8ea8425
Compare
ReubenBond
marked this pull request as ready for review
February 15, 2026 21:50
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Orleans runtime’s ActivationRebalancerMonitor startup behavior to avoid an eager GetReport() call during silo host start, reducing the number of potential failure points on startup.
Changes:
- Stop fetching the activation rebalancer report eagerly during
OnStart. - Start the monitor timer without performing an initial
GetReport()call.
Comments suppressed due to low confidence (1)
src/Orleans.Runtime/Placement/Rebalancing/ActivationRebalancerMonitor.cs:76
- Removing the eager GetReport() call means
_latestReportcan remain at its default value (Host =SiloAddress.Zero) until the worker reports.OnStopuses_latestReport.Hostto decide whether to migrate the rebalancer, so a silo which is actually hosting the worker could skip migration if it shuts down before receiving a report. Consider basing the migration decision solely on_activationDirectory.FindTarget(_rebalancerGrain.GetGrainId())(since this directory is local to the silo) instead of relying on_latestReport.Hostbeing up-to-date.
await this.RunOrQueueTask(() =>
{
_monitorTimer = RegisterGrainTimer(async ct =>
{
var elapsedSinceHeartbeat = _timeProvider.GetElapsedTime(_lastHeartbeatTimestamp);
Start the monitor timer immediately and fetch an initial report only when the cached host is unknown, preserving lazy startup while avoiding stale default state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Log whenever ActivationRebalancerMonitor fetches a report, including the initial fetch when host is unknown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ReubenBond
enabled auto-merge
February 15, 2026 22:21
rkargMsft
pushed a commit
to rkargMsft/orleans
that referenced
this pull request
Feb 27, 2026
…otnet#9228) * Start ActivationRebalancerMonitor without eagerly fetching a report * Fix monitor startup report initialization Start the monitor timer immediately and fetch an initial report only when the cached host is unknown, preserving lazy startup while avoiding stale default state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Always log when monitor fetches report Log whenever ActivationRebalancerMonitor fetches a report, including the initial fetch when host is unknown. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eager fetching of a report when starting the host adds an extra point of failure.
Microsoft Reviewers: Open in CodeFlow