You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(runtime): add RunRequestAsync and IGrainContext.ReceiveLocalRequest - #11505
Grains often do background work, such as stream handlers, messaging pumps or timer-adjacent callbacks, and then need to apply the result to grain state. To do that safely, a section of code has to run under the grain's scheduling rules: reentrancy, call filters and activation lifetime. Today there is no direct way to queue that work as a request on the activation.
Solution
IGrainContext.ReceiveLocalRequest(ILocalRequest request, LocalRequestOptions options) is the general primitive. It queues a request as a local-only, one-way request on the activation.
ILocalRequest is an IInvokable which receives its own outcome through void Complete(Response response): the response after the request and its call filters complete, or OrleansMessageRejectionException if the activation deactivates first. The request owns its outcome, including reporting or logging failures and keeping the first outcome it receives.
The request follows the grain's reentrancy rules, passes through grain call filters and [MayInterleave] predicates (which observe the request itself) and starts a new call chain, as timer ticks do. Like timer ticks, local requests are admitted regardless of MaxEnqueuedRequestsHardLimit.
LocalRequestOptions is a small readonly struct:
KeepAlive (default true) restarts the idle collection period.
Interleave lets the request interleave like an [AlwaysInterleave] call.
ActivationData implements the primitive and SystemTarget implements it explicitly. Other contexts keep the default implementation, which throws NotSupportedException. Rejected requests are completed after the activation lock is released, so Complete can safely queue further requests.
GrainBaseExtensions.RunRequestAsync overloads (synchronous or asynchronous, with and without state, with and without a result, with and without LocalRequestOptions) are the ergonomic API built on the primitive.
Filters observe the request as IGrainRequestInvoker.RunRequestAsync with no arguments. A custom ILocalRequest supplies its own interface name, method name and arguments.
The cancellation token cancels the request while it's queued. A filter that fails or sets the result without invoking the callback completes the returned task with that outcome.
Grain timers deliver ticks through ReceiveLocalRequest with their reusable per-timer invoker. As a result, TimerRegistry no longer depends on MessageFactory.
Rationale
A single primitive on IGrainContext means the grain context is the runner. It works from any thread, with no DI lookups or per-activation fields. Making the request its own completion keeps the message body the request itself, so invocation, call filters and scheduling predicates need no special handling, and custom requests cover anything beyond the default description.
The allocation cost is kept low:
RunRequestAsync allocates one GrainRequest, which is its own invokable, TaskCompletionSource and completion, plus one Message. Its description is constant.
Timer ticks allocate only their message.
ActivationData gets no new fields.
The public API surface in src/api gains RunRequestAsync, ILocalRequest, LocalRequestOptions and IGrainContext.ReceiveLocalRequest. The request scheduling docs gain compiled examples for running callbacks from background threads, configuring keep-alive and interleaving, and delivering custom requests. The timers and external tasks pages link to it.
Activation keep-alive holds were split out into #11506. This PR is the base of the durable Messaging stack (#11282 → #11284 → #11285 → #10693).
The current-main baseline is commit 6341f1a3ee and uses the same reviewed coverage matrix.
Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.
The comparison remains report-only while normal line and branch variance is calibrated.
Preserve warning when cancellation masks filter failure
src/Orleans.Runtime/Timers/GrainRequest.cs:109
When a filter fails without invoking the callback, cancellation can complete the task after the IsCompleted check, including while _registration.Dispose() waits for the cancellation callback. TrySetException then returns false, but OnInvoked still returns true. InsideRuntimeClient consequently logs the filter failure at Debug even though the caller only observes cancellation. Return the result of TrySetException so this unreported failure remains at Warning.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Scheduling, cancellation, and activation-lifecycle interactions warrant final human review despite focused regression coverage and no confirmed blocking findings.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Cross-cutting scheduling, cancellation, completion ownership, and timer-lifecycle changes warrant final human review despite no confirmed blocking findings.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Public scheduling contracts and interacting cancellation, completion, and activation-lifecycle changes warrant final human review despite substantial regression coverage.
Both callback-success paths publish the original TResult to the returned task and return Response.Completed, so the runtime's response copier never sees the result. A background caller receiving a grain-owned mutable collection can therefore modify grain state off-scheduler, bypassing the response isolation intended for local requests. Copy the result on the activation before publishing it, preserving the callback-first outcome semantics. Cover both synchronously and asynchronously completed callbacks with a mutable-result regression test.
Filter results bypass response copying and expose mutable grain state
A filter that short-circuits with context.Result, or replaces a custom request's result after invocation, bypasses the response copy in GrainMethodInvoker.Invoke. Local requests also skip SafeSendResponse, which copies the final response for ordinary calls. Consequently, Complete receives the filter's original mutable object, potentially exposing grain state to an off-activation caller. Copy the final local response before completion and add a regression test with a mutable filter-produced result.
Both success paths (Invoke and AwaitCallbackAsync) publish the original callback result to the returned task, then return Response.Completed. The runtime's response copiers therefore never see the actual result. If a background caller receives a mutable grain-owned list, it can modify that same list outside the grain scheduler, bypassing the intended response-copy isolation. Copy the callback result before publishing it on both paths, while preserving the documented callback-first outcome semantics. Add regression tests with a mutable result for synchronous and asynchronous completion.
A filter which short-circuits by setting context.Result, or replaces the result after invocation, bypasses the copier inside GrainMethodInvoker. Ordinary requests copy the final response in SafeSendResponse, but local requests pass it directly to Complete. A filter returning mutable grain state therefore shares it with an external observer, including RunRequestAsync callers when the callback was skipped. Copy the final local response before completion and add a mutable filter-result regression test.
On the two "previously missed" findings ("Copy callback results before publishing to prevent mutable state leaks" and "Copy filtered local responses before completion"): we're leaving both as they are, by design.
The runtime passes a local request by reference, without copying it, and passes its response the same way. The response copier no longer runs for local requests, with or without call filters. RunRequestAsync hands its callback's state and result to the caller by reference, and a custom ILocalRequest decides what it hands to its own observers. A developer who needs isolation can copy the inputs and outputs.
Add [TestSuite("BVT")] and [TestProvider("None")] so these regression tests are selected by CI. The filter in .github/workflows/ci.yml:808 requires both Provider=None and a matching Suite, but TestCategory emits only Category (test/TestInfrastructure/TestExtensions/TestCategory.cs:27–28). Without these attributes, the new cancellation-race and submission-cleanup tests are skipped. The neighboring GrainTimerTests.cs:15–16 uses both attributes.
🧠 Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Grains often do background work, such as stream handlers, messaging pumps or timer-adjacent callbacks, and then need to apply the result to grain state. To do that safely, a section of code has to run under the grain's scheduling rules: reentrancy, call filters and activation lifetime. Today there is no direct way to queue that work as a request on the activation.
Solution
IGrainContext.ReceiveLocalRequest(ILocalRequest request, LocalRequestOptions options)is the general primitive. It queues a request as a local-only, one-way request on the activation.ILocalRequestis anIInvokablewhich receives its own outcome throughvoid Complete(Response response): the response after the request and its call filters complete, orOrleansMessageRejectionExceptionif the activation deactivates first. The request owns its outcome, including reporting or logging failures and keeping the first outcome it receives.[MayInterleave]predicates (which observe the request itself) and starts a new call chain, as timer ticks do. Like timer ticks, local requests are admitted regardless ofMaxEnqueuedRequestsHardLimit.LocalRequestOptionsis a small readonly struct:KeepAlive(defaulttrue) restarts the idle collection period.Interleavelets the request interleave like an[AlwaysInterleave]call.ActivationDataimplements the primitive andSystemTargetimplements it explicitly. Other contexts keep the default implementation, which throwsNotSupportedException. Rejected requests are completed after the activation lock is released, soCompletecan safely queue further requests.GrainBaseExtensions.RunRequestAsyncoverloads (synchronous or asynchronous, with and without state, with and without a result, with and withoutLocalRequestOptions) are the ergonomic API built on the primitive.IGrainRequestInvoker.RunRequestAsyncwith no arguments. A customILocalRequestsupplies its own interface name, method name and arguments.Grain timers deliver ticks through
ReceiveLocalRequestwith their reusable per-timer invoker. As a result,TimerRegistryno longer depends onMessageFactory.Rationale
A single primitive on
IGrainContextmeans the grain context is the runner. It works from any thread, with no DI lookups or per-activation fields. Making the request its own completion keeps the message body the request itself, so invocation, call filters and scheduling predicates need no special handling, and custom requests cover anything beyond the default description.The allocation cost is kept low:
RunRequestAsyncallocates oneGrainRequest, which is its own invokable,TaskCompletionSourceand completion, plus oneMessage. Its description is constant.ActivationDatagets no new fields.The public API surface in
src/apigainsRunRequestAsync,ILocalRequest,LocalRequestOptionsandIGrainContext.ReceiveLocalRequest. The request scheduling docs gain compiled examples for running callbacks from background threads, configuring keep-alive and interleaving, and delivering custom requests. The timers and external tasks pages link to it.Activation keep-alive holds were split out into #11506. This PR is the base of the durable Messaging stack (#11282 → #11284 → #11285 → #10693).
Microsoft Reviewers: Open in CodeFlow