feat(opevents): attempt.success and attempt.failed operator events#989
Open
alexluong wants to merge 5 commits into
Open
feat(opevents): attempt.success and attempt.failed operator events#989alexluong wants to merge 5 commits into
alexluong wants to merge 5 commits into
Conversation
One shared payload shape (tenant, event, attempt, destination projection), mirroring alert.attempt.exhausted_retries; the topic split exists for subscription filtering. These fire once per delivery attempt, so the docs call out the volume implication — including for "*" subscribers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The failed event joins plan()'s owed-events list, riding the existing errgroup send + replay gate + ack/nack semantics. The success path sends its event ungated before the ack — gating would cost a Redis key per successful attempt (the dominant traffic) to dedup a rare redelivery re-emit, and opevents are at-least-once anyway. Every failed attempt's expected topic multiset in the characterization suite gains one attempt.failed (and successes one attempt.success); assertions that pinned arrival order now filter by topic. The sink gains a composite attemptID/topic fail key so the exhausted-window test can fail only the exhausted send — failing the whole attempt would cancel the suppression Exec mid-flight instead of exercising clear-on-failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Surfaced by the exhausted-window suppression test: a canceled ctx fails both the exec and the cleanup Del, leaving the key claimed until its TTL. Deliberately unfixed — bounded, self-healing, and rare. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
logmq's audit line inferred "delivered" from Emit returning nil, which is also what a filtered topic or the noop emitter returns — with per-attempt events that becomes a false audit line per attempt on deployments that never subscribed. The emitter knows the difference, so the line moves to where the knowledge lives: it fires iff an event was actually sent, and gains the generated opevent id (previously unloggable — minted inside Emit). Event.LogFields carries caller context; the payload constructors populate attempt_id/event_id/destination so the line keeps the fields the logmq version had, and emit call sites stay unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With every alert signal off, the failed path's replay gate protects nothing — Evaluate is a stateless no-op and attempt.failed is at-least-once like attempt.success — yet it still cost two Redis round trips per failed attempt. Skip it: emit attempt.failed ungated and ack. When no attempt topic is subscribed either, the pipeline can't produce anything, so entries ack straight after persistence with no goroutine spawned. Both bools derive from existing config at construction — Evaluator gains SignalsEnabled(), Emitter gains Enabled(topic) — so logmq never re-implements signal or topic-filter semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements #981
Adds two operator events, plus config-based optimizations along the way.
New topics:
attempt.successandattempt.failed— one event per delivery attempt, emitted from logmq after the attempt is persisted. The payload matches theexhausted_retriesshape. Note: existing*subscribers start receiving every attempt event on upgrade — worth a release-note callout.Skip unnecessary work when config doesn't need it — with alerting disabled, failed attempts skip the alert logic and its Redis calls; with attempt events also unsubscribed, entries just persist and ack.