Repository navigation
Description
Tracks things the review-pull-request skill and Copilot Code Review (CCR) get wrong or miss, so we can see what to improve.
Workflow & How to Help
- Run the review skill: You can run the
review-pull-requestskill locally in Copilot CLI / VS Code, or trigger the hosted agent on any PR by commenting/review.@PureWeenis also running the skill locally across active PRs (applyingagent-reviewed-approved/agent-reviewed-needs-changeslabels) to compare against hosted/reviewand CCR, validate findings, and iterate on improvements. - Report a bad finding or miss: If a review produces a false positive, wrong suggestion, or misses a real bug, let
@PureWeenknow, drop a comment on this issue, or ask Copilot to file a new sub-issue with#70231as the parent epic. - Or ask Copilot to diagnose and fix it directly using a prompt like:
Evaluate why
review-pull-requestproduced a bad finding (or missed an issue) on PR#<PR_NUMBER>(<COMMENT_LINK_OR_SUMMARY>). Trace whether the gap came from routed area guidance (docs/*.md/AGENTS.md), coordinator candidate packaging (.github/skills/review-pull-request/SKILL.md), or post-gate verification (.github/agents/review-candidate-verifier.agent.md), and update the relevant instructions, skill, oreng/skill-evals/review-pull-request/evals with a general fix that doesn't overfit to this single PR.
Tracked Failure Modes
- [Review agent] Distinguish primary errors from newly suppressed secondary failures (#69445) #70269 (Fixed by Improve reviewer handling of secondary failure suppression #70312) — Missed policy assessment / deduplicated feedback: conflated preservation of the primary processing error with justification for suppressing a secondary logging-provider failure (
#69445) - [Review agent] False positive: MessagePack count validation ignored on #70143 #70232 (Fixed by Refine review-candidate-verifier rules for external triggers, test gaps, and secondary-failure catches #70319) — False positive: external library behavior (
MessagePackReader.ReadArrayHeadercount validation) assumed without external package source in bundle (#70143) - [Review agent] Incorrect instrument-duplication claim and ineffective test fix on #70174 #70233 (Fixed by Refine review-candidate-verifier rules for external triggers, test gaps, and secondary-failure catches #70319) — False positive / wrong remedy: incorrect instrument-duplication claim and ineffective test fix (
#70174) - [Review agent] False positive: unsupported-concurrency race in AuthorizationMessageHandler on #70106 #70235 (Fixed by Add candidate verifier agent with tiered intent handling to review-pull-request #70250) — False positive: unsupported-concurrency race in
AuthorizationMessageHandleron single-threaded WebAssembly (#70106) - [Review agent] Pass PR description and linked issue context to review-candidate-verifier to avoid false intent-downgrade questions #70335 — False intent-downgrade question: PR description and linked issue context missing from
review-candidate-verifierinput (#70304, fixing#66322;#70534, fixing#19638) - [Review agent] Require exact test-endpoint reachability and non-copied fixture delta before keeping test-asset findings #70336 — False positive on an unexercised test endpoint and copied fixture code: the executed MSAL test never reaches
/subdir/oidc/token;RemoteMsalAuthenticationStartup.cs:77is a verbatim copy ofRemoteAuthenticationStartup.cs:82onmain(#69721) - [Review agent] Check cross-render-mode event-ordering differences when synchronous WebAssembly handlers call NavigationManager.NavigateTo on links #70337 — Cross-render-mode event ordering: check synchronous WebAssembly versus asynchronous Server navigation handlers when default anchor activation follows
EventDelegator(#70299) - [Review agent] Flag unrelated submodule gitlink pointer changes instead of downgrading them to UNRESOLVED (#70340) #70359 — Missed unrelated submodule pointer revert:
src/submodules/googletestgitlink change downgraded toUNRESOLVEDfor unmaterialized submodule tree bytes instead of flagged as an unrelated pointer change (#70340) - [Review agent] Calibrate Blazor DI scope teardown IJSObjectReference disposal findings (#69970, #70340) #70360 — Unestablished production reachability /
P2severity calibration onIJSObjectReferencedisposal at Blazor DI scope teardown (#69970,#70340) - [Review agent] Enforce strict 11-key JSON output from review-candidate-verifier with bounded normalization/retry (#70299, #70340) #70361 —
review-candidate-verifierstrict 11-key JSON schema compliance: extra"rationale"synonym key, markdown proseKEEPoutput, and innerlinksshape/literal-"n/a"drift (#70299,#70340,#70534,#70343) - [review-pull-request] Stop publishing contradicted [DISCARDED] candidates as inline GitHub PR review comments #70546 — Stop publishing contradicted
[DISCARDED]candidates as inline GitHub PR review comments (#70350,#70450,#70315) - [review-pull-request] Calibrate review-candidate-verifier external/unestablished handling and exclude unestablished DROPs from inline publication #70547 — Calibrate
review-candidate-verifierexternal/unestablished handling and exclude unestablishedDROPs from inline publication (#70504) - [review-pull-request] Pass sibling-worker refutations to review-candidate-verifier and validate remedies against in-repo tests (#70497) #70548 — Pass sibling-worker refutations to
review-candidate-verifierand validate remedies against in-repo tests (#70497,#69767) - [Review Agent Failure] Intentional test/sample opt-outs and secondary-failure exception suppression certified as defects #70550 — Intentional test/sample opt-outs and secondary-failure exception suppression certified as defects (
#70045,#69767,#67716) - [Review Agent Failure] Missing review-thread resolution metadata in feedback.json leads to re-litigating resolved threads #70551 — Missing review-thread resolution metadata in
feedback.jsonleads to re-litigating resolved threads (#69478,#69767) - [Review Agent Failure] Routed guides omit package-specific skills (e.g. port-components-platform-api) and verifier returns intent=n/a when PR body cites a repository skill (#70562) #70565 — Routed guides omit package-specific skills (e.g.
port-components-platform-api) andreview-candidate-verifierreturnsintent=n/awhenpull.jsoncites a repository skill (#70562) - [review-pull-request] CCR overview cards deduplicate distinct anchored worker candidates without verification (#70532) #70674 — CCR overview cards (
<!-- ccr-overview-v2 -->) deduplicate distinct anchored worker candidates intoEXISTING_FEEDBACK_COVERAGEwithout verification (#70532)
PRs Created to Improve Skills, Instructions & Evals
- Enable adaptive validation in hosted PR reviews #70087 (Merged) — Adaptive validation and stricter coordinator/worker evidence gates in
review-pull-request - Add candidate verifier agent with tiered intent handling to review-pull-request #70250 (Merged) — Post-gate
review-candidate-verifiersubagent with proof-chain checks, confidence tiers (verified,question,unresolved,discarded), and inline/summary preservation of disputed or unestablished candidates (fixes [Review agent] False positive: unsupported-concurrency race in AuthorizationMessageHandler on #70106 #70235) - Project Components permanent E2E boundary and preserve verifier candidate claims #70271 (Merged) — Coordinator faithful candidate packaging (
#70073) and Components permanent E2E regression-test boundary bundling (#70112) - Improve reviewer handling of secondary failure suppression #70312 (Merged) — Secondary failure suppression, existing-feedback deduplication handling, and
catch-policy.vally.yamlsource-first evals (#70269/#69445) - Add review-candidate-verifier Vally suite and evaluation harness #70318 (Merged) — Compact standard
review-pull-requestVally evaluation suite (eng/skill-evals/review-pull-request/eval.vally.yaml, ~18 KB fixtures) andstage_run.ps1/skill-evals.ymlstaging - Refine review-candidate-verifier rules for external triggers, test gaps, and secondary-failure catches #70319 (Merged) —
review-candidate-verifierrules for unbundled external triggers ([Review agent] False positive: MessagePack count validation ignored on #70143 #70232), test-gap remedy verification ([Review agent] Incorrect instrument-duplication claim and ineffective test fix on #70174 #70233), pre-existing collaborator throws (#70043), and secondary-failure catches ([Review agent] Distinguish primary errors from newly suppressed secondary failures (#69445) #70269) - Fence frozen-bundle review skill off from CCR and adapt domain skills for source-only PR review #70581 (Merged) — Fence frozen-bundle
review-pull-requestskill off from CCR and adapt domain skills for source-only PR review - Fix skill-eval result gate ownership and diagnostics #70626 (Merged) — Fix
skill-evalresult gate ownership and diagnostics - Keep discarded review candidates out of inline findings #70628 (Open) — Keep discarded review candidates out of inline findings ([review-pull-request] Stop publishing contradicted [DISCARDED] candidates as inline GitHub PR review comments #70546)
- Add review-pull-request reviewer regression Vally suite #70238 (Superseded by Add review-candidate-verifier Vally suite and evaluation harness #70318) (supersedes closed draft Add reviewer concurrency capability eval #70205) — Full-bundle Vally regression evaluation suite draft (
eng/skill-evals/review-pull-request/)
Activity
github-actions commentedon Nov 1, 2026
Triage Summary
Area: none assigned (the title and empty body do not identify an area with enough confidence)
Type:
Epic(preserved)Potential Duplicates