refactor: provider-name dispatch + fix agent finalization and rescue crash - #1
Merged
Conversation
…tion and rescue crash
A production run of PR-Lens failed with two compounding defects:
1. The ToolLoopAgent exhausted its 12-step budget without calling submit_report.
2. The rescue path then crashed with AI_NoObjectGeneratedError because the
generic @ai-sdk/openai-compatible provider was created without
supportsStructuredOutputs, so `Output.object({ schema })` silently sent the
schema-less response_format and the model returned text that didn't parse.
Rather than paper over (2) with a capability flag and keep the generic BYO
wrapper for every provider, this change pivots to provider-name dispatch:
users pass llm-provider = openai | anthropic | deepseek | google | groq |
mistral | xai | openrouter | openai-compatible, and each branch uses the
respective first-party @ai-sdk/* package which already knows its own
capability shape.
Other fixes in the same pass:
- Add prepareStep to force `submit_report` on the final allowed step, so
stopWhen: hasToolCall('submit_report') fires by construction and the rescue
becomes a true safety net rather than a normal fallback.
- Simplify rescue to a single generateText + Output.object call against
VerifyReportSchema directly. Delete the repairJson / Lenient-schema / alias-
normalisation layers and the @qraftr/json-repair dependency — structured
outputs guarantee the exact Zod shape.
- Keep @ai-sdk/openai-compatible as the explicit escape hatch
(llm-provider: openai-compatible) with supportsStructuredOutputs + includeUsage.
- Bump LanguageModelV2 → LanguageModelV3 everywhere; all provider SDKs at v3.x
implement V3.
- Fix .gitignore so dist/ actually ships (previous ignore line contradicted the
comment right below it and the README's stated intent).
Breaking change: existing workflows must add `llm-provider:` and drop
`llm-base-url:` unless they're using an OpenAI-compatible self-hosted endpoint.
README and self-test workflow updated accordingly.
Smoke-tested end-to-end against sindresorhus/ky#853 via Moonshot kimi-k2.5
(openai-compatible branch): 7 steps, submittedViaTool=true, 15k tokens, no
warnings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous commit mistakenly committed dist/index.js and un-ignored dist/. release.yml's header comment clearly states dist/ is gitignored on main and built+committed onto the tag commit at release time — that's the actual design. CI's `git diff --exit-code dist/` then kept failing on PRs because `bun run build --minify` output is not byte-stable across Bun versions (my local canary produced one bundle, CI's bun@latest produced another). Fixes: - Restore `dist/` to .gitignore; clarify the comment so nobody repeats my mistake. `git rm --cached dist/index.js` to untrack the accidental commit. - Pin CI and Release workflows to `bun-version: 1.3.12` so the bundle that ships at a tag matches what future CI would validate on PRs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Self-test was referencing `uses: ./` without dist/index.js present on the PR branch (dist is gitignored on main — only built at release-tag time). Add a bun install + bun run build step so the action bundle exists when the action step runs. Uses the same pinned Bun version as ci.yml and release.yml. This bug was latent in the initial commit but hadn't been hit before because there were no PRs against the repo yet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
llm-provider(one ofopenai | anthropic | deepseek | google | groq | mistral | xai | openrouter | openai-compatible) and uses the matching@ai-sdk/*SDK. Each provider's own package handles structured outputs, tool calling, and auth correctly — no capability shims.prepareStepcallback that restricts the final step tosubmit_reportonly and setstoolChoice: { type: 'tool', toolName: 'submit_report' }. The agent now finalizes reliably within budget.@qraftr/json-repairfirst pass, theLenientschema aliases, andnormaliseAliases. Rescue is now oneOutput.object({ schema: VerifyReportSchema })call — the provider's native structured-output path guarantees the exact Zod shape.@ai-sdk/openai-compatibleas the explicit escape hatch for self-hosted vLLM / Moonshot / Qwen, now withsupportsStructuredOutputs: trueset so it actually works..gitignoresodist/index.jsships with the commit (the olddist/ignore line contradicted the README and the comment right next to it).Motivation
A prod run of
Iris-Ares/pr-lens-github-actions@v0failed with:Two compounding defects: the ToolLoopAgent never called
submit_reportwithin the 12-step budget, and the rescue crashed because the generic BYO provider was built withoutsupportsStructuredOutputs— soOutput.object({ schema })silently sent a schema-less request.Breaking change
Existing workflows must update:
For self-hosted or non-first-party endpoints (Moonshot, local vLLM, Ollama):
README has per-provider examples and a model-id table.
Test plan
bun run typecheck— cleanbun test— 47/47 passbun run build— 376 modules,dist/index.js= 1.68 MBloadConfigreaches the expected required-input errorsindresorhus/ky#853via Moonshotkimi-k2.5(openai-compatible branch):responseFormatwarningsubmittedViaTool: true(well under the 12-step budget)LLM_PROVIDERvar (needs repo secret/var update before merging)prepareStepforce-finalization by running withSTEP_BUDGET=2(too small to finish) and confirming the single-call rescue completes cleanlyFiles touched
action.yml,src/config.ts— newllm-providerinput + conditional base-URL validationsrc/ai/provider.ts— provider dispatcher (9 branches)src/ai/agent.ts—prepareStep+ simplified rescue + V2→V3 type bumpsrc/run.ts— passllmProviderintobuildModelpackage.json,bun.lock— add 8 new provider packages, drop@qraftr/json-repairREADME.md,.env.example,scripts/local-run.ts,.github/workflows/self-test.yml— docs + local tooling for new input shape.gitignore— un-ignoredist/dist/index.js— rebuilt bundle🤖 Generated with Claude Code