Skip to content

Commit 5dcf049

Browse files
Iris-Aresclaude
Iris-Ares
and
claude
authored
refactor: provider-name dispatch + fix agent finalization and rescue crash (#1)
* refactor: switch to first-party provider dispatch; fix agent finalization and rescue crash A production run of PR-Lens failed with two compounding defects: 1. The ToolLoopAgent exhausted its 12-step budget without calling submit_report. 2. The rescue path then crashed with AI_NoObjectGeneratedError because the generic @ai-sdk/openai-compatible provider was created without supportsStructuredOutputs, so `Output.object({ schema })` silently sent the schema-less response_format and the model returned text that didn't parse. Rather than paper over (2) with a capability flag and keep the generic BYO wrapper for every provider, this change pivots to provider-name dispatch: users pass llm-provider = openai | anthropic | deepseek | google | groq | mistral | xai | openrouter | openai-compatible, and each branch uses the respective first-party @ai-sdk/* package which already knows its own capability shape. Other fixes in the same pass: - Add prepareStep to force `submit_report` on the final allowed step, so stopWhen: hasToolCall('submit_report') fires by construction and the rescue becomes a true safety net rather than a normal fallback. - Simplify rescue to a single generateText + Output.object call against VerifyReportSchema directly. Delete the repairJson / Lenient-schema / alias- normalisation layers and the @qraftr/json-repair dependency — structured outputs guarantee the exact Zod shape. - Keep @ai-sdk/openai-compatible as the explicit escape hatch (llm-provider: openai-compatible) with supportsStructuredOutputs + includeUsage. - Bump LanguageModelV2 → LanguageModelV3 everywhere; all provider SDKs at v3.x implement V3. - Fix .gitignore so dist/ actually ships (previous ignore line contradicted the comment right below it and the README's stated intent). Breaking change: existing workflows must add `llm-provider:` and drop `llm-base-url:` unless they're using an OpenAI-compatible self-hosted endpoint. README and self-test workflow updated accordingly. Smoke-tested end-to-end against sindresorhus/ky#853 via Moonshot kimi-k2.5 (openai-compatible branch): 7 steps, submittedViaTool=true, 15k tokens, no warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: keep dist/ gitignored on main and pin Bun version Previous commit mistakenly committed dist/index.js and un-ignored dist/. release.yml's header comment clearly states dist/ is gitignored on main and built+committed onto the tag commit at release time — that's the actual design. CI's `git diff --exit-code dist/` then kept failing on PRs because `bun run build --minify` output is not byte-stable across Bun versions (my local canary produced one bundle, CI's bun@latest produced another). Fixes: - Restore `dist/` to .gitignore; clarify the comment so nobody repeats my mistake. `git rm --cached dist/index.js` to untrack the accidental commit. - Pin CI and Release workflows to `bun-version: 1.3.12` so the bundle that ships at a tag matches what future CI would validate on PRs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: build dist/ in the self-test job before dogfooding the action Self-test was referencing `uses: ./` without dist/index.js present on the PR branch (dist is gitignored on main — only built at release-tag time). Add a bun install + bun run build step so the action bundle exists when the action step runs. Uses the same pinned Bun version as ci.yml and release.yml. This bug was latent in the initial commit but hadn't been hit before because there were no PRs against the repo yet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 4c5bb11 commit 5dcf049

14 files changed

Lines changed: 314 additions & 155 deletions

File tree

‎.env.example‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,14 @@
33
#
44
# .env is gitignored — never commit your real keys.
55

6-
# --- Required: LLM provider (BYO, OpenAI-compatible) ---
7-
LLM_BASE_URL=https://api.openai.com/v1
6+
# --- Required: LLM provider ---
7+
# LLM_PROVIDER is one of: openai, anthropic, deepseek, google, groq, mistral,
8+
# xai, openrouter, openai-compatible.
9+
# LLM_BASE_URL is required ONLY when LLM_PROVIDER=openai-compatible.
10+
LLM_PROVIDER=openai
811
LLM_API_KEY=sk-replace-me
9-
LLM_MODEL=gpt-4o
12+
LLM_MODEL=gpt-4.1
13+
# LLM_BASE_URL=https://vllm.internal/v1 # only with LLM_PROVIDER=openai-compatible
1014

1115
# --- Optional: GitHub auth ---
1216
# Public PRs work without this, but you'll hit the 60 req/h anonymous limit

‎.github/workflows/ci.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,10 @@ jobs:
1111
- uses: actions/checkout@v4
1212
- uses: oven-sh/setup-bun@v2
1313
with:
14-
bun-version: latest
14+
# Pinned: `bun build --minify` output is not byte-stable across Bun
15+
# versions. When bumping this, rebuild dist/ locally with the same
16+
# version and commit the result.
17+
bun-version: 1.3.12
1518
- run: bun install --frozen-lockfile
1619
- run: bun run typecheck
1720
- run: bun test

‎.github/workflows/release.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ jobs:
2222
fetch-depth: 0
2323

2424
- uses: oven-sh/setup-bun@v2
25+
with:
26+
# Keep in lockstep with .github/workflows/ci.yml so the dist shipped
27+
# at a tag is bit-for-bit what CI validates on PRs.
28+
bun-version: 1.3.12
2529

2630
- run: bun install --frozen-lockfile
2731
- run: bun run typecheck

‎.github/workflows/self-test.yml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,18 @@ jobs:
1515
- uses: actions/checkout@v4
1616
with:
1717
fetch-depth: 0
18+
19+
# dist/ is gitignored on main (only built at release-tag time), so to
20+
# dogfood the action on a PR we build the bundle in-place first.
21+
- uses: oven-sh/setup-bun@v2
22+
with:
23+
bun-version: 1.3.12
24+
- run: bun install --frozen-lockfile
25+
- run: bun run build
26+
1827
- name: Run PR-Lens on itself
1928
uses: ./
2029
with:
21-
llm-base-url: ${{ secrets.LLM_BASE_URL }}
30+
llm-provider: ${{ vars.LLM_PROVIDER || 'openai' }}
2231
llm-api-key: ${{ secrets.LLM_API_KEY }}
2332
llm-model: ${{ vars.LLM_MODEL || 'gpt-4o-mini' }}

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,5 +7,7 @@ node_modules/
77
.bun-cache/
88
coverage/
99
evals/results/
10+
# dist/ is gitignored on main; the release workflow builds and commits it on
11+
# top of the tagged commit (see .github/workflows/release.yml). Consumers
12+
# pin @v0.x.y or @v0 — both resolve to tag-commits that include the bundle.
1013
dist/
11-
# dist/ is intentionally committed — Node GitHub Actions run from the tagged dist bundle

‎README.md‎

Lines changed: 69 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -32,40 +32,88 @@ It is intentionally orthogonal to code-review bots. It answers *"what should a h
3232
with: { fetch-depth: 0 }
3333
- uses: Iris-Ares/pr-lens-github-actions@v0
3434
with:
35-
llm-base-url: ${{ secrets.LLM_BASE_URL }}
35+
llm-provider: openai
3636
llm-api-key: ${{ secrets.LLM_API_KEY }}
37-
llm-model: gpt-4o
37+
llm-model: gpt-4.1
3838
```
3939
4040
2. Store your provider credentials as repo secrets:
41-
- `LLM_BASE_URL` — any OpenAI-compatible endpoint (e.g. `https://api.openai.com/v1`, `https://api.deepseek.com/v1`, `https://openrouter.ai/api/v1`, or a local `https://vllm.internal/v1`).
42-
- `LLM_API_KEY` — the API key for that endpoint.
41+
- `LLM_API_KEY` — the API key for the provider you picked.
4342

4443
3. (Optional) Seed a starter memory file at `.pr-lens/business.md` with your project's domain glossary. The agent will read from it and propose additions over time.
4544

4645
## Inputs
4746

48-
| Name | Required | Default | Description |
49-
|-------------------|----------|----------------------------------|-------------|
50-
| `llm-base-url` | yes | — | OpenAI-compatible endpoint |
51-
| `llm-api-key` | yes | — | API key (auto-masked in logs) |
52-
| `llm-model` | yes | — | Model id at the endpoint (e.g. `gpt-4o`, `deepseek-chat`) |
53-
| `github-token` | no | `${{ github.token }}` | Used to read the PR and write the comment |
54-
| `memory-path` | no | `.pr-lens/business.md` | Path to the project business-memory doc |
55-
| `max-files` | no | `50` | Cap on changed files passed to the agent |
56-
| `max-diff-tokens` | no | `15000` | Soft cap on compressed diff tokens |
57-
| `step-budget` | no | `12` | Hard cap on agent tool-call steps |
58-
| `comment-marker` | no | `<!-- pr-lens:main -->` | HTML marker that identifies the sticky comment |
47+
| Name | Required | Default | Description |
48+
|-------------------|-----------|----------------------------------|-------------|
49+
| `llm-provider` | yes | — | One of `openai`, `anthropic`, `deepseek`, `google`, `groq`, `mistral`, `xai`, `openrouter`, `openai-compatible` |
50+
| `llm-api-key` | yes | — | API key for the provider (auto-masked in logs) |
51+
| `llm-model` | yes | — | Provider-native model id (see table below) |
52+
| `llm-base-url` | sometimes | — | Required **only** when `llm-provider: openai-compatible`. Ignored otherwise |
53+
| `github-token` | no | `${{ github.token }}` | Used to read the PR and write the comment |
54+
| `memory-path` | no | `.pr-lens/business.md` | Path to the project business-memory doc |
55+
| `max-files` | no | `50` | Cap on changed files passed to the agent |
56+
| `max-diff-tokens` | no | `15000` | Soft cap on compressed diff tokens |
57+
| `step-budget` | no | `12` | Hard cap on agent tool-call steps |
58+
| `comment-marker` | no | `<!-- pr-lens:main -->` | HTML marker that identifies the sticky comment |
5959

6060
## Supported providers
6161

62-
Anything that speaks the OpenAI `/chat/completions` protocol:
62+
Pick a provider by name. Each uses its first-party AI SDK package, so
63+
structured outputs, tool calling, and auth headers all work natively — no
64+
compatibility shims.
65+
66+
| `llm-provider` | Example `llm-model` | Env var the provider normally reads |
67+
|---------------------|--------------------------------------|--------------------------------------|
68+
| `openai` | `gpt-4.1`, `gpt-4o-mini` | `OPENAI_API_KEY` |
69+
| `anthropic` | `claude-sonnet-4-5-20250929` | `ANTHROPIC_API_KEY` |
70+
| `deepseek` | `deepseek-chat`, `deepseek-reasoner` | `DEEPSEEK_API_KEY` |
71+
| `google` | `gemini-2.5-pro` | `GOOGLE_GENERATIVE_AI_API_KEY` |
72+
| `groq` | `llama-3.3-70b-versatile` | `GROQ_API_KEY` |
73+
| `mistral` | `mistral-large-latest` | `MISTRAL_API_KEY` |
74+
| `xai` | `grok-4` | `XAI_API_KEY` |
75+
| `openrouter` | `openai/gpt-4.1`, `anthropic/claude-sonnet-4-5` | `OPENROUTER_API_KEY` |
76+
| `openai-compatible` | whatever the endpoint serves | — (set `llm-base-url` + `llm-api-key`) |
77+
78+
The `openai-compatible` escape hatch covers self-hosted vLLM / Ollama / llama.cpp
79+
and anything else that speaks the OpenAI `/chat/completions` protocol. Structured
80+
outputs are enabled, so the endpoint must accept `response_format: json_schema`
81+
(all modern vLLM builds with `--guided-json` do).
82+
83+
### Example: Anthropic
6384

64-
- OpenAI, Azure OpenAI (via their OpenAI-compatible route)
65-
- Anthropic Claude via the [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) or other compatibility proxies
66-
- DeepSeek, Moonshot Kimi, Qwen, Zhipu GLM
67-
- OpenRouter
68-
- Self-hosted vLLM / Ollama / llama.cpp servers
85+
```yaml
86+
- uses: Iris-Ares/pr-lens-github-actions@v0
87+
with:
88+
llm-provider: anthropic
89+
llm-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
90+
llm-model: claude-sonnet-4-5-20250929
91+
```
92+
93+
### Example: OpenRouter
94+
95+
```yaml
96+
- uses: Iris-Ares/pr-lens-github-actions@v0
97+
with:
98+
llm-provider: openrouter
99+
llm-api-key: ${{ secrets.OPENROUTER_API_KEY }}
100+
llm-model: openai/gpt-4.1
101+
```
102+
103+
### Example: self-hosted vLLM
104+
105+
```yaml
106+
- uses: Iris-Ares/pr-lens-github-actions@v0
107+
with:
108+
llm-provider: openai-compatible
109+
llm-base-url: https://vllm.internal/v1
110+
llm-api-key: ${{ secrets.LOCAL_API_KEY }}
111+
llm-model: qwen2.5-7b
112+
```
113+
114+
> **Breaking change from older PR-Lens releases.** Earlier versions took
115+
> `llm-base-url` as the primary switch. You now pass `llm-provider` by name;
116+
> `llm-base-url` is only used with `llm-provider: openai-compatible`.
69117

70118
## Business memory
71119

‎action.yml‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,30 @@
11
name: 'PR-Lens'
2-
description: 'Business-context-aware Verify Checklist for your PRs. BYO LLM (any OpenAI-compatible endpoint).'
2+
description: 'Business-context-aware Verify Checklist for your PRs. Pick a provider by name.'
33
author: 'Iris-Ares'
44
branding:
55
icon: 'eye'
66
color: 'white'
77
inputs:
8-
llm-base-url:
9-
description: 'OpenAI-compatible endpoint (e.g. https://api.openai.com/v1, https://api.deepseek.com/v1, https://ai-gateway.vercel.sh/v1).'
8+
llm-provider:
9+
description: >-
10+
LLM provider to use. One of:
11+
openai, anthropic, deepseek, google, groq, mistral, xai, openrouter, openai-compatible.
12+
Use "openai-compatible" together with llm-base-url for self-hosted vLLM or custom endpoints.
1013
required: true
1114
llm-api-key:
12-
description: 'API key for the endpoint. Will be masked via ::add-mask::.'
15+
description: 'API key for the selected provider. Will be masked via ::add-mask::.'
1316
required: true
1417
llm-model:
15-
description: 'Model identifier at the endpoint, e.g. "gpt-4o", "deepseek-chat", "anthropic/claude-sonnet-4-5".'
18+
description: >-
19+
Provider-native model identifier. Examples: "gpt-4.1" (openai),
20+
"claude-sonnet-4-5-20250929" (anthropic), "deepseek-chat" (deepseek),
21+
"openai/gpt-4o" (openrouter), "qwen2.5-7b" (openai-compatible).
1622
required: true
23+
llm-base-url:
24+
description: >-
25+
Base URL for the endpoint. Required when llm-provider is "openai-compatible";
26+
ignored otherwise.
27+
required: false
1728
github-token:
1829
description: 'Token used to read the PR and write the sticky comment. Defaults to the job token.'
1930
required: false

‎bun.lock‎

Lines changed: 28 additions & 15 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎package.json‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,15 @@
1818
"dependencies": {
1919
"@actions/core": "^1.11.1",
2020
"@actions/github": "^6.0.0",
21-
"@ai-sdk/openai-compatible": "^1.0.0",
22-
"@qraftr/json-repair": "^0.2.2",
21+
"@ai-sdk/anthropic": "^3.0.0",
22+
"@ai-sdk/deepseek": "^2.0.0",
23+
"@ai-sdk/google": "^3.0.0",
24+
"@ai-sdk/groq": "^3.0.0",
25+
"@ai-sdk/mistral": "^3.0.0",
26+
"@ai-sdk/openai": "^3.0.0",
27+
"@ai-sdk/openai-compatible": "^2.0.0",
28+
"@ai-sdk/xai": "^3.0.0",
29+
"@openrouter/ai-sdk-provider": "^2.0.0",
2330
"ai": "^6.0.0",
2431
"fast-glob": "^3.3.3",
2532
"zod": "^3.24.0"

0 commit comments

Comments
 (0)