Skip to content

Commit c61fa56

Browse files
Iris-Aresclaude
Iris-Ares
and
claude
committed
refactor: switch to first-party provider dispatch; fix agent finalization and rescue crash
A production run of PR-Lens failed with two compounding defects: 1. The ToolLoopAgent exhausted its 12-step budget without calling submit_report. 2. The rescue path then crashed with AI_NoObjectGeneratedError because the generic @ai-sdk/openai-compatible provider was created without supportsStructuredOutputs, so `Output.object({ schema })` silently sent the schema-less response_format and the model returned text that didn't parse. Rather than paper over (2) with a capability flag and keep the generic BYO wrapper for every provider, this change pivots to provider-name dispatch: users pass llm-provider = openai | anthropic | deepseek | google | groq | mistral | xai | openrouter | openai-compatible, and each branch uses the respective first-party @ai-sdk/* package which already knows its own capability shape. Other fixes in the same pass: - Add prepareStep to force `submit_report` on the final allowed step, so stopWhen: hasToolCall('submit_report') fires by construction and the rescue becomes a true safety net rather than a normal fallback. - Simplify rescue to a single generateText + Output.object call against VerifyReportSchema directly. Delete the repairJson / Lenient-schema / alias- normalisation layers and the @qraftr/json-repair dependency — structured outputs guarantee the exact Zod shape. - Keep @ai-sdk/openai-compatible as the explicit escape hatch (llm-provider: openai-compatible) with supportsStructuredOutputs + includeUsage. - Bump LanguageModelV2 → LanguageModelV3 everywhere; all provider SDKs at v3.x implement V3. - Fix .gitignore so dist/ actually ships (previous ignore line contradicted the comment right below it and the README's stated intent). Breaking change: existing workflows must add `llm-provider:` and drop `llm-base-url:` unless they're using an OpenAI-compatible self-hosted endpoint. README and self-test workflow updated accordingly. Smoke-tested end-to-end against sindresorhus/ky#853 via Moonshot kimi-k2.5 (openai-compatible branch): 7 steps, submittedViaTool=true, 15k tokens, no warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 4c5bb11 commit c61fa56

13 files changed

Lines changed: 559 additions & 154 deletions

File tree

‎.env.example‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,14 @@
33
#
44
# .env is gitignored — never commit your real keys.
55

6-
# --- Required: LLM provider (BYO, OpenAI-compatible) ---
7-
LLM_BASE_URL=https://api.openai.com/v1
6+
# --- Required: LLM provider ---
7+
# LLM_PROVIDER is one of: openai, anthropic, deepseek, google, groq, mistral,
8+
# xai, openrouter, openai-compatible.
9+
# LLM_BASE_URL is required ONLY when LLM_PROVIDER=openai-compatible.
10+
LLM_PROVIDER=openai
811
LLM_API_KEY=sk-replace-me
9-
LLM_MODEL=gpt-4o
12+
LLM_MODEL=gpt-4.1
13+
# LLM_BASE_URL=https://vllm.internal/v1 # only with LLM_PROVIDER=openai-compatible
1014

1115
# --- Optional: GitHub auth ---
1216
# Public PRs work without this, but you'll hit the 60 req/h anonymous limit

‎.github/workflows/self-test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,6 @@ jobs:
1818
- name: Run PR-Lens on itself
1919
uses: ./
2020
with:
21-
llm-base-url: ${{ secrets.LLM_BASE_URL }}
21+
llm-provider: ${{ vars.LLM_PROVIDER || 'openai' }}
2222
llm-api-key: ${{ secrets.LLM_API_KEY }}
2323
llm-model: ${{ vars.LLM_MODEL || 'gpt-4o-mini' }}

‎.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,5 +7,4 @@ node_modules/
77
.bun-cache/
88
coverage/
99
evals/results/
10-
dist/
1110
# dist/ is intentionally committed — Node GitHub Actions run from the tagged dist bundle

‎README.md‎

Lines changed: 69 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -32,40 +32,88 @@ It is intentionally orthogonal to code-review bots. It answers *"what should a h
3232
with: { fetch-depth: 0 }
3333
- uses: Iris-Ares/pr-lens-github-actions@v0
3434
with:
35-
llm-base-url: ${{ secrets.LLM_BASE_URL }}
35+
llm-provider: openai
3636
llm-api-key: ${{ secrets.LLM_API_KEY }}
37-
llm-model: gpt-4o
37+
llm-model: gpt-4.1
3838
```
3939
4040
2. Store your provider credentials as repo secrets:
41-
- `LLM_BASE_URL` — any OpenAI-compatible endpoint (e.g. `https://api.openai.com/v1`, `https://api.deepseek.com/v1`, `https://openrouter.ai/api/v1`, or a local `https://vllm.internal/v1`).
42-
- `LLM_API_KEY` — the API key for that endpoint.
41+
- `LLM_API_KEY` — the API key for the provider you picked.
4342

4443
3. (Optional) Seed a starter memory file at `.pr-lens/business.md` with your project's domain glossary. The agent will read from it and propose additions over time.
4544

4645
## Inputs
4746

48-
| Name | Required | Default | Description |
49-
|-------------------|----------|----------------------------------|-------------|
50-
| `llm-base-url` | yes | — | OpenAI-compatible endpoint |
51-
| `llm-api-key` | yes | — | API key (auto-masked in logs) |
52-
| `llm-model` | yes | — | Model id at the endpoint (e.g. `gpt-4o`, `deepseek-chat`) |
53-
| `github-token` | no | `${{ github.token }}` | Used to read the PR and write the comment |
54-
| `memory-path` | no | `.pr-lens/business.md` | Path to the project business-memory doc |
55-
| `max-files` | no | `50` | Cap on changed files passed to the agent |
56-
| `max-diff-tokens` | no | `15000` | Soft cap on compressed diff tokens |
57-
| `step-budget` | no | `12` | Hard cap on agent tool-call steps |
58-
| `comment-marker` | no | `<!-- pr-lens:main -->` | HTML marker that identifies the sticky comment |
47+
| Name | Required | Default | Description |
48+
|-------------------|-----------|----------------------------------|-------------|
49+
| `llm-provider` | yes | — | One of `openai`, `anthropic`, `deepseek`, `google`, `groq`, `mistral`, `xai`, `openrouter`, `openai-compatible` |
50+
| `llm-api-key` | yes | — | API key for the provider (auto-masked in logs) |
51+
| `llm-model` | yes | — | Provider-native model id (see table below) |
52+
| `llm-base-url` | sometimes | — | Required **only** when `llm-provider: openai-compatible`. Ignored otherwise |
53+
| `github-token` | no | `${{ github.token }}` | Used to read the PR and write the comment |
54+
| `memory-path` | no | `.pr-lens/business.md` | Path to the project business-memory doc |
55+
| `max-files` | no | `50` | Cap on changed files passed to the agent |
56+
| `max-diff-tokens` | no | `15000` | Soft cap on compressed diff tokens |
57+
| `step-budget` | no | `12` | Hard cap on agent tool-call steps |
58+
| `comment-marker` | no | `<!-- pr-lens:main -->` | HTML marker that identifies the sticky comment |
5959

6060
## Supported providers
6161

62-
Anything that speaks the OpenAI `/chat/completions` protocol:
62+
Pick a provider by name. Each uses its first-party AI SDK package, so
63+
structured outputs, tool calling, and auth headers all work natively — no
64+
compatibility shims.
65+
66+
| `llm-provider` | Example `llm-model` | Env var the provider normally reads |
67+
|---------------------|--------------------------------------|--------------------------------------|
68+
| `openai` | `gpt-4.1`, `gpt-4o-mini` | `OPENAI_API_KEY` |
69+
| `anthropic` | `claude-sonnet-4-5-20250929` | `ANTHROPIC_API_KEY` |
70+
| `deepseek` | `deepseek-chat`, `deepseek-reasoner` | `DEEPSEEK_API_KEY` |
71+
| `google` | `gemini-2.5-pro` | `GOOGLE_GENERATIVE_AI_API_KEY` |
72+
| `groq` | `llama-3.3-70b-versatile` | `GROQ_API_KEY` |
73+
| `mistral` | `mistral-large-latest` | `MISTRAL_API_KEY` |
74+
| `xai` | `grok-4` | `XAI_API_KEY` |
75+
| `openrouter` | `openai/gpt-4.1`, `anthropic/claude-sonnet-4-5` | `OPENROUTER_API_KEY` |
76+
| `openai-compatible` | whatever the endpoint serves | — (set `llm-base-url` + `llm-api-key`) |
77+
78+
The `openai-compatible` escape hatch covers self-hosted vLLM / Ollama / llama.cpp
79+
and anything else that speaks the OpenAI `/chat/completions` protocol. Structured
80+
outputs are enabled, so the endpoint must accept `response_format: json_schema`
81+
(all modern vLLM builds with `--guided-json` do).
82+
83+
### Example: Anthropic
6384

64-
- OpenAI, Azure OpenAI (via their OpenAI-compatible route)
65-
- Anthropic Claude via the [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) or other compatibility proxies
66-
- DeepSeek, Moonshot Kimi, Qwen, Zhipu GLM
67-
- OpenRouter
68-
- Self-hosted vLLM / Ollama / llama.cpp servers
85+
```yaml
86+
- uses: Iris-Ares/pr-lens-github-actions@v0
87+
with:
88+
llm-provider: anthropic
89+
llm-api-key: ${{ secrets.ANTHROPIC_API_KEY }}
90+
llm-model: claude-sonnet-4-5-20250929
91+
```
92+
93+
### Example: OpenRouter
94+
95+
```yaml
96+
- uses: Iris-Ares/pr-lens-github-actions@v0
97+
with:
98+
llm-provider: openrouter
99+
llm-api-key: ${{ secrets.OPENROUTER_API_KEY }}
100+
llm-model: openai/gpt-4.1
101+
```
102+
103+
### Example: self-hosted vLLM
104+
105+
```yaml
106+
- uses: Iris-Ares/pr-lens-github-actions@v0
107+
with:
108+
llm-provider: openai-compatible
109+
llm-base-url: https://vllm.internal/v1
110+
llm-api-key: ${{ secrets.LOCAL_API_KEY }}
111+
llm-model: qwen2.5-7b
112+
```
113+
114+
> **Breaking change from older PR-Lens releases.** Earlier versions took
115+
> `llm-base-url` as the primary switch. You now pass `llm-provider` by name;
116+
> `llm-base-url` is only used with `llm-provider: openai-compatible`.
69117

70118
## Business memory
71119

‎action.yml‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,30 @@
11
name: 'PR-Lens'
2-
description: 'Business-context-aware Verify Checklist for your PRs. BYO LLM (any OpenAI-compatible endpoint).'
2+
description: 'Business-context-aware Verify Checklist for your PRs. Pick a provider by name.'
33
author: 'Iris-Ares'
44
branding:
55
icon: 'eye'
66
color: 'white'
77
inputs:
8-
llm-base-url:
9-
description: 'OpenAI-compatible endpoint (e.g. https://api.openai.com/v1, https://api.deepseek.com/v1, https://ai-gateway.vercel.sh/v1).'
8+
llm-provider:
9+
description: >-
10+
LLM provider to use. One of:
11+
openai, anthropic, deepseek, google, groq, mistral, xai, openrouter, openai-compatible.
12+
Use "openai-compatible" together with llm-base-url for self-hosted vLLM or custom endpoints.
1013
required: true
1114
llm-api-key:
12-
description: 'API key for the endpoint. Will be masked via ::add-mask::.'
15+
description: 'API key for the selected provider. Will be masked via ::add-mask::.'
1316
required: true
1417
llm-model:
15-
description: 'Model identifier at the endpoint, e.g. "gpt-4o", "deepseek-chat", "anthropic/claude-sonnet-4-5".'
18+
description: >-
19+
Provider-native model identifier. Examples: "gpt-4.1" (openai),
20+
"claude-sonnet-4-5-20250929" (anthropic), "deepseek-chat" (deepseek),
21+
"openai/gpt-4o" (openrouter), "qwen2.5-7b" (openai-compatible).
1622
required: true
23+
llm-base-url:
24+
description: >-
25+
Base URL for the endpoint. Required when llm-provider is "openai-compatible";
26+
ignored otherwise.
27+
required: false
1728
github-token:
1829
description: 'Token used to read the PR and write the sticky comment. Defaults to the job token.'
1930
required: false

‎bun.lock‎

Lines changed: 28 additions & 15 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)