Skip to content

feat(wren-ai-service): add support for new GPT-5 models in configuration files (ai-env-changed) #1875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cyyeh merged 6 commits into from
Aug 8, 2025
Merged

feat(wren-ai-service): add support for new GPT-5 models in configuration files (ai-env-changed) #1875

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ab7edec
feat(wren-ai-service): add support for new GPT-5 models in configurat…
yichieh-lu Aug 8, 2025
a450d66
fix
cyyeh Aug 8, 2025
648d089
fix(config): update context window size for GPT-5 models to 380000 ac…
yichieh-lu Aug 8, 2025
39d83f7
fix(config): standardize context window size for all GPT-5 models to …
yichieh-lu Aug 8, 2025
02b6f74
update
cyyeh Aug 8, 2025
0720398
Merge branch 'main' into feat/wren-ai-service/support-gpt-5
cyyeh Aug 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion deployment/kustomizations/base/cm.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,24 @@ data:
n: 1
seed: 0
temperature: 0

- model: gpt-5-nano-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-mini-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-2025-08-07
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Add temperature to keep generation behaviour consistent

All existing GPT-4.1 entries explicitly set temperature: 0 inside kwargs.
Omitting it for the three new GPT-5 variants means the provider will fall back to its own default (often ≈ 0.7), changing determinism and potentially breaking tests that assume zero-temperature generation.

     kwargs:
       max_tokens: 4096
       n: 1
       seed: 0
+      temperature: 0

Please confirm whether non-zero temperature is intended; if not, the above patch aligns behaviour with the existing models.

🏁 Script executed:

#!/bin/bash
set -e

echo "Existing GPT-4 entries (showing 2 lines before and 5 after each match):"
rg -n "model: gpt-4" -B2 -A5 deployment/kustomizations/base/cm.yaml

echo -e "\nNew GPT-5 entries (showing 2 lines before and 5 after each match):"
rg -n "model: gpt-5" -B2 -A5 deployment/kustomizations/base/cm.yaml

Length of output: 1516

Add explicit temperature: 0 to new GPT-5 entries for deterministic behavior

The existing GPT-4.1 variants all include

kwargs:
  …
  seed: 0
  temperature: 0

but the new GPT-5 models omit temperature, causing the provider to fall back to its default (≈0.7). To keep generation behavior consistent and tests from breaking, please add:

@@ deployment/kustomizations/base/cm.yaml
     kwargs:
       max_tokens: 4096
       n: 1
       seed: 0
+      temperature: 0

Applies to lines 78–90.

🤖 Prompt for AI Agents 
In deployment/kustomizations/base/cm.yaml lines 78 to 90, the new GPT-5 model
entries are missing the explicit temperature: 0 setting under kwargs, which
causes non-deterministic behavior. To fix this, add temperature: 0 alongside
seed: 0 in the kwargs section for each GPT-5 model entry to ensure consistent
and deterministic generation behavior.

context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
---
type: embedder
provider: litellm_embedder
Expand Down
20 changes: 19 additions & 1 deletion docker/config.example.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,25 @@ models:
n: 1
seed: 0
temperature: 0

- model: gpt-5-nano-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-mini-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0

Copy link
Contributor

@coderabbitai coderabbitai bot Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

temperature missing → non-deterministic GPT-5 responses
All existing GPT-4.1 entries hard-code temperature: 0; the new GPT-5 models omit it, so Litellm/OpenAI will default to ~1.0. That yields very different, non-reproducible answers and breaks tests or embeddings that assume deterministic output.
Add temperature: 0 (or another explicit value) unless randomness is intended.

       seed: 0
+      temperature: 0
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- model: gpt-5-nano-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-mini-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-nano-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
temperature: 0
- model: gpt-5-mini-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
temperature: 0
- model: gpt-5-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
temperature: 0
🧰 Tools
🪛 YAMLlint (1.37.1)

[error] 45-45: trailing spaces

(trailing-spaces)

🤖 Prompt for AI Agents 
In docker/config.example.yaml between lines 27 and 45, the new GPT-5 model
entries lack the temperature setting, causing non-deterministic outputs due to
the default temperature of ~1.0. To fix this, add the line "temperature: 0"
under each GPT-5 model's kwargs section to ensure deterministic responses
consistent with the existing GPT-4.1 entries.

---
type: embedder
provider: litellm_embedder
Expand Down
796 changes: 694 additions & 102 deletions wren-ai-service/poetry.lock
View file
Open in desktop

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions wren-ai-service/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ package-mode = false
[tool.poetry.dependencies]
python = ">=3.12.*, <3.13"
fastapi = "^0.115.2"
uvicorn = {extras = ["standard"], version = "^0.30.1"}
uvicorn = {extras = ["standard"], version = "^0.29.0"} # Litellm requires uvicorn < 0.30.0
python-dotenv = "^1.0.1"
haystack-ai = "==2.7.0"
openai = "^1.40.0"
Expand All @@ -33,8 +33,8 @@ pydantic-settings = "^2.5.2"
google-auth = "^2.35.0"
tiktoken = "^0.8.0"
jsonschema = "^4.23.0"
litellm = "^1.70.0"
boto3 = "^1.35.90"
litellm = {extras = ["proxy"], version = "^1.75.2"}
boto3 = "^1.34.34" # Litellm requires boto3 = 1.34.34
qdrant-client = "==1.11.0"

[tool.poetry.group.dev.dependencies]
Expand Down
18 changes: 18 additions & 0 deletions wren-ai-service/tools/config/config.example.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,24 @@ models:
n: 1
seed: 0
temperature: 0
- model: gpt-5-nano-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-mini-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
Copy link
Contributor

@coderabbitai coderabbitai bot Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Inconsistent defaults & missing temperature for GPT-5 models
Same issue as in docker/config.example.yaml: omit temperature leads to divergent behaviour. Mirror the GPT-4.1 rows or justify the change.

Also, note that only the GPT-4.1-nano row has alias: default; switching to GPT-5 via wren-launcher will not update the alias and the pipelines will still resolve litellm_llm.default → GPT-4.1-nano. Consider adding an alias or updating PrepareConfigFileForOpenAI.

🏁 Script executed:

#!/bin/bash
set -e

# Locate all config.example.yaml files
echo "Config files in repo:"
fd --type f config.example.yaml

echo
# Show GPT-4.1 entries in wren-ai-service config
echo "=== GPT-4.1 entries in wren-ai-service/tools/config/config.example.yaml ==="
rg -n "model: gpt-4.1" -A5 wren-ai-service/tools/config/config.example.yaml

echo
# Show alias lines in wren-ai-service config
echo "=== alias entries in wren-ai-service/tools/config/config.example.yaml ==="
rg -n "alias:" -A2 wren-ai-service/tools/config/config.example.yaml || echo "No alias found"

echo
# Show temperature settings in wren-ai-service config
echo "=== temperature occurrences in wren-ai-service/tools/config/config.example.yaml ==="
rg -n "temperature" -n wren-ai-service/tools/config/config.example.yaml || echo "No temperature settings found"

echo
# Show GPT-5 entries in wren-ai-service config (for context)
echo "=== GPT-5 entries in wren-ai-service/tools/config/config.example.yaml ==="
rg -n "model: gpt-5" -A4 wren-ai-service/tools/config/config.example.yaml

echo
# Locate docker config example file
DOCKER_CFG=$(fd -g "docker/config.example.yaml")
echo "docker config file: $DOCKER_CFG"
echo
# Preview first 50 lines of docker/config.example.yaml
head -n 50 "$DOCKER_CFG"

Length of output: 2373

🏁 Script executed:

#!/bin/bash
set -e

echo "=== First 50 lines of docker/config.example.yaml ==="
head -n 50 docker/config.example.yaml || { echo "Failed to read docker/config.example.yaml"; exit 1; }

Length of output: 1145

Ensure consistent temperature and default alias for GPT-5 models
The GPT-5 entries currently omit temperature (falling back to provider defaults) and lack an alias: default, which causes divergent behavior and prevents wren-launcher from switching the default. Please mirror the GPT-4.1 blocks or justify this change:

  • wren-ai-service/tools/config/config.example.yaml
    • Under the three GPT-5 models (gpt-5-nano-2025-08-07, gpt-5-mini-2025-08-07, gpt-5-2025-08-07), add

      temperature: 0

    • Add

    - alias: default

    to the gpt-5-nano-2025-08-07 block so that litellm_llm.default no longer resolves to the old GPT-4.1-nano model.

  • docker/config.example.yaml
    Apply the same updates for the GPT-5 entries: include temperature: 0 and set alias: default on the gpt-5-nano row.

If there’s an intentional reason to drop these defaults (e.g., a new default temperature or alias scheme), please document it or adjust PrepareConfigFileForOpenAI accordingly.

🤖 Prompt for AI Agents 
In wren-ai-service/tools/config/config.example.yaml between lines 27 and 44, the
GPT-5 model entries lack the temperature setting and a default alias, causing
inconsistent behavior and preventing wren-launcher from switching defaults. Add
temperature: 0 under each GPT-5 model's configuration and add alias: default to
the gpt-5-nano-2025-08-07 block to align with GPT-4.1 blocks. Also, apply the
same changes to docker/config.example.yaml for consistency. If these defaults
are intentionally omitted, document the reason or update
PrepareConfigFileForOpenAI accordingly.


---
type: embedder
Expand Down
18 changes: 18 additions & 0 deletions wren-ai-service/tools/config/config.full.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,24 @@ models:
n: 1
seed: 0
temperature: 0
- model: gpt-5-nano-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-mini-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0
- model: gpt-5-2025-08-07
context_window_size: 1000000
kwargs:
max_tokens: 4096
n: 1
seed: 0

---
type: embedder
Expand Down
5 changes: 4 additions & 1 deletion wren-launcher/commands/launch.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ func askForGenerationModel() (string, error) {

prompt := promptui.Select{
Label: "Select an OpenAI's generation model",
Items: []string{"gpt-4.1", "gpt-4.1-mini", "gpt-4.1-nano"},
Items: []string{"gpt-4.1", "gpt-4.1-mini", "gpt-4.1-nano", "gpt-5", "gpt-5-mini", "gpt-5-nano"},
}

_, result, err := prompt.Run()
Expand Down Expand Up @@ -389,6 +389,9 @@ func getOpenaiGenerationModel() (string, bool) {
"gpt-4.1": true,
"gpt-4.1-mini": true,
"gpt-4.1-nano": true,
"gpt-5": true,
"gpt-5-mini": true,
"gpt-5-nano": true,
}
if !validModels[openaiGenerationModel] {
pterm.Error.Println("Invalid generation model", openaiGenerationModel)
Expand Down
Loading