Skip to content

fix(oai): surface actionable error when CHAT_MODEL/EMBEDDING_MODEL lack a LiteLLM provider prefix#1412

Open
genisis0x wants to merge 1 commit into
microsoft:mainfrom
genisis0x:fix/1016-litellm-provider-routing
Open

fix(oai): surface actionable error when CHAT_MODEL/EMBEDDING_MODEL lack a LiteLLM provider prefix#1412
genisis0x wants to merge 1 commit into
microsoft:mainfrom
genisis0x:fix/1016-litellm-provider-routing

Conversation

@genisis0x
Copy link
Copy Markdown

@genisis0x genisis0x commented May 13, 2026

Summary

Fix

  • Validate the chat and embedding model strings up front in `LiteLLMAPIBackend.init` by calling `litellm.utils.get_llm_provider`.
  • If it raises `BadRequestError`, reraise as `RuntimeError` with a clear message that names the offending env var (`CHAT_MODEL` / `EMBEDDING_MODEL`), shows the bad value, and lists corrected examples (`openai/gpt-4o`, `deepseek/deepseek-chat`, `ollama/qwen2:7b`, `ollama/nomic-embed-text`). The original `BadRequestError` is preserved as `cause` for advanced debugging.

Compatibility

  • The default `chat_model="gpt-4-turbo"` and `embedding_model="text-embedding-3-small"` continue to resolve through LiteLLM's bare-name fallback and remain valid.
  • Any user who already has `CHAT_MODEL=ollama/qwen2:7b` etc. is unaffected.
  • Users who previously hit the buried `LLM Provider NOT provided` error will now see a clear actionable message before any HTTP call.

Tests

  • New `test/oai/test_litellm_model_validation.py` covers: default chat / embedding names pass; provider-prefixed names (openai, deepseek, ollama, azure) pass; bare unknown names raise `RuntimeError` whose message mentions the env var, the bad value, and the `/` hint; original `BadRequestError` is preserved as `cause`.
  • No live LLM endpoint required — tests use the same client-side `get_llm_provider` resolution the validation function uses.

Checklist

  • Issue linked
  • Tests added
  • No behavioural change for the default model configuration
  • CLA will be confirmed in a follow-up comment

📚 Documentation preview 📚: https://RDAgent--1412.org.readthedocs.build/en/1412/

…ck a LiteLLM provider prefix

Closes microsoft#1016.

When `LITELLM_CHAT_MODEL` is set to a bare name LiteLLM cannot map to a
known provider (`deepseek-chat`, `qwen2:7b`, ...) the eventual
`completion()` call raises a generic
`BadRequestError: LLM Provider NOT provided` ten retries deep inside
the base APIBackend retry loop, with no hint that the fix is to
prefix the model with `<provider>/`. Users reported this as
"impossible to use Ollama / DeepSeek even though docs say it is
supported".

Validate the chat and embedding model strings up front in
`LiteLLMAPIBackend.__init__` by calling `litellm.utils.get_llm_provider`.
If it raises `BadRequestError`, reraise as `RuntimeError` with a
clear message that:

- names the offending env var (`CHAT_MODEL` / `EMBEDDING_MODEL`),
- shows the bad value,
- gives several corrected examples
  (`openai/gpt-4o`, `deepseek/deepseek-chat`, `ollama/qwen2:7b`),
- chains the original `BadRequestError` for advanced debugging.

The default `chat_model="gpt-4-turbo"` and
`embedding_model="text-embedding-3-small"` continue to resolve through
LiteLLM's bare-name fallback and remain valid, so existing setups are
unaffected.

Adds `test/oai/test_litellm_model_validation.py` covering: default
chat / embedding names pass; provider-prefixed names (openai, deepseek,
ollama, azure) pass; bare unknown names raise `RuntimeError` whose
message mentions the env var, the bad value, and the
`<provider>/<model>` hint; original `BadRequestError` is preserved as
`__cause__`.
@genisis0x
Copy link
Copy Markdown
Author

Read the CLA — all clear from my side.

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] RD-Agent/LiteLLM: "LLM Provider NOT provided" even with correct provider/model for Ollama/DeepSeek (env, patch, CLI all fail)

1 participant