fix(oai): surface actionable error when CHAT_MODEL/EMBEDDING_MODEL lack a LiteLLM provider prefix#1412
Open
genisis0x wants to merge 1 commit into
Open
Conversation
…ck a LiteLLM provider prefix Closes microsoft#1016. When `LITELLM_CHAT_MODEL` is set to a bare name LiteLLM cannot map to a known provider (`deepseek-chat`, `qwen2:7b`, ...) the eventual `completion()` call raises a generic `BadRequestError: LLM Provider NOT provided` ten retries deep inside the base APIBackend retry loop, with no hint that the fix is to prefix the model with `<provider>/`. Users reported this as "impossible to use Ollama / DeepSeek even though docs say it is supported". Validate the chat and embedding model strings up front in `LiteLLMAPIBackend.__init__` by calling `litellm.utils.get_llm_provider`. If it raises `BadRequestError`, reraise as `RuntimeError` with a clear message that: - names the offending env var (`CHAT_MODEL` / `EMBEDDING_MODEL`), - shows the bad value, - gives several corrected examples (`openai/gpt-4o`, `deepseek/deepseek-chat`, `ollama/qwen2:7b`), - chains the original `BadRequestError` for advanced debugging. The default `chat_model="gpt-4-turbo"` and `embedding_model="text-embedding-3-small"` continue to resolve through LiteLLM's bare-name fallback and remain valid, so existing setups are unaffected. Adds `test/oai/test_litellm_model_validation.py` covering: default chat / embedding names pass; provider-prefixed names (openai, deepseek, ollama, azure) pass; bare unknown names raise `RuntimeError` whose message mentions the env var, the bad value, and the `<provider>/<model>` hint; original `BadRequestError` is preserved as `__cause__`.
Author
|
Read the CLA — all clear from my side. @microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix
Compatibility
Tests
Checklist
📚 Documentation preview 📚: https://RDAgent--1412.org.readthedocs.build/en/1412/