Skip to content

feat(providers): Support DeepInfra as an LLM provider#5521

Open
ats3v wants to merge 1 commit intoNousResearch:mainfrom
ats3v:feat/deepinfra-integration
Open

feat(providers): Support DeepInfra as an LLM provider#5521
ats3v wants to merge 1 commit intoNousResearch:mainfrom
ats3v:feat/deepinfra-integration

Conversation

@ats3v
Copy link
Copy Markdown

@ats3v ats3v commented Apr 6, 2026

What does this PR do?

Adds DeepInfra as a first-class LLM provider, giving users access to 100+ top open source and frontier models (GLM-5, DeepSeek-V3+, Qwen3+, Kimi-K2+, Nemotron, etc.) via a single API key.

DeepInfra is one of the largest open-model inference platforms and a top provider on OpenRouter. This integration lets users connect directly — lower latency, simpler billing, no middleman.

The implementation follows the exact same pattern as HuggingFace, KiloCode, and other API-key providers. Purely additive — no changes to existing provider behavior.

Related Issue

N/A — new provider integration.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • hermes_cli/auth.py — Register deepinfra in PROVIDER_REGISTRY (api_key auth, https://api.deepinfra.com/v1/openai) + deep-infra alias
  • hermes_cli/config.py — Add DEEPINFRA_API_KEY and DEEPINFRA_BASE_URL to OPTIONAL_ENV_VARS
  • hermes_cli/models.py — Add 19 curated models, provider label, alias, and display ordering
  • hermes_cli/main.py — Add to provider selection menu, --provider CLI choices, and _model_flow_api_key_provider dispatch
  • agent/auxiliary_client.py — Register mistralai/Mistral-Nemo-Instruct-2407 as default auxiliary model (fast, cheap, reliable for summarization)
  • agent/model_metadata.py — Add deepinfra/deep-infra to provider prefixes and api.deepinfra.com to URL-to-provider mapping
  • .env.example / cli-config.yaml.example — Document the new env var and provider option
  • tests/test_api_key_providers.py — 7 new tests: registration, env vars, base URL, alias resolution, auto-detection, model list consistency

How to Test

  1. Setup:
    # Get a key at https://deepinfra.com/dash/api_keys
    echo "DEEPINFRA_API_KEY=<your-key>" >> ~/.hermes/.env
    
  2. Provider selection:

hermes model # Select "DeepInfra" from the menu

  1. Direct chat:

hermes chat --provider deepinfra -m "deepseek-ai/DeepSeek-V3-0324" -q "Hello"

  1. Tool calling (agent loop):

hermes chat --provider deepinfra -m "deepseek-ai/DeepSeek-V3-0324" -t terminal -q "List files in the current directory"

  1. Auto-detection:

With only DEEPINFRA_API_KEY set (no other provider keys), hermes chat auto-selects DeepInfra.

  1. Unit tests:

pytest tests/test_api_key_providers.py -k deepinfra -v

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Ubuntu 24.04 on WSL2.0

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

image image image

@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from 15cc954 to e3cc103 Compare April 8, 2026 10:51
@ats3v ats3v force-pushed the feat/deepinfra-integration branch 3 times, most recently from 9b24f0c to c7fe34f Compare April 18, 2026 04:15
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from c7fe34f to 2da92e5 Compare April 23, 2026 06:22
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/cli CLI entry point, hermes_cli/, setup wizard comp/agent Core agent loop, run_agent.py, prompt builder area/auth Authentication, OAuth, credential pools labels Apr 23, 2026
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from 2da92e5 to 48e309d Compare April 23, 2026 11:16
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from 48e309d to 47de1be Compare April 28, 2026 06:56
@ats3v
Copy link
Copy Markdown
Author

ats3v commented Apr 28, 2026

@alt-glitch PR up to date and passing all tests. Would really appreciate a review. Thank you!

@ats3v ats3v force-pushed the feat/deepinfra-integration branch 2 times, most recently from ebd0a12 to 65845f4 Compare May 4, 2026 04:41
@ats3v ats3v force-pushed the feat/deepinfra-integration branch from 65845f4 to 6f97129 Compare May 7, 2026 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent loop, run_agent.py, prompt builder comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants