1.2.0

Introducing any-llm Gateway 🚀

In release 1.2.0 we're excited to release the any-llm gateway: allowing you to manage users, budgets, and LLM providers in your infrastructure.

Check out the docs: https://mozilla-ai.github.io/any-llm/gateway/overview/

Enjoy!

New Providers

We add support for minimax and zai providers, as well as the gateway client so that you can easily communicate with the any-llm gateway from the any-llm client 🙏

What's Changed

• feat(gateway): add gateway provider by @njbrake in #587
• feat(zai): Add ZaiProvider for z.ai API integration by @njbrake in #583
• feat(minimax): Add Minimax provider support by @njbrake in #585
• chore(databricks): disable integration tests until we get more signal… by @njbrake in #590
• fix(tests): fix integration test filter with dynamic job determination by @njbrake in #589
• chore: loosen tool test by @njbrake in #592
• fix: hf wakeup script catch both exception types by @njbrake in #593
• Update guide by @angpt in #591
• chore: more reliable models for testing by @njbrake in #594
• pre-commit autoupdate by @github-actions[bot] in #595
• pre-commit autoupdate by @github-actions[bot] in #597
• Update readme by @angpt in #596
• feat: any-llm-gateway by @njbrake in #598
• chore: gateway image name by @njbrake in #601
• chore: Nebius through portkey dropped qwen3-14b by @njbrake in #603

New Contributors

• @angpt made their first contribution in #591

Full Changelog: 1.1.0...1.2.0

1.1.0

Batch API

We're excited to release our initial API for Batch! For inference requests that aren't urgent, many providers offer a batch service that have discounts over using the completions/responses API. Our initial API functions are marked experimental and subject to changes, and the initial implementation is only for OpenAI. We are interested to hear from you regarding which other providers you would like to have prioritized, feel free to search or file Github issues. Thanks!

What's Changed

• pre-commit autoupdate by @github-actions[bot] in #574
• Minor docs update to dogfood our new external contribution guide. by @JohnDickerson in #576
• refactor(mistral): Put all preprocessing logic inside `_convert_compl… by @daavoo in #577
• feat(batch): Support for Native Batch API by @njbrake in #578

New Contributors

• @JohnDickerson made their first contribution in #576

Full Changelog: 1.0.0...1.1.0

1.0.0

Release Version 1.0

Any-LLM has now reached what we believe to be a stable API, we will now work to be careful prior to making any changes that will affect user integration 🎉

What's Changed

• feat(cohere): support reasoning by @njbrake in #545
• feat(azure): support dict input with json_schema by @njbrake in #544
• chore: remove 'source code' column in favor of pre-amble by @njbrake in #548
• chore(deps): bump pre-commit-uv from 4.1.5 to 4.2.0 by @dependabot[bot] in #538
• chore(deps): bump astral-sh/setup-uv from 6 to 7 by @dependabot[bot] in #537
• feat(deepseek): add reasoning support by @njbrake in #549
• feat(cerebras): enable reasoning support in CerebrasProvider by @njbrake in #550
• feat(fireworks): enable reasoning support by @njbrake in #551
• feat(moonshot): support completion reasoning content by @njbrake in #555
• pre-commit autoupdate by @github-actions[bot] in #553
• feat(databricks): completion reasoning by @njbrake in #556
• feat(bedrock): completion reasoning by @njbrake in #557
• feat(huggingface): support reasoning tokens by @njbrake in #558
• feat(llamacpp): Completion Reasoning support by @njbrake in #560
• feat(llamafile): completion reasoning support by @njbrake in #561
• feat(nebius); completion reasoning by @njbrake in #562
• Update test_responses.py by @njbrake in #554
• feat(sambanova): completion reasoning by @njbrake in #564
• feat(together): support for completion reasoning by @njbrake in #565
• chore: disable watsonx CI tests by @njbrake in #566
• fix: use supported image model by @njbrake in #567
• chore: small update to docs for difference in stateful vs stateless apis by @njbrake in #568
• feat(portkey): completion reasoning by @njbrake in #563
• fix(huggingface): Endpoint doesn't accept max_new_tokens anymore by @njbrake in #570
• chore: fix anthropic integration test by @njbrake in #571
• make fireworks happy by @njbrake in #572
• fix(google): add usage data by @njbrake in #573

Full Changelog: 0.21.0...1.0.0

0.21.0

What's Changed

• Feat/openrouter reasoning 291 by @mihirahuja1 in #345
• fix(openrouter): Only override _convert_completion_params. by @daavoo in #493
• docs: Fix AnyLLM Class snippet. by @daavoo in #495
• tests: Drop mocks in test-docs. by @daavoo in #496
• pre-commit autoupdate by @github-actions[bot] in #491
• feat: Add support for BUILT_IN_TOOLS. by @daavoo in #497
• fix(gemini): Support BUILT_IN_TOOLS. by @daavoo in #499
• tests: Enable groq integration tests. by @daavoo in #500
• fix(inception):.Update SUPPORTED properties. by @daavoo in #502
• tests: Enable inception integration tests. by @daavoo in #503
• fix(databricks): Update SUPPORTED properties. by @daavoo in #505
• tests: Enable databricks integration tests. by @daavoo in #506
• fix(moonshot): Update SUPPORTED properties. by @daavoo in #507
• tests: Enable moonshot integration tests. by @daavoo in #508
• fix(sambanova): Convert response_format to expected type. by @daavoo in #509
• tests: Enable sambanova integration tests. by @daavoo in #510
• fix(demos): Updates to use alist_models from public API. by @daavoo in #511
• fix(together): Disable SUPPORTS_COMPLETION_PDF. by @daavoo in #514
• fix(together): Don't raise exception with empty choices. by @daavoo in #515
• tests: Enable together integration tests. by @daavoo in #516
• tests: Enable cohere integration tests. by @daavoo in #518
• tests: Enable deepseek integration tests. by @daavoo in #521
• Make any_llm.version attribute available by @HareeshBahuleyan in #512
• pre-commit autoupdate by @github-actions[bot] in #513
• tests: Enable portkey integration tests. by @daavoo in #523
• tests: Add azureopenai unit tests. by @daavoo in #526
• tests: Add deepseek unit tests. by @daavoo in #528
• fix(fireworks): Properly handle reasoning content. by @daavoo in #531
• tests: Add voyage unit tests. by @daavoo in #533
• pre-commit autoupdate by @github-actions[bot] in #534
• Add image support for Anthropic by @JosephChataignon in #536
• feat(mistral): add support for OpenAI-style response format dictionaries with Mistral by @HareeshBahuleyan in #535

Full Changelog: 0.20.3...0.21.0

0.20.3

What's Changed

• tests: Avoid waking up hf endpoint if filter was provided and doesn't need it. by @daavoo in #435
• chore(deps): Bump ruff to rev: "v0.13.0" by @daavoo in #436
• chore(deps): Pin pydantic>2,<3. by @daavoo in #440
• 381 revisit public api options by @daavoo in #438
• chore: Drop run-mypy.sh in favor of un run mypy. by @daavoo in #441
• refactor: Replace list_models_async with alist_models. by @daavoo in #442
• refactor(together): Cleanup unused `_stream_completion. by @daavoo in #445
• refactor(watsonx): Cleanup unused `_stream_completion. by @daavoo in #446
• fix(vertexai): Don't use custom env vars. by @daavoo in #447
• fix(vertexai): Don't check for unused api_key. by @daavoo in #448
• refactor: Introduce _init_client for instantiating client once. by @daavoo in #444
• refactor(bedrock): Drop instructor usage and raise error for `respons… by @daavoo in #452
• refactor(together): Drop instructor and use native response_format … by @daavoo in #454
• refactor(sambanova): Drop instructor usage. by @daavoo in #455
• fix(llamafile): Don't onverride __init__. by @daavoo in #457
• refactor(sagemaker): Drop instructor usage. by @daavoo in #459
• refactor(utils): Drop instructor module. by @daavoo in #460
• fix(azureopenai): Drop unused _get_client. Use `super()._init_clien… by @daavoo in #463
• refactor(bedrock): Drop AWS_REGION getenv. by @daavoo in #465
• refactor: Drop api_base getenv for providers inheriting from `BaseO… by @daavoo in #467
• refactor(watsonx): Drop WATSONX_SERVICE_URL getenv. by @daavoo in #469
• refactor(sagemaker): Drop AWS_REGION getenv. by @daavoo in #471
• chore: Merge tests-integration-local into tests-integration as a separate job. by @daavoo in #473
• tests: Enable perplexity integration tests. by @daavoo in #474
• refactor: Drop ClientConfig. by @daavoo in #476
• tests: Fix test_list_models for huggingface. by @daavoo in #479
• pre-commit autoupdate by @github-actions[bot] in #478
• fix(llamacpp): Allow to instantiate without api_key. by @daavoo in #481
• tests: Enable llamacpp integration tests by @daavoo in #482
• Allow override of allow_running_loop parameter by @HareeshBahuleyan in #483
• refactor(GoogleProvider): Move all preprocessing logic inside `_conve… by @daavoo in #484
• fix: threading approach in run_async_in_sync didn't wait for all tasks to complete by @njbrake in #487

New Contributors

• @HareeshBahuleyan made their first contribution in #483

Full Changelog: 0.20.2...0.20.3

0.20.2

What's Changed

• fix: parallel tool call handling by @njbrake in #425
• chore: docs for types by @njbrake in #428
• chore: split into multiple files by @njbrake in #429
• tests: Enable nebius integration tests. by @daavoo in #432
• fix(google): Handle content in list format. by @daavoo in #434

Full Changelog: 0.20.1...0.20.2

0.20.1

What's Changed

• feat: separate test for local providers by @njbrake in #422
• fix: local tests shouldnt run nonlocal providers by @njbrake in #423
• cerebras keys by @njbrake in #424
• fix: missing default for pop by @njbrake in #427

Full Changelog: 0.20.0...0.20.1

0.20.0

What's Changed

• feat: experimental AWS Sagemaker support by @njbrake in #411
• Demo Number 2: Find a model across providers by @njbrake in #414
• feat: support for passing images into completion api by @njbrake in #417
• feat: use internal API for more organized implementations by @njbrake in #412
• feat: testing for pdf usage by @njbrake in #420

Full Changelog: 0.19.1...0.20.0

0.19.1

What's Changed

• fix(gemin): Implement _verify_and_set_api_key. by @daavoo in #410
• fix: accept an expected_providers env var by @njbrake in #409
• chore(deps): bump actions/setup-python from 5 to 6 by @dependabot[bot] in #407
• chore(deps): bump actions/stale from 9 to 10 by @dependabot[bot] in #406
• chore(deps): bump pre-commit-uv from 4.1.4 to 4.1.5 by @dependabot[bot] in #405

Full Changelog: 0.19.0...0.19.1

0.19.0

BREAKING CHANGE

aws provider has been renamed to bedrock. In preparation for #183

Sorry for the disruptive change, when we reach Version 1 of any-llm we will begin deprecation cycles for changes like this to avoid the disruption. You can see that roadmap here: https://github.com/mozilla-ai/any-llm/milestone/1

What's Changed

• chore: rename 'aws' to 'bedrock' by @njbrake in #403
• chore: Add missing py.typed marker. by @daavoo in 90e970a

Full Changelog: 0.18.0...0.19.0