wip: migrate to any llm

[mscolnick]

This removes fragile logic for converting to different LLMs and instead rely on https://github.com/mozilla-ai/any-llm.

Some oustanding blocking issues:

• thinking does not seem to be supported for anthropic, openai, google
• bedrock not tested

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Project	Deployment	Preview	Comments	Updated (UTC)
marimo-docs	✅ Ready	Preview	Comment	Aug 13, 2025 0:36am

[njbrake]

Hi! I'm on the any-llm team. Feel free to let me know if there are features not yet supported in any-llm that you need. I see the ticket mentions lack of "thinking" support, not sure if you're referencing thinking as an input parameter or as an output item.

[mscolnick]

hey @njbrake, it did not seem like any reasoning tokens were returned. Could be an issue in our code, but I couldn't get reasoning/thinking tokens from the completions API

[njbrake]

Ah, I figured it out: we don't support reasoning tokens in all providers yet: see https://mozilla-ai.github.io/any-llm/providers/ for the compatibility matrix.

OpenAI (the provider) doesn't return reasoning tokens in their completion api, only in their responses API. See https://blog.mozilla.ai/standardized-reasoning-content-a-first-look-at-using-openais-gpt-oss-on-multiple-providers-using-any-llm/ for the explanation of those differences.

For anthropic and google, we don't support it yet but I believe those providers do return reasoning information, so we would only need to implement the conversion logic to be able to return it.

[mscolnick]

@njbrake awesome, thank you! will definitely upgrade a try it out.