fetch_url and fetch_many in web/crawl4ai_provider.py wrap their async work with a blocking _run_coro helper that joins the thread from the event-loop thread, so the MCP server serves nothing for the duration of a fetch (raised by @jordan-gibbs in #86, follow-up requested there).
Making fetch_url async def and awaiting anyio.to_thread.run_sync directly would keep the server responsive during a fetch. It touches the provider's fetch signature more than #86's scope, which is why it stayed out of that PR.
Not something I've built yet, opening this so it doesn't get lost.
fetch_urlandfetch_manyinweb/crawl4ai_provider.pywrap their async work with a blocking_run_corohelper that joins the thread from the event-loop thread, so the MCP server serves nothing for the duration of a fetch (raised by @jordan-gibbs in #86, follow-up requested there).Making
fetch_urlasync defand awaitinganyio.to_thread.run_syncdirectly would keep the server responsive during a fetch. It touches the provider's fetch signature more than #86's scope, which is why it stayed out of that PR.Not something I've built yet, opening this so it doesn't get lost.