[BUG] Ollama long running generate error

[sebiweise]

Describe the bug
When using Ollama server and a Vercel deployment I´m running into some timeouts

Screenshots

Error
Error: Your function was stopped as it did not return an initial response within 25s

Additional context
Ollama server running on Hetzner Cloud with a Nginx Proxy Manager in front to proxy the ollama server to https and enable Access Lists for HTTP Basic Auth

[jorge-menjivar]

Is this different from #55?

[sebiweise]

Yes and no, the problem in #55 was the fact that the /api/models call had a timeout. But the /api/chat call has the same problem when using Ollama. Everything longer then just a "Test" is taking to long to respond and result into the error I have described here.
The example "Why is the sky blue?" was the prompt that wasn't working here so nothing special.

[jorge-menjivar]

Have you been able to narrow down the cause of this?

[sebiweise]

No not now, I will try to create a more powerfull server with a gpu, maybe that will make ollama more reliable.

[daniel-farina]

I just saw this, I guess it's fixed here now: #81

Please give it a try it should be working now.