Support for embedding models

[DLNovice]

I want to use embedding model services provided by Ollama or vLLM, but the current configuration file doesn't allow me to specify them directly. I tried modifying server.py, and it seems to work, but I feel like my code isn't very clean or standard. Does the official team have any plans to support this?

[nealrauhauser]

I forked this code base and one of my mods was using a 768 dimension embedding instead of 384. This performs well enough on my little M1 Macs, but I have large volumes of documents to process in my future and an Ollama option is interesting. Are your mods available anywhere?

[onestardao]

looks like a config/integration gap: the server can't point at an external embedding service (ollama / vLLM) out of the box. that’s a classic embedding-integration problem — maps to ProblemMap No.5 (Semantic ≠ Embedding).

short options you can try

1. add a small config block to point to an embedding endpoint and proxy requests to ollama/vLLM.
2. run a tiny adapter process that translates your app’s embedding calls into the external service API.
3. if you want to keep it local, switch to a compatible embedding model dim (eg 384 vs 768) or normalize at ingestion.

if you want, i can paste a 3-line config snippet and a 1-line curl test. reply “send” and i’ll drop them. also, ping me if you want the ProblemMap link (No.5) and i’ll share it.