File ingesting is stuck

[pavelber]

Running private gpt with recommended setup ("ui llms-ollama embeddings-ollama vector-stores-qdrant") on WSL (Ubuntu, Windows 11, 32 gb RAM, i7, Nvidia GeForce RTX 4060 ). LLM Chat (no context from files) works well. When trying to upload a small (1Kb) text file it stucks either on 0% while generating embeddings.
No errors in ollama service log.
What is the issue?

[d1g33k]

Had the same issue with ollama, it just wasn't generating the embeddings on WSL. Worked fine when I switched to llama.cpp though.

[iotnxt]

@d1g33k
-I have the same issue with ollama (too slow to ingest)
-can you please provide the steps you used when you switched to llama.cpp?

if I try this for llama.cpp:
poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"
then I have authentication issues with huggingface when running this: poetry run python scripts/setup
are you using huggingface API? Where do you configure it?

I am running on ubuntu using a docker container

[dbzoo]

If you are using ollama see #1691

[HyuPete]

In my case, I don't use ollama. The file ingesting process is stuck with a tabular-data file (like .csv or .xlsx files). This is what I use:

embedding:
  mode: gemini
  # Should be matching the value above in most cases
  ingest_mode:
    # simple
    pipeline  # the fastest mode according to docs
  count_workers:
    # 4
    # 8
    16  # depends on your machine
  embed_dim: 768

gemini:
  api_key: ${GOOGLE_API_KEY:}
  model:
    models/gemini-1.5-pro
    # models/gemini-1.5-flash
  embedding_model: models/text-embedding-004

To solve the slow ingestion for this file, I split each row of the tabular data into a single-row table file. Then I ingest all those files at once. And the performance of file ingestion for this whole tabular data really speeds up.

Another thing I think we should investigate is the transformations parameter here.
I have tried to not use these transformations or change from SentenceWindowNodeParser to SimpleFileNodeParser, and do see a faster file ingesting speed.

Just hope these notes could help someone out there using this great project.