Chroma query fails: "Collection expecting embedding with dimension of 1024, got 384"

[QingWind6]

Hello! I encountered the following error while querying the ESP-IDF documentation using the Chroma database:
Error executing tool chroma_query_documents: Failed to query documents from collection 'espidf_v5.1': Collection expecting embedding with dimension of 1024, got 384
What does this error mean and how can I resolve this error? Thank you!

[jairad26]

hi, i’m assuming you created the collection before v1.0, and populated it with embeddings. unfortunately, one of the downsides of chroma-mcp is that using pre-existing collections is difficult before v1.0 because we didn’t have collection configurations.

what embedding function did you use to populate, or did you manually insert embeddings?

[QingWind6]

I'm using a custom embedding function implemented based on the bge-large-zh-v1.5 model, and I've confirmed that its word_embedding_dimension is 1024. It appears that when the AI Agent invokes the mcp tool chroma_query_documents, it might default to using the ONNXMiniLM_L6_V2 embedding function for querying. This function relies on the all-MiniLM-L6-v2 model, which has a word_embedding_dimension of 384. I'm wondering if this discrepancy between the embedding model used to populate the collection and the one expected during querying could be contributing to the issue. Could you please suggest any potential solutions to resolve this problem? Many thanks.

[jairad26]

off the top of my head i don’t think there’s a solution that can be implemented in the chroma mcp for custom embedding functions.

but what you could do is create a new mcp server with a tool that just calls your model and returns the embeddings. then you can query using those embeddings directly

[stevenaldinger]

How is this supposed to be used as its currently implemented?

I get the same error but I'm not using a custom embeddings function, just standard OpenAI from chromadb.utils.embedding_functions. I'm looking to use this MCP server to connect to a ChromaDB collection that's been created ahead of time.

Agent output:

Error:
Error executing tool chroma_query_documents: Failed to query documents from collection 'my-collection': Collection expecting embedding with dimension of 1536, got 384

I have a pre-existing ChromaDB instance running that's been populated with OpenAI's embedding function to try to keep compatibility with this MCP server, and it's the latest Docker release.

  chromadb:
    image: ghcr.io/chroma-core/chroma:latest
    environment:
      - IS_PERSISTENT=TRUE
    volumes:
      - chroma-data:/chroma/chroma/
    ports:
      - 8000:8000

The Python client that created the collection is using chromadb==1.0.5 and openai==1.75.0.

Code that creates the collection and inserts documents:

from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction

embedding_fn = OpenAIEmbeddingFunction(
    api_key=os.environ["OPENAI_API_KEY"],
)

collection = client.get_or_create_collection(
    name=collection_name,
    embedding_function=embedding_fn
)

# ... some extra logic before here just splitting some documents...

collection.add(
    documents=[doc.page_content for doc in markdown_contents_documents_split],
    metadatas=[doc.metadata for doc in markdown_contents_documents_split],
    ids=[f"markdown_{i}" for i in range(len(markdown_contents_documents_split))],
)

Configuring the MCP server to connect to it works just fine, which looks like this:

    "chroma": {
      "command": "uvx",
      "args": [
        "chroma-mcp",
        "--client-type",
        "http",
        "--host",
        "127.0.0.1",
        "--port",
        "8000",
        "--ssl",
        "false"
      ],
      "env": {
        "CHROMA_OPENAI_API_KEY": "<redacted>"
      }
  }

[jairad26]

@stevenaldinger that env should work.. is your server on 1.0.5 as well? could you run check collection.configuration and let me know what you get back? this uses a new collection configuration feature that was recently added, so i need to double check that the server side is aware you're using the openai embedding function.

[eromoe]

@jairad26 How to use OpenAIEmbeddingFunction with self-deploy Embeddings Service ?
Does chroma remember the base_url ? How can I do if the base_url changed ?

[eromoe]

I set the .env by

CHROMA_OPENAI_API_BASE=http://localhost:8080
CHROMA_OPENAI_API_KEY=
CHROMA_OPENAI_MODEL=BAAI/bge-small-zh-v1.5

npx @modelcontextprotocol/inspector uvx chroma-mcp --client-type http --host localhost --port 8000 --ssl false --dotenv-path .env

Try with

but result is wrong , it use DefaultEmbeddingFunction

OK I find it need "openai" , but it doesn't use my custom embedding service(my service don't have a key, I use random string to pass the key check. )

[jairad26]

what do you mean by you don’t have a key? without a key it won’t be able to make the API call. if you set CHROMA_OPENAI_API_KEY= with an openai key it will invoke the embedding function.

I have not added support for setting the configuration on the other variables, will add that this weekend

[jairad26]

it also does not work with custom embedding functions if that’s what you’re using, since it would need access to your code. i haven’t come up with an elegant solution for that yet

[vperset3700]

@stevenaldinger that env should work.. is your server on 1.0.5 as well? could you run check collection.configuration and let me know what you get back? this uses a new collection configuration feature that was recently added, so i need to double check that the server side is aware you're using the openai embedding function.

I have the same problem as stevenaldinger, when trying to query chroma database previously populated by azure open ai embeddingseven though i set my env like this. You told steven his env should work but where could he tell the mcp which embedding function to use for semantic search ? I tried with CHROMA_OPENAI_MODEL_NAME but the loggs show chroma mcp uses all-Minim from hunggingface...
env={ "CHROMA_OPENAI_API_KEY": os.environ.get("AZURE_OPENAI_API_KEY"), "CHROMA_OPENAI_API_BASE": os.environ.get("AZURE_OPENAI_API_URL"), "CHROMA_OPENAI_MODEL_NAME": os.environ.get("EMBEDDING"), "CHROMA_OPENAI_API_TYPE": "azure", "CHROMA_OPENAI_API_VERSION": "2024-10-21" }

[jairad26]

sorry for the very late reply @vperset3700 , how chroma embedding functions work post 1.0.5 is they persist the embedding function the collection was created with in the sqlite db. if you could send get the collection.configuration_json for your collection and paste that i can furhter help fixing this.

[testA113]

Hey @jairad26
I'd love some help with this also:

I'm using a voyage ai embedding model, with collections created pre v1.

The current collection config_json_str (from the collections table in the sqlite db) is :

{"hnsw_configuration": {"space": "l2", "ef_construction": 100, "ef_search": 100, "num_threads": 8, "M": 16, "resize_factor": 1.2, "batch_size": 100, "sync_threshold": 1000, "_type": "HNSWConfigurationInternal"}, "_type": "CollectionConfigurationInternal"}

I had a try changing it to:
{"hnsw_configuration": {"space": "l2", "ef_construction": 100, "ef_search": 100, "num_threads": 8, "M": 16, "resize_factor": 1.2, "batch_size": 100, "sync_threshold": 1000, "_type": "HNSWConfigurationInternal"}, "_type": "CollectionConfigurationInternal", "ef_config": {"model_name": "voyage-law-2", "api_key_env_var": "CHROMA_VOYAGE_API_KEY"}}

But with no luck. Any advice would be amazing.

My collections are pretty big, so would much prefer these small changes, rather than spending 2 days and ~$20 to create the new collections

[jairad26]

Hi @testA113 , sorry for the late response. you can replace the config_json_str with this and it will work

{
  "vector_index": {
    "hnsw": {
      "space": "l2",
      "ef_construction": 100,
      "ef_search": 100,
      "max_neighbors": 16,
      "resize_factor": 1.2,
      "sync_threshold": 1000
    }
  },
  "embedding_function": {
    "type": "known",
    "name": "voyageai",
    "config": {
      "api_key_env_var": "CHROMA_VOYAGE_API_KEY",
      "model_name": "voyage-law-2"
    }
  }
}

[shigedangao]

Hi, i'm also encountering this issue. I have store my database into chroma cloud and I also set the embedding model to my database. To my surprise chromadb-mcp could not use the embedding model that i've set despite having set the CHROMA_OPENAI_API_KEY.

Trying to query my database locally by getting the collection and specifying the embedding models seems to work fine. Any idea how to specify the embedding model

[jairad26]

Hi, could you do a get_collection from chroma cloud and print out the collection.configuration?