[Feature]: Add embeddings api for Llama

[Harsha-Pulagam]

currently I load openai api server using the command
python3 -m vllm.entrypoints.openai.api_server --model Llama3-8B-Instruct --dtype auto --host 0.0.0.0 --port 8051 --gpu-memory-utilization 0.8 --enforce-eager
I want to try embedding using llama3 but. after loading i can see that embedding API is not loaded

I couldn't find any parm to enable embeddings.

Help me to enable embeddings API

[DarkLight1337]

This means that the model doesn't support embeddings in vLLM yet. You can edit this issue to request such implementation.

[Harsha-Pulagam]

Which models can generate embeddings in vllm endpoint?
I am using Llama 3

[DarkLight1337]

Currently, only Mistral is supported (#3734).

https://github.com/vllm-project/vllm/blob/main/vllm/model_executor/models/__init__.py#L82

[Harsha-Pulagam]

Thanks for the reply.
I have edited the issue to request

[github-actions]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[DarkLight1337]

Closed as completed by #9806. Note that you need to set your architecture to LlamaModel for this to work.