Publish vllm GPU container image

[surajssd]

What would you like to be added:

Please publish the vllm (and other providers) container image (on each release or each merge to main) instead of asking the user to build themselves. This would improve the ease of project usage.

Why is this needed:

So that users don't have to build the container images themselves.

Completion requirements:

Images published to a container registry (ghcr is easiest I guess!) that users can use, without having to modify the configuration here:

lws/docs/examples/vllm/GPU/lws.yaml

Line 17 in 5462520

image: <image-built-from-dockerfile>

This enhancement requires the following artifacts:

• Docs update
• Configs / artifacts update
• Add GH action pipelines

[kerthcet]

I think we can remove the ray_init.sh to simplify the deployment, at least for GPU, this is how we handle the command:
https://github.com/InftyAI/llmaz/blob/3f3bea7e67945783b0d78fa442ffe9dd1724a40e/chart/templates/backends/vllm.yaml#L15-L57
cc @ahg-g @Edwinhr716

Calling for volunteer.

[yankay]

I think we can remove the ray_init.sh to simplify the deployment, at least for GPU, this is how we handle the command: https://github.com/InftyAI/llmaz/blob/3f3bea7e67945783b0d78fa442ffe9dd1724a40e/chart/templates/backends/vllm.yaml#L15-L57 cc @ahg-g @Edwinhr716

Calling for volunteer.

HI @kerthcet

If we put ray_init.sh into a configmap, like: https://itnext.io/kubernetes-tips-using-scripts-inside-configmaps-9df03e17ac35, wouldn't this solve the problem while also preventing the command in lws.yaml looks too long?

What do you think?

[kerthcet]

I'm ok with both approaches, just to avoid another step to build the image.

[Edwinhr716]

We have a PR out to add the script to the vLLM image vllm-project/vllm#12913, currently waiting for the PR to be merged.

[Edwinhr716]

PR has been merged. vLLM doesn't do nightly releases for their GPU images, so this script will only be available on the next vLLM release. Will keep open until that happens

[surajssd]

@Edwinhr716 I guess, a new vllm image release would also mean we remove the script from this repo?

[Edwinhr716]

correct

[kerthcet]

Seems will be included in v0.7.4? We can refactor the yaml at that time.

[kerthcet]

Seems will be included in v0.7.4? We can refactor the yaml at that time.

[Edwinhr716]

Yeah, I don't believe vLLM does cherrypicking for their patch releases so if they release v0.7.4 it should be included there. Or v0.8.0 if they don't do another patch release. I'm keeping track of their releases, so I'll assign it to myself to clean up the yamls once the release is ready

/assign @Edwinhr716

[yankay]

We have a PR out to add the script to the vLLM image vllm-project/vllm#12913, currently waiting for the PR to be merged.

It has been released https://github.com/vllm-project/vllm/releases/tag/v0.8.0