Skip to content

vllm 0.16.0 Support in current plugin #769

@sducouedic

Description

@sducouedic

Feature description

Add support vllm 0.16.0 in the current plugin

Motivation and context

vllm version 0.16.0 contains the PR#32863 that will fix wrong error message bug #33418. There is an internal request to fix this bug appearing when the length of the request doesn't fit the max_context length

cc: @karthick-vasakar @yannicks1

Proposed solution

No response

Checklist

  • I have searched for similar feature requests

Metadata

Metadata

Labels

help wantedExtra attention is neededvllm-spyre-oldRelated to the continued maintenance of the old `vllm-spyre` plugin on the `torch_sendnn` stack.

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions