Disable outlines cache by default #14837
Conversation
GHSA-mgrm-fgjv-mhv8 Outlines provides a cache for its compiled grammars on the local filesystem. This cache has been on by default in vLLM. Outlines is also available by default through the OpenAI compatible API server. A malicious user can send a stream of very short decoding requests with unique schemas, resulting in an addition to the cache for each request. This can result in a Denial of Service if the filesystem runs out of space. Note that even if vLLM was configured to use a different backend by default, it is still possible to choose outlines on a per-request basis using the `guided_decoding_backend` key of the `extra_body` field of the request. This issue applies to the V0 engine only. The V1 engine is not affected. Signed-off-by: Russell Bryant <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
Signed-off-by: Louis Ulmer <[email protected]>
Signed-off-by: Mu Huai <[email protected]>
3 participants
GHSA-mgrm-fgjv-mhv8
Outlines provides a cache for its compiled grammars on the local
filesystem. This cache has been on by default in vLLM. Outlines is also
available by default through the OpenAI compatible API server.
A malicious user can send a stream of very short decoding requests with
unique schemas, resulting in an addition to the cache for each request.
This can result in a Denial of Service if the filesystem runs out of
space.
Note that even if vLLM was configured to use a different backend by
default, it is still possible to choose outlines on a per-request basis
using the
guided_decoding_backendkey of theextra_bodyfield of therequest.
This issue applies to the V0 engine only. The V1 engine is not affected.
Signed-off-by: Russell Bryant [email protected]