Skip to content

[Bug]: v1 speculate decoding NgramProposer experiences service exceptions during stress testing #14742

@skylee-01

Description

@skylee-01

Your current environment

The output of `python collect_env.py`
Your output of `python collect_env.py` here

🐛 Describe the bug

Image
Force the propose function to always output np.array([87, 15, 284, 16752]) upon each invocation, and under long-term stress testing, an exception occurred.

Image

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions