Commit 6a962ef
authored
Fix trtllm-gen attention illegal memory access (#2002)
<!-- .github/pull_request_template.md -->
## 📌 Description
This PR fixes illegal memory access of trtllm-gen attention kernels. It
changes the workspace buffer from `int_workspace_buffer` to
`float_workspace_buffer`. `int_workspace_buffer` is a fixed sized buffer
and not initialized to zero, which should not be used.
<!-- What does this PR do? Briefly describe the changes and why they’re
needed. -->
## 🔍 Related Issues
Issue #1928
## 🚀 Pull Request Checklist
Thank you for contributing to FlashInfer! Before we review your pull
request, please make sure the following items are complete.
### ✅ Pre-commit Checks
- [x] I have installed `pre-commit` by running `pip install pre-commit`
(or used your preferred method).
- [x] I have installed the hooks with `pre-commit install`.
- [x] I have run the hooks manually with `pre-commit run --all-files`
and fixed any reported issues.
> If you are unsure about how to set up `pre-commit`, see [the
pre-commit documentation](https://pre-commit.com/).
## 🧪 Tests
- [x] Tests have been added or updated as needed.
- [x] All tests are passing (`unittest`, etc.).
## Reviewer Notes
<!-- Optional: anything you'd like reviewers to focus on, concerns, etc.
-->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed memory allocation in the decode module to improve computation
accuracy and stability during text generation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->1 parent bb6b620 commit 6a962ef
1 file changed
+1
-1
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1988 | 1988 | | |
1989 | 1989 | | |
1990 | 1990 | | |
1991 | | - | |
| 1991 | + | |
1992 | 1992 | | |
1993 | 1993 | | |
1994 | 1994 | | |
| |||
0 commit comments