Skip to content

Commit 522c554

Browse files
namanlalitnyurtourgeman
authored andcommitted
[Bugfix] Optimize CpuGpuBuffer initialization (vllm-project#25447)
Signed-off-by: Naman Lalit <nl2688@nyu.edu>
1 parent 81982f3 commit 522c554

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

vllm/v1/utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,7 +117,7 @@ def __init__(
117117
dtype=dtype,
118118
device="cpu",
119119
pin_memory=pin_memory)
120-
self.gpu = self.cpu.to(device)
120+
self.gpu = torch.zeros_like(self.cpu, device=device)
121121
self.np: np.ndarray
122122
# To keep type hints simple (avoiding generics and subclasses), we
123123
# only conditionally create the numpy array attribute. This can cause

0 commit comments

Comments
 (0)