Skip to content

Commit 9944bbf

Browse files
jokerwytjimoosciuc
authored andcommitted
Free metadata_buffer_index after transfer finished (sgl-project#5364)
1 parent dc23fb5 commit 9944bbf

1 file changed

Lines changed: 5 additions & 0 deletions

File tree

python/sglang/srt/disaggregation/prefill.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,11 @@ def process_disagg_prefill_inflight_queue(self: Scheduler) -> None:
226226
elif poll == KVPoll.Failed:
227227
raise Exception("Transferring failed")
228228

229+
for req in done_reqs:
230+
self.disagg_prefill_pending_queue.req_to_metadata_buffer_idx_allocator.free(
231+
req.metadata_buffer_index
232+
)
233+
229234
# Stream requests which have finished transfer
230235
self.stream_output(done_reqs, False, None)
231236

0 commit comments

Comments
 (0)