Skip to content

Commit ef76173

Browse files
xiezhq-hermannTianQiLin666666
authored andcommitted
HiCache, fixing hash value indexing (sgl-project#8636)
1 parent bf7ddb6 commit ef76173

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

python/sglang/srt/mem_cache/hiradix_cache.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -436,7 +436,7 @@ def check_prefetch_progress(self, req_id: str):
436436
last_host_node,
437437
fetched_token_ids,
438438
written_indices,
439-
hash_value[:min_completed_tokens],
439+
hash_value[: min_completed_tokens // self.page_size],
440440
)
441441
if len(written_indices):
442442
self.cache_controller.mem_pool_host.update_prefetch(written_indices)
@@ -529,7 +529,7 @@ def _insert_helper_host(self, node: TreeNode, key: List, host_value, hash_value)
529529
prefix_len = self.key_match_fn(node.key, key)
530530
key = key[prefix_len:]
531531
host_value = host_value[prefix_len:]
532-
hash_value = hash_value[prefix_len:]
532+
hash_value = hash_value[prefix_len // self.page_size :]
533533
matched_length += prefix_len
534534

535535
if prefix_len < len(node.key):

0 commit comments

Comments
 (0)