Skip to content

Fix swa available memory check#15867

Merged
ispobock merged 2 commits intosgl-project:mainfrom
ispobock:fix-swa-memory-check
Dec 26, 2025
Merged

Fix swa available memory check#15867
ispobock merged 2 commits intosgl-project:mainfrom
ispobock:fix-swa-memory-check

Conversation

@ispobock
Copy link
Copy Markdown
Collaborator

@ispobock ispobock commented Dec 26, 2025

Motivation

When running swa model like mimo-v2-flash for a while, it will raise error:

[2025-12-26 03:42:15 DP1 TP4] Scheduler hit an exception: Traceback (most recent call last):
  File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 2932, in run_scheduler_process
    scheduler.event_loop_overlap()
  File "/usr/local/lib/python3.12/dist-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 1122, in event_loop_overlap
    batch = self.get_next_batch_to_run()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 1844, in get_next_batch_to_run
    self.running_batch = self.update_running_batch(self.running_batch)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/managers/scheduler.py", line 2093, in update_running_batch
    old_available_tokens = self.token_to_kv_pool_allocator.available_size()
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/sgl-workspace/sglang/python/sglang/srt/mem_cache/allocator.py", line 215, in available_size
    raise NotImplementedError()
NotImplementedError

It's related to this change: #15694
The available size of swa token_to_kv_pool_allocator should get seperately, ref changes in #7367

cc: @hnyls2002 @hanming-lu for double-check

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@ispobock
Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@ispobock ispobock merged commit c28c536 into sgl-project:main Dec 26, 2025
63 of 86 checks passed
YChange01 pushed a commit to YChange01/sglang that referenced this pull request Jan 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant