Skip to content

[rollout] feat: add optional tags parameter to release()#5537

Open
cavities12 wants to merge 3 commits intoverl-project:mainfrom
cavities12:feat/release-tags
Open

[rollout] feat: add optional tags parameter to release()#5537
cavities12 wants to merge 3 commits intoverl-project:mainfrom
cavities12:feat/release-tags

Conversation

@cavities12
Copy link
Copy Markdown
Collaborator

@cavities12 cavities12 commented Mar 9, 2026

What does this PR do?

Adds an optional tags parameter to release() across all rollout backends (SGLang, vLLM, TRT-LLM), matching the existing resume(tags) API introduced in #1911.

release() currently always releases both kv_cache and weights. With this change, callers can selectively release ["weights"], ["kv_cache"], or both ["kv_cache", "weights"] (the default, preserving backward compatibility).

This completes the symmetry between resume(tags) and release(tags) that was left unfinished in #1911, enabling use cases like resyncing only weights after update_weights() without touching kv_cache.

Checklist Before Starting

Test

15 unit tests covering all 3 backends:

  • Default (None) releases both — backward compatible
  • Selective release (["weights"], ["kv_cache"])
  • Unknown tag validation (ValueError)
  • Edge cases (non-leader noop, free_cache_engine disabled)
  • vLLM: ["weights"]-only raises NotImplementedError (sleep levels don't support it)
tests/workers/rollout/test_release_tags.py .... 15 passed in 0.03s

API and Usage Example

# Before (still works — backward compatible)
await rollout.release()  # releases both kv_cache and weights

# After — selective release
await rollout.release(tags=["weights"])    # release only weights
await rollout.release(tags=["kv_cache"])   # release only kv_cache

Design & Code Changes

File Change
base.py ABC signature: release(self, tags: list[str] | None = None)
sglang_rollout.py Delegates tags to engine, validates unknown tags
vllm_rollout.py Maps tags to sleep levels (1=kv_cache, 2=both), NotImplementedError for weights-only
trtllm_rollout.py Resolves abstract "weights" to internal _WEIGHTS_TAGS, validates via set
test_release_tags.py 15 async tests across all 3 backends

All backends validate unknown tags with ValueError. Tags are normalized to set() to prevent duplicates.

Checklist Before Submitting

  • Read the Contribute Guide.
  • Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
  • Add / Update the documentation. N/A — no existing rollout API reference doc.
  • Add unit or end-to-end test(s) to the CI workflow to cover all the code.
  • Once your PR is ready for CI, send a message in the ci-request channel.
  • If your PR is related to the recipe submodule, update the reference. — N/A.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 9, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an optional tags parameter to the release() method across SGLang, vLLM, and TRT-LLM backends, enhancing granular control over resource release while maintaining backward compatibility and including appropriate validation. A security audit confirmed that these changes do not introduce any high or critical security vulnerabilities. However, a significant maintainability concern has been identified regarding the testing strategy.

@cavities12 cavities12 force-pushed the feat/release-tags branch 2 times, most recently from 193b5f9 to e894627 Compare March 9, 2026 23:27
Add tags parameter to release() across all rollout backends (SGLang,
vLLM, TRT-LLM), matching the existing resume(tags) signature from verl-project#1911.
Callers can now selectively release ["weights"], ["kv_cache"], or both.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@@ -0,0 +1,287 @@
# Copyright 2025 Bytedance Ltd. and/or its affiliates
Copy link
Copy Markdown
Collaborator

@ETOgaosion ETOgaosion Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!
I suggest split vllm, trtllm and sglang tests as diffferent function with arguments, and seperate them to vllm/sgl/trtllm CI test using args to control.
So that when rollout engines upgrade, we can know the API changes at time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants