[roadmap] Rollout Module Development Progress & Roadmap

[chenhaiq]

Recent goals for the rollout module include: vllm/sglang as the asynchronous inference engine, multi-turn, and tool using.

🔧 Tools Using

1. Code Sandbox (P0)

• Use the code tool via Sandbox Fusion. Owners: @Irvingwangjr @SwordFaith | PR: feat: sandbox fusion for multi-turn #1525
• End2End reproducing Retool. @SwordFaith

2. Web Search (P0)

• Search tool invocation during Actor rollout; integrates local dense retriever or custom search engines. Owner: @Lins-01 | PR: [sglang] Feat: Search Tool Invocation in Multi-Turn RL Training #1682

3. MCP Integration (P0)

• Replace function calls with MCP for richer tool usage. Owner: @AlecHenx | Issue: Support MCP tool using for multi turn #1837

4. Utils (P0)

• Global tool registration for extensibility and multimodal tool support. Owner: @wuxibin89 | Issue: server base: Global tool registry interface #1723

---

🔄 Server-based Async Multi-turn Rollout

1. vLLM Async Rollout (P0)

• The initial async rollout design using vLLM. Owner: @wuxibin89 | PR: [rollout] feat: introduce vLLM AsyncLLM to support multi-turn rollout #1138
• Performance issue in vLLM async engine. Owner: @wuxibin89

2. SGLang Async Rollout (P0) : Multi-turn rollout Status & Roadmap · Issue #131 · zhaochenyang20/Awesome-ML-SYS-Tutorial

• Async Engine & multi-turn & tool using. Owners: @SwordFaith , @zhaochenyang20 | PR: [sglang] feat: Add SGLang async multi-turn rollout with tool support #1037
• Performance Optimizations with 75% time reduction. Owner: @chenhaiq | PR: [rollout] feat: ChatScheduler requests sglang fully async #1769

3. Unified SGLang & vLLM Usage (P0)

• chat_scheduler asOpenAI tool calling schema for both SGlang and vLLM. Owner: @wuxibin89 | PR: [rollout] feat: follow OpenAI tool calling schema in chat scheduler #1831 | Issue: server base: ChatScheduler requests sglang openai-compatible server base #1721

4. Load-balanced Rollout (P1)

• request generation workload load balance among dp ranks Owner: @Irvingwangjr | PR: [vLLM]feat: support micro batch for vllm #1818 | Issue: server base: ChatScheduler send out requests to workers in loadbalance #1722

5. Performance Optimizations (P1)

• async compute score | Issue: Async pipeline in generate and compute score #1584

6. Observability (P1)

• rollout tracing Owner: @chenhaiq | Issue: [feature request] log generation data to help debugging #2188

---

⚡️ Adanced Optimizations

1. Partial Rollout. Owner: stargazerZJ | PR: #1826

2. FP8 Rollout. | Issue: #1803

3. New Algorithm Truncated PPO