Skip to content

Commit abe82f2

Browse files
Crispigliaochangyue
andauthored
[rollout] fix: Fix local rank binding issue when setting RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES (volcengine#2967)
### What does this PR do? Fix local rank binding issue when setting RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES ### Checklist Before Starting [done] Search for similar PR(s). ### Design & Code Changes change verl/verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py ### Checklist Before Submitting [ done ] Read the [Contribute Guide](https://github.com/volcengine/verl?tab=readme-ov-file#contribution-guide). [ done ] Apply [pre-commit checks](https://github.com/volcengine/verl?tab=readme-ov-file#code-linting-and-formatting). --------- Co-authored-by: liaochangyue <[email protected]>
1 parent 2724e8d commit abe82f2

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

verl/workers/rollout/vllm_rollout/vllm_rollout_spmd.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,7 @@
5353

5454
from verl import DataProto
5555
from verl.utils.profiler import GPUMemoryLogger
56+
from verl.utils.ray_utils import ray_noset_visible_devices
5657
from verl.utils.torch_functional import get_response_mask, pad_2d_list_to_length
5758
from verl.workers.rollout.base import BaseRollout
5859

@@ -459,8 +460,7 @@ def get_zeromq_address(self):
459460
def init_worker(self, all_kwargs: list[dict[str, Any]]):
460461
"""Initialize worker engine."""
461462
all_kwargs[0]["rank"] = int(os.environ["RANK"])
462-
all_kwargs[0]["local_rank"] = 0
463-
463+
all_kwargs[0]["local_rank"] = 0 if not ray_noset_visible_devices() else int(os.environ.get("RAY_LOCAL_RANK", 0))
464464
self.vllm_config = all_kwargs[0]["vllm_config"]
465465
self.inference_engine = WorkerWrapperBase(vllm_config=self.vllm_config)
466466
self.inference_engine.init_worker(all_kwargs)

0 commit comments

Comments
 (0)