[Core][Test] move local_rank to the last arg with default value to keep api compatible #3711
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cadedaniel
approved these changes
Mar 29, 2024
Collaborator
cadedaniel
left a comment
cadedaniel
left a comment
There was a problem hiding this comment.
Thanks for the quick turnaround
Comment on lines
+17
to
+19
| env['LOCAL_RANK'] = str(i) | ||
| env['WORLD_SIZE'] = str(number_of_processes) | ||
| env['LOCAL_WORLD_SIZE'] = str(number_of_processes) |
Collaborator
There was a problem hiding this comment.
Should we also test the case where LOCAL_* is not present in env?
Member
Author
There was a problem hiding this comment.
The tests using ray already tested this, i.e. the default value of local_rank=-1.
WoosukKwon
approved these changes
Mar 29, 2024
Collaborator
|
I can try bringing docker container multi node test next week! |
xjpang
pushed a commit
to xjpang/vllm
that referenced
this pull request
Mar 31, 2024
…project#3711) [Core][Test] move local_rank to the last arg with default value to keep api compatible (vllm-project#3711)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
#3686 makes usage of
local_rank, but the API change is breaking. This PR modifies the API to be backward compatible, and add test for the local_rank argument.This is kind of smoke test, as we only test the multi-node code in single node. But that should be fine, because single-node program is also a special case of multi-node program.
#TODO when we find better solution to test multi-node, we can actually test it in multi-node environment.
cc @esmeetu @cadedaniel @simon-mo