Improve model config/tokenizer format defaults#1031
Open
Daniel-Schenker wants to merge 5 commits into
Open
Conversation
|
👋 Hi! Thank you for contributing. We also recommend installing prek and configuring it to check your code before every local commit. |
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
9413816 to
76d7755
Compare
…cached online models Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR improves upon the faulty logic used in #1027.
The previous fix attempted to exclude Llama models from mistral-format detection by checking whether params.json existed only inside an original/ subdirectory, using two calls to
any_pattern_in_repo_files. This did not work as the vLLM implementation ofany_pattern_in_repo_filesdelegates tolist_filtered_repo_files, which matches patterns usingfnmatch.fnmatch(os.path.basename(file), pattern). This always returns false sinceos.path.basenameis applied to every file path before fnmatch runs. Llama models with only original/params.json were still being assigned mistral format.fix: correctly detect root-level params.json for mistral format detection.
Replace both
any_pattern_in_repo_files callswith a singlelist_repo_filescall followed by an exact string membership test.Related Issues
Test Plan
Checklist
bash format.sh)Signed-off-by:line (DCO compliance)