Skip to content

Improve model config/tokenizer format defaults#1031

Open
Daniel-Schenker wants to merge 5 commits into
torch-spyre:mainfrom
Daniel-Schenker:model_config_format_fix_v2
Open

Improve model config/tokenizer format defaults#1031
Daniel-Schenker wants to merge 5 commits into
torch-spyre:mainfrom
Daniel-Schenker:model_config_format_fix_v2

Conversation

@Daniel-Schenker

@Daniel-Schenker Daniel-Schenker commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR improves upon the faulty logic used in #1027.

The previous fix attempted to exclude Llama models from mistral-format detection by checking whether params.json existed only inside an original/ subdirectory, using two calls to any_pattern_in_repo_files. This did not work as the vLLM implementation of any_pattern_in_repo_files delegates to list_filtered_repo_files, which matches patterns using fnmatch.fnmatch(os.path.basename(file), pattern). This always returns false since os.path.basename is applied to every file path before fnmatch runs. Llama models with only original/params.json were still being assigned mistral format.

fix: correctly detect root-level params.json for mistral format detection.

Replace both any_pattern_in_repo_files calls with a single list_repo_files call followed by an exact string membership test.

Related Issues

Test Plan

Checklist

  • I have read the contributing guidelines
  • My code follows the project's code style (run bash format.sh)
  • I have added tests for my changes (if applicable)
  • I have updated the documentation (if applicable)
  • My commits include a Signed-off-by: line (DCO compliance)

@github-actions

Copy link
Copy Markdown

👋 Hi! Thank you for contributing.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
@Daniel-Schenker Daniel-Schenker force-pushed the model_config_format_fix_v2 branch from 9413816 to 76d7755 Compare July 15, 2026 17:21
…cached online models

Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Signed-off-by: Daniel Schenker <daniel.schenker@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant