Skip to content

Conversation

@donpromax
Copy link
Contributor

Prevent loss of MLATransformerConfig keys in check_and_disable_incompatible_configs for dsv3

What does this PR do?

Add concise overview of what this PR aims to achieve or accomplish. Reference related GitHub issues and PRs that help with the review.

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: ...
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly addresses an issue where valid configuration keys for MLATransformerConfig were being removed. By generalizing the check_and_disable_incompatible_configs function to accept a configuration class, you've made it more reusable and fixed the bug for the dsv3 model configuration. I have one suggestion to improve the implementation of check_and_disable_incompatible_configs to avoid side effects, which is a good practice for writing robust and predictable code.

@ETOgaosion
Copy link
Collaborator

Thanks for contribution~ But seems duplicate with #2933

@ETOgaosion
Copy link
Collaborator

Since #2933 has been merged, close this PR, thanks for contribution.

@donpromax Can you try #2933 ?

@ETOgaosion ETOgaosion closed this Aug 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants