Skip to content

[Minor] Always append newline after image token when parsing chat message #6797

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zhyncs merged 1 commit into from
Jun 1, 2025
Merged

[Minor] Always append newline after image token when parsing chat message #6797

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions python/sglang/srt/conversation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,14 +562,11 @@ def generate_chat_conv(
if content.type == "image_url":
num_image_url += 1
conv.modalities.append(content.modalities)
if num_image_url > 1:
image_token = conv.image_token
else:
image_token = (
conv.image_token + "\n"
if conv.name != "qwen2-vl"
else conv.image_token
)
image_token = (
conv.image_token + "\n"
if conv.name != "qwen2-vl"
else conv.image_token
)
Comment on lines +565 to +569
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The simplification to always append a newline (except for 'qwen2-vl') is clear and aligns with the PR's goal.

Given that this change affects token sequences, which can be critical for model behavior, could you please elaborate on the testing strategy for this modification? Specifically:

  • Are there existing unit tests that cover multi-image scenarios, and have they been updated or verified with this change?
  • If not, would it be beneficial to add new unit tests to explicitly validate the new token sequence for cases with single and multiple images, across different representative models (including one that is not 'qwen2-vl')?

The PR checklist for 'Add unit tests' is currently unchecked, so seeking clarification here. Ensuring robust test coverage will help prevent potential regressions.

add_token_as_needed: bool = (
conv.name in _MODELS_REQUIRING_MODALITY_SUPPLEMENT
)
Expand Down
Loading