[1/N] Initial Implementation of Parser for ResponsesAPI by qandrew · Pull Request #32712 · vllm-project/vllm

qandrew · 2026-01-20T18:49:19Z

Purpose

Initial Implementation of #32713. This PR introduces

the concept of a Parser class to consolidate reasoning / tool calling, a ParserManager, with the first example being the MinimaxM2 parser.
hook up the Parser into responsesAPI.
parser manager lazy loads, similar to [Refactor] Lazy-loaded reasoning_parser #28092
No functional changes expected, just refactors.

Future goals

hook up GPT-OSS to use parser
have parser be used in ChatCompletions

Test Plan

vllm serve MiniMaxAI/MiniMax-M2   --tensor-parallel-size 4   --tool-call-parser minimax_m2   --reasoning-parser minimax_m2    --enable-auto-tool-choice --trust-remote-code

 curl -X POST "http://localhost:8000/v1/responses" \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer dummy-api-key" \
    -d '{
      "model": "MiniMaxAI/MiniMax-M2",
      "input": "What is the weather in Paris in Celsius today?",
      "tools": [
        {
          "type": "function",
          "name": "get_weather",
          "description": "Get the current weather in a given location.",
          "parameters": {
            "type": "object",
            "properties": {
              "location": { "type": "string" },
              "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
            },
            "required": ["location", "unit"]
          }
        }
      ]
    }' | jq

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

gemini-code-assist

Code Review

This pull request introduces a new abstraction layer for unifying reasoning and tool parsing, which is a significant architectural improvement. The Parser abstract class, DelegatingParser, and ParserManager provide a modular and extensible system for handling different parsing strategies. The integration into serving.py and responses/serving.py correctly utilizes this new abstraction. Type hints and docstrings are well-maintained, contributing to code clarity and maintainability.

vllm/entrypoints/openai/engine/serving.py

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-01-22T15:23:36Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @qandrew.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-01-22T17:59:37Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

cursor · 2026-01-22T18:07:01Z

vllm/entrypoints/openai/engine/serving.py

+        _WrappedParser.reasoning_parser_cls = reasoning_parser_cls
+        _WrappedParser.tool_parser_cls = tool_parser_cls
+
+        return _WrappedParser


Global class attribute mutation causes shared state bug

Medium Severity

The _get_parser method modifies class-level attributes on the globally-shared _WrappedParser class (reasoning_parser_cls and tool_parser_cls). Since _WrappedParser is imported from a module, these attributes are shared across all usages. If multiple OpenAIServingResponses instances are created with different parser configurations (e.g., in tests, multi-model deployments, or configuration changes), later instances will overwrite the class attributes, affecting all previous instances that reference _WrappedParser.

cursor · 2026-01-22T18:07:01Z

vllm/entrypoints/openai/engine/serving.py

+                    )
+                    return parser
+                except KeyError:
+                    pass


Strategy 2 silently overrides user's explicit parser choices

Medium Severity

Strategy 2 in _get_parser can silently override the user's explicit parser configuration. When a user specifies different parsers for tools and reasoning (e.g., --tool-call-parser hermes --reasoning-parser minimax_m2), Strategy 2 iterates through both names and returns the first unified parser found. If "minimax_m2" has a unified parser registered, the user's explicit "hermes" tool parser choice is silently ignored and MinimaxM2ToolParser is used instead. This contradicts the PR's stated intent of "No functional changes expected."

cursor · 2026-01-22T18:07:01Z

vllm/entrypoints/openai/responses/serving.py

-                            reasoning_parser_cls=self.reasoning_parser,
+                            reasoning_parser_cls=self.parser.reasoning_parser_cls
+                            if self.parser
+                            else None,


Missing null check causes crash with ParsableContext

Medium Severity

When VLLM_USE_EXPERIMENTAL_PARSER_CONTEXT=True, the code passes self.parser.reasoning_parser_cls to ParsableContext, but only checks if self.parser is not None. If a user configures only a tool parser (no reasoning parser), self.parser will be _WrappedParser with reasoning_parser_cls = None. This passes None to ParsableContext, which raises ValueError("reasoning_parser_cls must be provided.") at runtime. The check needs to verify that reasoning_parser_cls is also not None before using ParsableContext.

mergify · 2026-01-28T18:52:27Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-01-28T21:13:35Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

chaunceyjiang

Thank you. I will research this issue in the coming days.
Also worth mentioning is that over the past two days, while looking into #33215, I've noticed some limitations in the current design of the ReasoningParser.
It might require a refactoring.

chaunceyjiang · 2026-01-30T01:50:53Z

vllm/parser/minimax_m2_parser.py

+logger = init_logger(__name__)
+
+
+@ParserManager.register_module("minimax_m2")


I don't really recommend adding another ParserManager.
Is it possible to have a self-contained mechanism to handle these tasks?

i would ideally like to merge reasoning parser and tool calling parser managers into a single ParserManager, then we can deprecate ReasoningManager / ToolCallingManager. would you have thoughts on this?

qandrew · 2026-01-30T07:18:01Z

Thank you. I will research this issue in the coming days. Also worth mentioning is that over the past two days, while looking into #33215, I've noticed some limitations in the current design of the ReasoningParser. It might require a refactoring.

i'd be curious on your thoughts for ReasoningParser refactor and seeing how we could put it inside Parser :)

mergify · 2026-01-31T12:56:03Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @qandrew.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

chaunceyjiang · 2026-02-02T06:38:08Z

i'd be curious on your thoughts for ReasoningParser refactor and seeing how we could put it inside Parser :)

I’m not quite sure why this needs to be placed in a parser.

For some older models that don’t have reasoning, how are you planning to handle them?

qandrew · 2026-02-02T17:52:41Z

i'd be curious on your thoughts for ReasoningParser refactor and seeing how we could put it inside Parser :)

I’m not quite sure why this needs to be placed in a parser.

For some older models that don’t have reasoning, how are you planning to handle them?

if there's no reasoning parser, there will be no parser, and the behavior will not be changed
https://github.com/vllm-project/vllm/pull/32712/changes#diff-56ef3613a051485ec100a639ae3964f0ba1a1897372186cd79826f7a390c13ddR335

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-02-02T18:26:15Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-02-03T04:41:22Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

chaunceyjiang

Thanks to @qandrew for the patience and for walking me through the design in detail.

I think it makes sense to unify the tool parser and reasoning.

For this PR, I’d suggest starting by introducing get_parser, class Parser, and class WrappedParser.

ParserManager and MiniMaxM2Parser can be introduced in a separate PR(ParserManager hasn’t been invoked yet.
).

vllm/entrypoints/openai/engine/serving.py

chaunceyjiang · 2026-02-03T05:59:48Z

vllm/parser/minimax_m2_parser.py

+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+
+"""
+MiniMax M2 Parser - A unified parser for MiniMax M2 models.


As we discussed offline, this is just an example. We can add it in a separate PR.

vllm/parser/abstract_parser.py

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-02-03T18:30:53Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: Andrew Xia <axia@fb.com>

mergify · 2026-02-03T21:16:38Z

Hi @qandrew, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: Andrew Xia <axia@fb.com>

chaunceyjiang

Thanks~

chaunceyjiang · 2026-02-04T02:55:49Z

vllm/parser/parser_manager.py

+                except KeyError:
+                    pass
+
+        # Strategy 3: Create a DelegatingParser with the individual parser classes


To be honest, personally I tend to keep only Strategy 3. I’d prefer to drop Strategy 1 and Strategy 2.

We can use a single Parser to unify the interface, but still keep the implementations of reasoning_parser and tool_parser separate. Basically, keep the current structure.

This is also why I’d like you to remove vllm/parser/minimax_m2_parser.py.

Given that this PR is fairly urgent for you right now, I’m okay with merging it first, and we can discuss this part afterward.

sounds good, appreciate the help! let me follow up with cleaning up minimax_m2_parser. It might be more clear to motivate strategy 1 through GPTOSS harmony parser, or if we want to support models that do interleaved reasoning :)

i think i'd prefer to move reasoning_parser / tool_parser into parser, but happy to chat about that more

…#32712) Signed-off-by: Andrew Xia <axia@fb.com> Co-authored-by: Andrew Xia <axia@fb.com> Signed-off-by: felix01.yu <felix01.yu@vipshop.com>

…#32712) Signed-off-by: Andrew Xia <axia@fb.com> Co-authored-by: Andrew Xia <axia@fb.com>

mergify bot added the frontend label Jan 20, 2026

gemini-code-assist bot reviewed Jan 20, 2026

View reviewed changes

vllm/entrypoints/openai/engine/serving.py Outdated Show resolved Hide resolved

qandrew mentioned this pull request Jan 20, 2026

[RFC]: Unified Parser for reasoning, tool calling #32713

Open

6 tasks

Andrew Xia added 5 commits January 21, 2026 19:33

initial commit

785fc3f

Signed-off-by: Andrew Xia <axia@fb.com>

abstract parser and minimax implementation

545c006

Signed-off-by: Andrew Xia <axia@fb.com>

delegating parser

3fad022

Signed-off-by: Andrew Xia <axia@fb.com>

use parser in responses/serving

479b998

Signed-off-by: Andrew Xia <axia@fb.com>

refactor

11ca440

Signed-off-by: Andrew Xia <axia@fb.com>

qandrew force-pushed the parser branch from 0111087 to 11ca440 Compare January 22, 2026 04:32

mergify bot added the needs-rebase label Jan 22, 2026

qandrew changed the title ~~[WIP] Parser~~ Initial Implementation of Parser for ResponsesAPI Jan 22, 2026

Andrew Xia added 2 commits January 22, 2026 09:47

remove unneeded functions

4ea1af1

Signed-off-by: Andrew Xia <axia@fb.com>

Merge branch 'main' into parser

3898bcf

Signed-off-by: Andrew Xia <axia@fb.com>

qandrew marked this pull request as ready for review January 22, 2026 17:58

qandrew requested review from aarnphm and chaunceyjiang as code owners January 22, 2026 17:58

mergify bot removed the needs-rebase label Jan 22, 2026

cursor bot reviewed Jan 22, 2026

View reviewed changes

Merge branch 'main' into parser

bdd9c4e

fix?

598e109

Signed-off-by: Andrew Xia <axia@fb.com>

qandrew changed the title ~~Initial Implementation of Parser for ResponsesAPI~~ [1/N] Initial Implementation of Parser for ResponsesAPI Jan 28, 2026

qandrew mentioned this pull request Jan 29, 2026

[2/N] move responses/serving _make_response_output_items logic to parser #33281

Merged

5 tasks

chaunceyjiang reviewed Jan 30, 2026

View reviewed changes

mergify bot added the needs-rebase label Jan 31, 2026

Merge branch 'main' into parser

a06eec4

Signed-off-by: Andrew Xia <axia@fb.com>

mergify bot removed the needs-rebase label Feb 2, 2026

lazy

f203056

Signed-off-by: Andrew Xia <axia@fb.com>

qandrew requested a review from chaunceyjiang February 3, 2026 04:42

chaunceyjiang reviewed Feb 3, 2026

View reviewed changes

chauncey comments

ef12ba1

Signed-off-by: Andrew Xia <axia@fb.com>

luccafong added the ready ONLY add when PR is ready to merge/full CI is needed label Feb 3, 2026

fix

cbfc18c

Signed-off-by: Andrew Xia <axia@fb.com>

pre-commit

28d6194

Signed-off-by: Andrew Xia <axia@fb.com>

qandrew requested review from DarkLight1337, NickLucche and robertgshaw2-redhat as code owners February 3, 2026 23:05

qandrew requested a review from chaunceyjiang February 4, 2026 02:41

chaunceyjiang approved these changes Feb 4, 2026

View reviewed changes

chaunceyjiang reviewed Feb 4, 2026

View reviewed changes

chaunceyjiang merged commit e1bf04b into vllm-project:main Feb 4, 2026
43 of 44 checks passed

ItzDEXX pushed a commit to ItzDEXX/vllm that referenced this pull request Feb 19, 2026

[1/N] Initial Implementation of Parser for ResponsesAPI (vllm-project…

3c249c9

…#32712) Signed-off-by: Andrew Xia <axia@fb.com> Co-authored-by: Andrew Xia <axia@fb.com>

This was referenced Mar 3, 2026

[Responses API] Structured output + reasoning via structural tag embedding #35873

Closed

[Responses API] Structured output + reasoning via structural tag embedding #35904

Open

liuchenbing2026 pushed a commit to liuchenbing2026/vllm that referenced this pull request Apr 4, 2026

[1/N] Initial Implementation of Parser for ResponsesAPI (vllm-project…

0a479f6

…#32712) Signed-off-by: Andrew Xia <axia@fb.com> Co-authored-by: Andrew Xia <axia@fb.com>

		logger = init_logger(__name__)


		@ParserManager.register_module("minimax_m2")

Uh oh!

Conversation

qandrew commented Jan 20, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

mergify bot commented Jan 22, 2026

Uh oh!

mergify bot commented Jan 22, 2026

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Jan 22, 2026

Choose a reason for hiding this comment

Global class attribute mutation causes shared state bug

Uh oh!

cursor bot Jan 22, 2026

Choose a reason for hiding this comment

Strategy 2 silently overrides user's explicit parser choices

Uh oh!

cursor bot Jan 22, 2026

Choose a reason for hiding this comment

Missing null check causes crash with ParsableContext

Uh oh!

mergify bot commented Jan 28, 2026

Uh oh!

mergify bot commented Jan 28, 2026

Uh oh!

chaunceyjiang left a comment

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

qandrew Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

qandrew commented Jan 30, 2026

Uh oh!

mergify bot commented Jan 31, 2026

Uh oh!

chaunceyjiang commented Feb 2, 2026

Uh oh!

qandrew commented Feb 2, 2026

Uh oh!

mergify bot commented Feb 2, 2026

Uh oh!

mergify bot commented Feb 3, 2026

Uh oh!

chaunceyjiang left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chaunceyjiang Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mergify bot commented Feb 3, 2026

Uh oh!

mergify bot commented Feb 3, 2026

Uh oh!

chaunceyjiang left a comment

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Feb 4, 2026

qandrew commented Jan 20, 2026 •

edited by github-actions bot

Loading

chaunceyjiang left a comment •

edited

Loading