[Docs] add docs for cuda graph v1

[fhl2000]

Purpose

Add design documents for the changes in #20059.

Previews:
https://vllm--24374.org.readthedocs.build/en/24374/design/cuda_graphs.html
https://vllm--24374.org.readthedocs.build/en/24374/design/torch_compile.html

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request adds a new design document for CUDA Graph v1, which is a great addition for understanding the new features. The document is comprehensive and well-structured. My main feedback is on the language quality of the new document, which has numerous typos and grammatical errors. I've left a single comment with a list of examples. Fixing these will significantly improve the document's quality. The update to torch_compile.md to link to this new document is appropriate.

[hmellor]

Thank you for the PR! I've not looked closely at the content of the document yet but I have left some high level comments that can be actioned in the meantime.

[hmellor]

With V0 effectively gone, we can drop V1 from the title

[hmellor]

• Could you remove the Pre: and Now: labels baked into the images and use Before: and After: in the markdown content to introduce the images?
• Could you also not truncate PIECEWISE to PIECE.? IKt's not explained anywhere and only saves 3 characters, it's better to be explicit.

[fhl2000]

Why did the pre-commit fail? @hmellor

Unordered list style [Expected: dash; Actual: asterisk]

I remember previously it asked for asterisk instead of dash.

[hmellor]

Why did the pre-commit fail? @hmellor

Unordered list style [Expected: dash; Actual: asterisk]

I remember previously it asked for asterisk instead of dash.

Sorry this was my mistake. pre-commit wants the style to be the same throughout the document and it chooses whichever is used first. In my formatting change I added an unordered list and used a different symbol to the one you used

[fhl2000]

Is this ok? The truncated PIECE. in some places are kept for cleanliness (not much room to add)

[hmellor]

Is this ok? The truncated PIECE. in some places are kept for cleanliness (not much room to add)

These new diagrams do look very nice. It does look like PIECEWISE should fit in the CUDAGraphWrapper boxes. For the data flows at the bottom could the PIECEWISE be expanded to the left?

[fhl2000]

Any thoughts on this new pre-commit error?

Error: docs/design/cuda_graphs.md:63:12 MD026/no-trailing-punctuation Trailing punctuation in heading [Punctuation: ':']

Maybe just remove : or highlight it but not a heading?

--
Edited: it's fixed.

[fhl2000]

The notes for attention ops fusion is changed, since #24281 merged!

[fhl2000]

Have had some updates since #23046 and #25444 were merged.

[simon-mo]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Breezy!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting

• @codex fix this CI failure
• @codex address that feedback

My formatting requests have been addressed

[ProExpertProg]

Thanks for this increadible writeup! A few thoughts & suggestions below

[ProExpertProg]

Great work, thanks for writing this up!