docs: restructure KVBM documentation into three-tier format

[akshatha-k]

Summary

• Migrate KVBM documentation to a new three-tier structure:

  • Tier 1: README.md (Quick Start) - overview, link to user guide, feature matrix, architecture
  • Tier 2: kvbm_guide.md (Guide) - installation, configuration, deployment for all user paths (pip wheel, via trtllm/vllm or via the dynamo integrations with other kv offloading solutions)
  • Tier 3: kvbm_design.md (Design) - architecture deep dive, components, data flows, framework integrations

• Create integrations folder with:

  • flexkv_integration.md - new FlexKV integration guide from PR feat: FlexKV integration in Dynamo #5858
  • lmcache_integration.md - migrated from backends/vllm/
  • sglang_hicache.md - migrated from backends/sglang/

• Add AGENTS.md for KVBM component to guide AI agents

• Update docs/index.rst to add "KV Cache Offloading" as first item under User Guides

Test plan

• Verify docs website renders correctly with new structure
• Verify all links work (internal cross-references and external links)
• Verify all diagrams render properly
• Verify KVBM usage guides are correct and up to date

Summary by CodeRabbit

Release Notes

• Documentation
  • Reorganized KV Cache Offloading (KVBM) documentation with a new user guide, design documentation, and deployment guidance.
  • Added comprehensive FlexKV integration documentation.
  • Enhanced SGLang Hierarchical Cache documentation with configuration, deployment, benchmarking, and troubleshooting guidance.
  • Restructured documentation navigation for improved accessibility and clarity.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Walkthrough

This pull request restructures KVBM documentation by consolidating scattered backend-specific guides into unified integration documentation, removing fragmented setup files, and introducing comprehensive design and operational guides. Updates navigation structure via toctree changes to reflect the new documentation organization.

Changes

Cohort / File(s)	Summary
KVBM Documentation Consolidation docs/kvbm/kvbm_architecture.md, docs/kvbm/kvbm_components.md, docs/kvbm/kvbm_design_deepdive.md, docs/kvbm/kvbm_integrations.md, docs/kvbm/kvbm_motivation.md, docs/kvbm/kvbm_reading.md, docs/kvbm/trtllm-setup.md, docs/kvbm/vllm-setup.md	Removes 8 documentation files (894 lines total), including fragmented architecture, components, design deep-dive, integration guides, and framework-specific setup instructions.
KVBM Documentation Consolidation docs/kvbm/README.md, docs/kvbm/kvbm_design.md, docs/kvbm/kvbm_guide.md	Adds 3 new files (850 lines total) providing unified KVBM overview, design documentation, and operational guide for deployment and configuration.
Integration Documentation docs/integrations/flexkv_integration.md, docs/integrations/sglang_hicache.md	Adds 2 comprehensive integration guides (692 lines total) for FlexKV and SGLang HiCache, consolidating previously scattered backend-specific examples into integration-focused documentation.
Documentation Navigation docs/index.rst, docs/kvbm/kvbm_intro.rst, docs/backends/sglang/sgl-hicache-example.md	Updates toctree structure and navigation references to reflect new documentation organization; removes SGLang HiCache example from backends directory (65 lines).
Content Refinements docs/integrations/lmcache_integration.md	Minor textual updates, punctuation corrections, and list reflowing without altering functionality or configuration guidance (+35/−17 lines).
AI Agent Documentation lib/bindings/kvbm/AGENTS.md	Adds new comprehensive guide (288 lines) for AI agents, detailing KVBM architecture, usage patterns, key commands, and integration points.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A burrow of docs, once scattered and wide,
Now nestled together, organized with pride!
New guides like carrots, so golden and clear,
For FlexKV and HiCache to users so dear.
The old paths are pruned, the new ones take flight,
Architecture and guides, all gleaming so bright! ✨

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes the main change: restructuring KVBM documentation into a three-tier format, which is the primary objective of this PR.
Description check	✅ Passed	The description covers all required template sections with substantive details: Overview (summary of three-tier structure), Details (specific file changes and additions), implied reviewer guidance, and test plan checklist.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

docs/kvbm/kvbm_intro.rst (1)

23-23: ⚠️ Potential issue | 🟡 Minor

Tighten spacing/wording in the memory list item.

“GPU memory(in future) ,” reads as a typo. Consider “GPU memory (future)” and remove the extra space before the comma.

🤖 Fix all issues with AI agents

In `@docs/integrations/flexkv_integration.md`:
- Around line 70-74: The docs reference three missing launch
scripts—agg_flexkv.sh, agg_flexkv_router.sh, and disagg_flexkv.sh—so either add
those scripts to the launch scripts set with the expected
aggregator/router/disaggregator start logic (matching the style and env var
usage of the other launch scripts) or update the documentation to point to the
actual script names or remove the calls; make the change by adding the three
shell scripts with clear usage/help text or editing the doc examples to use the
correct existing script names so the examples run as shown.

In `@docs/integrations/sglang_hicache.md`:
- Line 1: Remove the trailing whitespace at the end of line 1 in
docs/integrations/sglang_hicache.md (the empty/HTML comment line at the top),
save the file, and commit the change so the pre-commit hook and pipeline no
longer fail.
- Line 445: Update the broken markdown link in
docs/integrations/sglang_hicache.md by replacing the removed target
"../kvbm/kvbm_architecture.md" with the new file "../kvbm/kvbm_design.md" so the
"**[KVBM Architecture](... )**" link points to the existing kvbm_design.md;
ensure the link text ("KVBM Architecture") remains the same and the relative
path is correct.
- Around line 122-124: Update the fenced code block that contains the formula
"Host KV Cache Size = Device KV Cache Size × hicache-ratio" to include a
language specifier (use "text") after the opening triple backticks so the block
is rendered correctly; locate the code block in
docs/integrations/sglang_hicache.md by searching for that exact formula and
change the opening fence from ``` to ```text.

In `@docs/kvbm/kvbm_design.md`:
- Around line 103-105: The fenced code block containing the formula
"block_stride_in_bytes = align_up(num_layers × layer_stride, alignment);" should
include a language specifier (e.g., text) so renderers treat it as plain
text/pseudocode; update the triple-backtick fence to include `text` (or convert
the single-line formula to inline code) around the expression involving
block_stride_in_bytes, align_up, num_layers, layer_stride, and alignment.

In `@docs/kvbm/kvbm_guide.md`:
- Line 1: Remove the trailing whitespace found on line 1 of the
docs/kvbm/kvbm_guide.md file: edit the file to delete the extra space at the end
of the first line (so it ends with the last visible character, not a space),
save and recommit so the pre-commit hook can pass; no code changes required
beyond trimming that whitespace.
- Around line 46-50: Update the broken link to the KVBM bindings README by
replacing the lowercase filename reference "readme.md" with the actual uppercase
filename "README.md" in the link text that points to the "KVBM bindings README"
(the line containing "To build KVBM from source, see the detailed instructions
in the [KVBM bindings
README](../../lib/bindings/kvbm/readme.md#build-from-source)"). Ensure the
Markdown link now uses "README.md" so it works on case-sensitive filesystems.

[dagil-nvidia]

/ok to test ce8103b

[dagil-nvidia]

/ok to test a1405da

[dagil-nvidia]

/ok to test 130ed97

[dagil-nvidia]

/ok to test 9edc4ba

[nv-kmcgill53]

I tried to inline everything the best I could. I approve of the new layout, just some changes on the content

[dagil-nvidia]

/ok to test cf7598a

[dagil-nvidia]

/ok to test e86c9ed