Skip to content

[Migration] Sendnn inference rename#946

Merged
joerunde merged 10 commits intotorch-spyre:mainfrom
joerunde:sendnn_inference_rename
Apr 22, 2026
Merged

[Migration] Sendnn inference rename#946
joerunde merged 10 commits intotorch-spyre:mainfrom
joerunde:sendnn_inference_rename

Conversation

@joerunde
Copy link
Copy Markdown
Collaborator

@joerunde joerunde commented Apr 21, 2026

Description

This PR updates the vllm_spyre package to sendnn_inference.

Docs and publication workflows have been updated in preparation to publish this as sendnn-inference==2.0.0

❗❗❗

Breaking configuration changes!

  • The plugin name for VLLM_PLUGINS is now sendnn_inference
  • All config options are now SENDNN_INFERENCE_*
  • The precompiled model parser now expects sendnn_inference_version

Related Issues

Closes torch-spyre/spyre-inference#8

Test Plan

  • CPU unit tests
  • Spyre unit tests
  • Spot checks with full model integration tests
  • Ensure Test PyPI deploy works

Checklist

  • I have read the contributing guidelines
  • My code follows the project's code style (run bash format.sh)
  • I have added tests for my changes (if applicable)
  • I have updated the documentation (if applicable)
  • My commits include a Signed-off-by: line (DCO compliance)

Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <joe@joerun.de>
@github-actions
Copy link
Copy Markdown

👋 Hi! Thank you for contributing.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, run ./format.sh.
Now you are good to go 🚀.

We also recommend installing prek and configuring it to check your code before every local commit.

Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <joe@joerun.de>
Signed-off-by: Joe Runde <joe@joerun.de>
@joerunde
Copy link
Copy Markdown
Collaborator Author

If we want to keep pushing docker images (that nobody uses?) then we'll also need to update our quay configs. I've set things up so that they could run, but disabled the build for now.

PyPI and Test PyPI projects have been updates to publish the new sendnn-inference project once this PR merges

@joerunde
Copy link
Copy Markdown
Collaborator Author

bot:test
MARKERS="spyre and prefix_caching and not quantized and not multi"

Signed-off-by: Joe Runde <joe@joerun.de>
@joerunde joerunde mentioned this pull request Apr 21, 2026
5 tasks
@joerunde
Copy link
Copy Markdown
Collaborator Author

bot:test
MARKERS="spyre and chunked_prefill"

@Nhan-Hoang
Copy link
Copy Markdown
Collaborator

@joerunde yes I think it is overlapping. We can close this as dup. I actually had this ready last week just for the documentation but fail to submit.

@sducouedic
Copy link
Copy Markdown
Collaborator

sducouedic commented Apr 22, 2026

I think these two changes are still missing:

  • _local_envs_for_test.sh line 10 VLLM_SPYRE_DYNAMO_BACKEND
  • .yapfignore: line 1 vllm_spyre/model_executor/...

Copy link
Copy Markdown
Collaborator

@rafvasq rafvasq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming docker/Dockerfile.amd64 is being left alone for now, a few other references to "vllm spyre" include:

@tjohnson31415
Copy link
Copy Markdown
Collaborator

tjohnson31415 commented Apr 22, 2026

Adding some of my searching as well, with commands (overlaps with above comments except for some more changes in .github):

# General search (excluding dot files/dirs)
# excluding docs since that intends to use old names in some places
$ grep -ri 'vllm.spyre' * | grep -v -e '.egg-info' -e '__pycache__' -e 'Binary file' -e 'docs/' -e 'docker/'
_local_envs_for_test.sh:export VLLM_SPYRE_DYNAMO_BACKEND=eager
CONTRIBUTING.md:# Contributing to vLLM Spyre
CONTRIBUTING.md:For details on contributing to vLLM-Spyre, see the **[contributing guide](https://docs.vllm.ai/projects/spyre/en/latest/contributing)**.
examples/online_inference/spyre_vllm_benchmark.py:    parser = argparse.ArgumentParser(description="VLLM Spyre inference benchmarking script.")
tests/aftu/test_compare_graphs.py:"""Compare graphs generated by vLLM-Spyre vs AFTU.
tests/aftu/test_compare_graphs.py:This test compares computation graphs generated by vLLM-Spyre against those
tests/aftu/graph_compare_utils.py:"""Utilities for comparing computation graphs between vLLM-Spyre and AFTU.
tests/e2e/test_spyre_mm.py:    # and vllm spyre running with the eager backend.

# Search dotfiles
$ grep -d skip -i 'vllm.spyre' .*
.yapfignore:vllm_spyre/model_executor/model_loader/spyre_setup.py

# Recurse into specific . directories (avoid .git, .venv, etc)
$ grep -ir 'vllm.spyre' .github/ .vscode/
.github/CODEOWNERS:# This lists cover the "core" components of vLLM-Spyre that require careful review
.github/ISSUE_TEMPLATE/config.yml:    about: Read the vLLM Spyre documentation
.github/ISSUE_TEMPLATE/config.yml:    about: Learn how to contribute to vLLM Spyre
.github/ISSUE_TEMPLATE/feature-request.yml:      Why is this feature important? How would it benefit vLLM Spyre users?

Signed-off-by: Joe Runde <joe@joerun.de>
@joerunde
Copy link
Copy Markdown
Collaborator Author

Travis' commands are now clean- y'all think this is good to go?

Copy link
Copy Markdown
Collaborator

@sducouedic sducouedic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@joerunde joerunde merged commit 105dead into torch-spyre:main Apr 22, 2026
12 checks passed
@joerunde
Copy link
Copy Markdown
Collaborator Author

good luck us!

@joerunde
Copy link
Copy Markdown
Collaborator Author

test pypi deployment was successful! 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Migration] Rename vllm-spyre to sendnn-inference

5 participants