At the end of a run, EngineCore exits with:
RuntimeError: Cannot access accelerator device when none is available.
from cleanup_dist_env_and_memory() in vllm/distributed/parallel_state.py:2088. Generation succeeds — this only fires during shutdown — but the engine process exits non-zero.
vLLM gates the cleanup with if not current_platform.is_cpu(). TorchSpyrePlatform inherits from CpuPlatform but sets _enum = PlatformEnum.OOT (needed so custom ops dispatch via forward_oot — see spyre_worker.py:49-57), so is_cpu() returns False and vLLM falls through to torch.accelerator.empty_cache(), which has no real accelerator to talk to (torch-spyre registers as PrivateUse1, not through torch.accelerator).
Repro: python torch_spyre_inference.py -n 1.
At the end of a run,
EngineCoreexits with:from
cleanup_dist_env_and_memory()invllm/distributed/parallel_state.py:2088. Generation succeeds — this only fires during shutdown — but the engine process exits non-zero.vLLM gates the cleanup with
if not current_platform.is_cpu().TorchSpyrePlatforminherits fromCpuPlatformbut sets_enum = PlatformEnum.OOT(needed so custom ops dispatch viaforward_oot— seespyre_worker.py:49-57), sois_cpu()returnsFalseand vLLM falls through totorch.accelerator.empty_cache(), which has no real accelerator to talk to (torch-spyre registers asPrivateUse1, not throughtorch.accelerator).Repro:
python torch_spyre_inference.py -n 1.