IBM makes Spyre a first-class PyTorch backend

IBM says it has made its Spyre inference accelerator a first-class PyTorch backend through torch-spyre, turning a goal announced last year into a working native-device path. Developers can target Spyre through familiar PyTorch interfaces while keeping model graphs in PyTorch’s compiler stack.

The implementation gives teams a PyTorch-native route to Spyre for AI workloads running alongside applications and data on IBM Z, LinuxONE and Power systems. PyTorch is also gaining a native backend for Google’s TPUs.

IBM said in October 2025 that it planned to deepen Spyre’s PyTorch and vLLM integration during 2026, including first-class backend participation and multi-accelerator scheduling. A March 2026 roadmap scoped work on the compiler, runtime, distributed inference, vLLM and continuous integration to FP16 and FP8 inference. The current update delivers parts of that plan, but IBM has not said every roadmap item is complete.

At the device layer, PyTorch’s technical account says torch-spyre uses the framework’s PrivateUse1 registration mechanism to identify the device as spyre. The installed package is picked up through PyTorch’s backend autoload entry point and exposes a torch.spyre API, according to the project documentation. Together, those hooks let PyTorch recognize Spyre, keep tensors on it and order work through the same kinds of abstractions used for other accelerators.

From there, the compiler route runs through TorchDynamo and TorchInductor. Spyre-specific code handles decompositions, data layouts, tiling, work division and scratchpad allocation before the device compiler generates instructions. IBM’s developers said the Inductor-based compiler is roughly seven times smaller than the separate compiler stack it replaced. That is a vendor-reported code-size comparison, not an independently reproduced performance result.

IBM also cited speed gains from a Granite 3.3 8B demonstration at batch size 1 and sequence length 1,024. The company said moving graph construction off the data-transfer path improved prefill by 9.9% and decode by 13.6%. After further compute-dispatch changes, it reported total gains of 1.7 times for prefill and 2.4 times for decode. The reviewed sources do not include an independent reproduction of those figures.

For model serving, the public Spyre Inference project identifies itself as a vLLM platform plugin that connects to the accelerator through torch-spyre and PyTorch’s Inductor compiler. IBM said a hardware-agnostic path was validated with Gemma 4, Qwen3 and Granite 4.2. It separately claimed that these portable vLLM layers ran within 3.4% of native total-token throughput on NVIDIA H100 GPUs, measured as a geometric mean across three models. The result concerns portability on H100 GPUs; it is not a Spyre-versus-H100 benchmark.

IBM said its engineers run end-to-end models against CPU or GPU references to catch failures involving combinations of operations, shapes and compiler behavior. A PyTorch-hosted account of the testing setup says Torch Spyre reached Level 2 integration with PyTorch’s Cross-Repository CI Relay, which sends upstream changes to downstream accelerator test workflows.

One integration boundary remains unfinished: the PyTorch compiler post says torch-spyre currently attaches its custom pre-scheduling passes through a monkey-patch on Inductor’s GraphLowering.codegen() because the required first-class extension hook is still being discussed upstream.