Databricks details Proteus harness for shape-specialized GPU kernels

Databricks detailed Proteus, a harness that uses agents to draft shape-specialized GPU kernels with an outer evaluation loop controlling correctness checks and performance measurements. The company said selected Proteus-generated kernels for parts of the Qwen3.5 122B Gated DeltaNet path ran 1.8 to 5.2 times faster than the best implementations it referenced in vLLM.

Those figures are Databricks benchmark claims, not independently reproduced results. They cover individual kernels, not a full model-specific serving engine or end-to-end model throughput.

In the Proteus workflow, an agent proposes kernel code but does not decide whether the code is correct or fast. The outer loop supplies scoped memory and evaluation feedback, checks outputs against a controlled reference implementation and only lets verified candidates advance to benchmarking. Databricks said those checks run on real GPUs, in isolation and more than once.

Winning candidates are timed again before they can become parents for later rounds. Databricks said the harness can cross-check CUDA event, wall-clock and CUPTI timing, clear compiled state that should not survive between tests, and keep setup and teardown ordering consistent. It also keeps some tests hidden from the candidate and flags speedups that exceed physical bandwidth or compute limits. Databricks cited gains above 100 times as the kind of result that demands scrutiny.

The controls target several benchmark failure modes identified by Databricks: stale compiled code, comparisons in which only one side uses CUDA graphs, and overfitting to visible input sizes. In the company’s packed-decode case study, the loop validated the task contract, measured the reference, generated candidates, ran static checks and builds, checked correctness, benchmarked verified candidates and then remeasured the leading results.

Databricks reported a 0.025-millisecond reference measurement and a lowest candidate measurement of 0.018 milliseconds in that trace. It separately reported a best shape-specific speedup of 1.6 times for the serving-decode path. The winning serving fragment was specialized for a batch-4, key-128, value-128 layout and processed the value dimension in chunks of 64, so Databricks presented it as safe for that shape rather than as a universal replacement.

Proteus also limits what historical information reaches the kernel-writing agent. Databricks said the prompt receives concise, high-trust lessons and failure notes from closely related parent runs, selected through hierarchical tag filters and a combination of keyword and semantic search. Background jobs handle the more expensive work of reorganizing and distilling accumulated lessons, avoiding multi-step retrieval on every generation attempt.

Qwen’s model card identifies Qwen3.5-122B-A10B as a 122-billion-parameter model with 10 billion activated parameters and a hybrid architecture that includes Gated DeltaNet blocks. It lists vLLM as a supported serving route. The vLLM project’s performance note also describes Qwen3.5 as mixing full-attention and Gated Delta Network layers.

Databricks said its packed-decode case study used a Triton backend on NVIDIA B200 GPUs; NVIDIA’s DGX B200 guide documents B200 Tensor Core GPUs in that system. The Databricks post does not include a reproducible benchmark package or raw result set, identify the exact vLLM commit and full test matrix behind its headline range, or say whether Proteus or its generated kernels will be released publicly.