Morning Edition · Wednesday, October 21, 2026Published at 2:08 AM EDT · New York

PolyGraph Replaces 64 CUDA Graphs With One for Adaptive Speculative Decoding as SonicBoom Ships a Python-Free Inference Runtime

On an H800, the PolyGraph prototype matched baseline decode latency while cutting graph-build time from 3,915 to 93 milliseconds, and HardenedVault's SonicBoom removes Python from both the build and the runtime to shrink the attack surface.

Adaptive speculative decoding conflicts with CUDA Graphs. The adaptive scheduler changes the active batch size B and the draft length d at every step, but a captured graph replays one fixed launch configuration. The verification width M = B…