Dynamic Shapes
Recent
Unbacked Dynamic Shapes Shouldn't Be Slower — Now They Aren't
TL;DR – Unbacked dynamic shapes had 2x–20% slowdowns on TorchBench and ~30% regressions on vLLM. We fixed the root causes — now unbacked matches backed across all tested models and configurations. Motivation These regressions were blocking adoption in Frontier workloads like vLLM. Demand for unbacked shapes is growing — just in the past week, multiple users needed them to control recompilations — …
Read more →Reducing Compile-Time Overhead in Unbacked-Symbol-Heavy torch.export Traces
TL;DR – A regression report revealed that exporting a model with many unbacked (data-dependent) symbols took 264s. Profiling showed the latency was dominated by repeated symbolic reasoning in the shape system. A series of targeted, generally applicable optimizations reduced tracing time to 87s (~3x faster). Background A report indicated a severe slowdown when exporting a model that heavily uses …
Read more →Backed to Unbacked: From Guardable to Guardless Shapes in PyTorch
TL;DR – We expect unbacked dynamic shapes to become the dominant shape mechanism for Frontier-style workloads due to their better predictability and controllability. However, some blockers remain for their ideal usage, most notably the performance gap, which is a primary focus for the first half of 2026. Origins Recently, unbacked dynamic shapes have become a hot topic. But many people still …
Read more →Slaying Framework Data-Dependent Errors Dragon 🐉
TL;DR – Framework DDE dragon has been slain! 🐉 We’ve eliminated the vast majority of framework data-dependent errors — reducing user issues by over 85% — and unlocked specialization-free full graph capture that just works. This lays the groundwork for emerging unbacked use cases in vLLM, MoE graphs, and PT2-Frontier. Tackling Data-Dependent Errors Data-dependent errors (DDEs) have long been a …
Read more →Guard-Free Dynamic Shapes
TL;DR – Data-dependent errors (DDEs) are the dominant barrier to exporting models with dynamic shapes. There is widespread consensus that DDEs are a significant issue for export — among the various errors observed, data-dependent errors are the most dominant. We launched an initiative to eliminate them via explicit unbacked semantics — explicitly defining how code should behave when inputs are …
Read more →All Dynamic Shapes Logs
- 2026-03-25 Unbacked Dynamic Shapes Shouldn't Be Slower — Now They Aren't
- 2026-02-27 Reducing Compile-Time Overhead in Unbacked-Symbol-Heavy torch.export Traces
- 2026-01-20 Backed to Unbacked: From Guardable to Guardless Shapes in PyTorch
- 2025-10-29 Slaying Framework Data-Dependent Errors Dragon 🐉
- 2025-07-08 Guard-Free Dynamic Shapes