Laith Sakka (@laithsakka) · March 25, 2026
dynamic_shapesunbackedperformancevllmtorchbenchinductor
TL;DR – Unbacked dynamic shapes had 2x–20% slowdowns on TorchBench and ~30% regressions on vLLM. We fixed the root causes — now unbacked matches backed across all tested models and configurations.
These regressions were blocking adoption in Frontier workloads like vLLM. Demand for unbacked shapes is growing — just in the past week, multiple users needed them to control recompilations — so the gap was not acceptable.
We’ve now solved this: unbacked matches backed across all HuggingFace TorchBench models (up to 2x faster) and 30+ vLLM models across multiple configurations.
The key idea behind this work is simple:
For a given graph G and guard set E, unbacked shapes must match the performance …
Continue reading →Laith Sakka (@laithsakka), Aditya Venkataraman (@aditvenk) · February 27, 2026
dynamic_shapesunbackedtorch.exportcompile_timesymbolic_shapes
TL;DR – A regression report revealed that exporting a model with many unbacked (data-dependent) symbols took 264s. Profiling showed the latency was dominated by repeated symbolic reasoning in the shape system. A series of targeted, generally applicable optimizations reduced tracing time to 87s (~3x faster).
A report indicated a severe slowdown when exporting a model that heavily uses data-dependent operations (i.e., unbacked symbolic shapes). Profiling showed that most of the time was spent inside the symbolic shape system.
At the time of investigation, torch.export did not support profiling out of the box, which made root-cause analysis difficult. After enabling profiling for export, a …
Continue reading →Laith Sakka (@laithsakka), Aditya Venkataraman (@aditvenk), Bob Ren (@bobrenjc93) · January 20, 2026
dynamic_shapesunbackedbackedtorch.compilefrontier
TL;DR – We expect unbacked dynamic shapes to become the dominant shape mechanism for Frontier-style workloads due to their better predictability and controllability. However, some blockers remain for their ideal usage, most notably the performance gap, which is a primary focus for the first half of 2026.
Recently, unbacked dynamic shapes have become a hot topic. But many people still don’t fully understand (1) what backed vs unbacked dynamic shapes actually are, and (2) why that choice matters for performance, UX, and Frontier.
In this post, I’ll walk through a simplified story of how we got here, why unbacked shapes are becoming more important, and what’s still blocking them. This post is …
Continue reading →Laith Sakka (@laithsakka) · October 29, 2025
dynamic_shapesunbackedddeexportguard_freetorch.compile
TL;DR – Framework DDE dragon has been slain! 🐉 We’ve eliminated the vast majority of framework data-dependent errors — reducing user issues by over 85% — and unlocked specialization-free full graph capture that just works. This lays the groundwork for emerging unbacked use cases in vLLM, MoE graphs, and PT2-Frontier.
Data-dependent errors (DDEs) have long been a major pain point for framework export users, as detailed in the previous post. Six months ago, we launched an initiative to eliminate these issues by implementing explicit unbacked semantics — explicitly defining how code should behave when inputs are unbacked.
That work is now complete. We’ve moved into the maintenance phase, and …
Continue reading →Laith Sakka (@laithsakka), Brian Hirsh (@bdhirsh), Angela Yi (@angelayi), Colin Peppler (@colinpeppler), Bob Ren (@bobrenjc93), Avik Chaudhuri (@avikchaudhuri), Aaron Orenstein (@aorenste), Pian Pawakapan (@pianpwk) · July 8, 2025
dynamic_shapesunbackedddeguard_freeexporttorch.compile
TL;DR – Data-dependent errors (DDEs) are the dominant barrier to exporting models with dynamic shapes. There is widespread consensus that DDEs are a significant issue for export — among the various errors observed, data-dependent errors are the most dominant. We launched an initiative to eliminate them via explicit unbacked semantics — explicitly defining how code should behave when inputs are unbacked. In the first half, we generalized more than 60% of the relevant code, achieving a 55% reduction in framework DDEs and saving weeks of engineering time on complex model exports.
When exporting models, graph breaks aren’t allowed. Therefore, when a non-guardable dynamic size is introduced …
Continue reading →