Laith Sakka (@laithsakka) · July 10, 2026
dynamic_shapesfxsymintcorrectness
TL;DR – If you’re creating FX graph nodes with raw SymInt arguments, use Graph.materialize_symints or the targeted create_size_node / create_stride_node / create_storage_offset_node helpers instead of passing the raw symbolic value directly. This fixes a common, subtle class of correctness bugs — we’ve already found 3 in PyTorch Inductor and 6 across executorch’s ARM backend passes. Passing raw symbolic values will become a hard error soon.
This is a pattern that’s easy to write and hard to catch. Today it emits a warning; it will become a hard error soon, once we land the executorch fixes:
# `val` is the example/fake tensor stored on a node's meta — under dynamic # shapes its …
Continue reading →Sanket Purandare (@sanketpurandare) · July 10, 2026
dynamic_shapesunbackeddistributeddtensorflex_attentioninductortracing
TL;DR – To capture a whole distributed training step as one FX graph, PT2 has to trace models whose batch and sequence dimensions are unbacked SymInts. When we tried this on the TorchTitan DeepSeek-V3 MoE trainer, every layer of the stack either silently specialized those dims to concrete ints or blew up with a data-dependent error (DDE). This post walks through the 11 fixes — spanning ATen meta kernels, ProxyTensor, the ShapeEnv, Inductor, collective bucketing, DTensor, and FlexAttention — that make each layer keep tensor semantics symbolic while allowing hints for policy decisions only. That contract is what unblocked end-to-end symbolic tracing for the graph trainer and expert-parallel …
Continue reading →Laith Sakka (@laithsakka), Xiao Fu (@fxdawnn) · June 24, 2026
dynamic_shapesunbackedexport
TL;DR – A new dynamic shapes API is available and ready to use! It provides a unified, consistent way for specifying dynamic specs across compile, export and make_fx, brings native unbacked support to torch.export and make_fx, and completes the unbacked story described earlier by providing unified, predictable, declarative control over the shapes of compiled artifacts.
Consider the example below: the user has a function project(x, w) with a fast path for small batches and a general matmul path. The user wants to compile a dynamic-shape artifact that takes the fast path.
The ShapesSpec says x has dynamic shape [B, D] and w has shape [D, D], and the assumption B < 32 commits this artifact to …
Continue reading →Laith Sakka (@laithsakka) · March 25, 2026
dynamic_shapesunbackedperformancevllmtorchbenchinductor
TL;DR – Unbacked dynamic shapes had 2x–20% slowdowns on TorchBench and ~30% regressions on vLLM. We fixed the root causes — now unbacked matches backed across all tested models and configurations.
These regressions were blocking adoption in Frontier workloads like vLLM. Demand for unbacked shapes is growing — just in the past week, multiple users needed them to control recompilations — so the gap was not acceptable.
We’ve now solved this: unbacked matches backed across all HuggingFace TorchBench models (up to 2x faster) and 30+ vLLM models across multiple configurations.
The key idea behind this work is simple:
For a given graph G and guard set E, unbacked shapes must match the performance …
Continue reading →Laith Sakka (@laithsakka), Aditya Venkataraman (@aditvenk) · February 27, 2026
dynamic_shapesunbackedtorch.exportcompile_timesymbolic_shapes
TL;DR – A regression report revealed that exporting a model with many unbacked (data-dependent) symbols took 264s. Profiling showed the latency was dominated by repeated symbolic reasoning in the shape system. A series of targeted, generally applicable optimizations reduced tracing time to 87s (~3x faster).
A report indicated a severe slowdown when exporting a model that heavily uses data-dependent operations (i.e., unbacked symbolic shapes). Profiling showed that most of the time was spent inside the symbolic shape system.
At the time of investigation, torch.export did not support profiling out of the box, which made root-cause analysis difficult. After enabling profiling for export, a …
Continue reading →