Workflows#
Created On: Jan 29, 2026 | Last Updated On: Jan 29, 2026
This page provides an overview of the quantization and training workflows available in torchao.
Stable Workflows#
🟢 = stable, 🟡 = prototype, 🟠 = planned, ⚪ = not supported
recommended hardware |
weight |
activation |
quantized training |
QAT |
PTQ data algorithms |
quantized inference |
|---|---|---|---|---|---|---|
H100, B200 GPUs |
float8 rowwise |
float8 rowwise |
🟢 (link) |
🟢 (link) |
⚪ |
🟢 (link) |
Intel® BMG GPUs |
float8 tensor/rowwise |
float8 tensor/rowwise |
🟠 |
🟢 (link) |
⚪ |
🟢 (link) |
H100 GPUs |
int4 |
float8 rowwise |
⚪ |
🟢 (link) |
🟠 |
🟢 (link) |
A100 GPUs |
int4 |
bfloat16 |
⚪ |
🟢 (link) |
🟢 (link) |
|
Intel® BMG GPUs |
int4 |
float16/bfloat16 |
⚪ |
🟢 (link) |
🟡: AWQ |
🟢 (link) |
A100 GPUs |
int8 |
bfloat16 |
⚪ |
🟢 (link) |
⚪ |
🟢 (link) |
A100 GPUs |
int8 |
int8 |
🟡 (link) |
🟢 (link) |
⚪ |
🟢 (link) |
Intel® BMG GPUs |
int8 |
int8 |
🟠 |
🟢 (link) |
⚪ |
🟢 (link) |
edge |
intx (1..7) |
bfloat16 |
⚪ |
🟢 (link) |
⚪ |
🟢 (link) |
Prototype Workflows#
🟢 = stable, 🟡 = prototype, 🟠 = planned, ⚪ = not supported
recommended hardware |
weight |
activation |
quantized training |
QAT |
PTQ data algorithms |
quantized inference |
|---|---|---|---|---|---|---|
B200, MI350x GPUs |
mxfp8 |
mxfp8 |
⚪ |
⚪ |
🟡 (link) |
|
B200 GPUs |
nvfp4 |
nvfp4 |
🟠 |
🟡 (link) |
⚪ |
🟡 (link) |
B200, MI350x GPUs |
mxfp4 |
mxfp4 |
⚪ not supported |
🟠 |
🟠 |
🟡 (link) |
H100 |
float8 128x128 (blockwise) |
float8 1x128 |
🟠 |
⚪ |
⚪ |
🟡 |
Quantization-Aware Training (QAT)#
See the QAT documentation for details on how to use quantization-aware training to improve model accuracy after quantization.
Other#
Sparsity README.md, includes different techniques such as 2:4 sparsity and block sparsity
the prototype folder for other prototype features
