Precision & Quantization

Control numerical precision with FP16, BF16, and mixed-precision autocast, and reduce model size with INT8/FP8/FP4 quantization via ModelOpt or TorchAO.

• Compile Mixed Precision models with Torch-TensorRT
• An example of using Torch-TensorRT Autocast
• Quantization (INT8 / FP8 / FP4)
• Deploy Quantized Models using Torch-TensorRT
• Example: ViT FP8 Quantization
• Example: TorchAO FP8 WOQ Linear
• Example: TorchAO Static FP8 Linear
• Example: FLUX.1-dev FP8 WOQ
• Example: TorchAO INT4 WOQ Linear
• Example: FLUX.1-dev INT4 WOQ
• Example: Qwen3-8B INT4 WOQ
• Example: TorchAO NVFP4 WOQ Linear
• Example: FLUX.1-dev NVFP4 WOQ
• Example: TorchAO MXFP4 Linear
• Example: FLUX.1-dev MXFP4