Precision & Quantization

Control numerical precision with FP16, BF16, and mixed-precision autocast, and reduce model size with INT8/FP8/FP4 quantization via ModelOpt.

• Compile Mixed Precision models with Torch-TensorRT
• An example of using Torch-TensorRT Autocast
• Quantization (INT8 / FP8 / FP4)
• Deploy Quantized Models using Torch-TensorRT