KernelPreference

class torchao.quantization.quantize_.common.KernelPreference(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Enum for specifying the groups of kernels that’s used for quantization, matrix multiplication or other compute ops for quantized tensor

Examples of how options affects the selected kernels can be found in tensor subclass implementations under torchao/quantization/quantize_/workflows

AUTO = 'auto'

Use the most efficient quantize and mm kernels chosen for user based on hardware and library availabilities and versions etc.

DEEPGEMM = 'deepgemm'

Use kernels from the optional DeepGEMM library.

EMULATED = 'emulated'

Emulates gemm_lowp(A, B) with gemm_fp32(A.dequantize(), B.dequantize()). Intended use cases are: 1. Running CI for product logic on hardware which does not support the

actual lowp gemm.

2. Debugging kernel numerics issues.

MSLK = 'mslk'

Use quantize and quantized mm kernels from mslk library, requires mslk library

TORCH = 'torch'

Use torch native quantize and quantized mm kernels

TRITON = 'triton'

Use pure-triton RHT + stochastic rounding kernels already in TorchAO. Full NVFP4 training recipe: RHT on forward activations, stochastic rounding + RHT on backward gradients. Requires bfloat16 input; M, K, N all divisible by 128.