KernelPreference

class torchao.quantization.quantize_.common.KernelPreference(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Enum for specifying the groups of kernels that’s used for quantization, matrix multiplication or other compute ops for quantized tensor

Examples of how options affects the selected kernels can be found in tensor subclass implementations under torchao/quantization/quantize_/workflows

AUTO = 'auto'

Use torch native quantize and quantized mm kernels

TORCH = 'torch'

Use fbgemm quantize and quantized mm kernels, requires fbgemm_gpu_genai library