torchao.utils

Tensor Subclass Utils

TorchAOBaseTensor

A util tensor subclass that provides commonly used functions

torchao.quantization.quantize_.common

quantize_ API Common Utils

KernelPreference	Enum for specifying the groups of kernels that's used for quantization, matrix multiplication or other compute ops for quantized tensor
PackingFormat	Packing format for quantized data in Tensor subclasses in torchao, represents how the values are packed and laid out in the quantized data.
QuantizeTensorKwargs	Base class for keyword argument container for quantized tensor creation.
_choose_quant_func_and_quantize_tensor	Given a tensor and a kwargs container, chooses a derived dtype (float8, int8, etc) to quantize tensor to, based on the type of quant_kwargs quantizes tensor to the derived dtype chosen in (1) This is needed to support flexible quantization of activation to various derived dtypes.