IntxFakeQuantizeConfig

class torchao.quantization.qat.IntxFakeQuantizeConfig(dtype: dtype | TorchAODType, granularity: Granularity | str | None = None, mapping_type: MappingType | None = None, scale_precision: dtype = torch.float32, zero_point_precision: dtype = torch.int32, zero_point_domain: ZeroPointDomain = ZeroPointDomain.INT, is_dynamic: bool = True, range_learning: bool = False, eps: float | None = None, *, group_size: int | None = None, is_symmetric: bool | None = None, quant_min: int | None = None, quant_max: int | None = None)

Config for how to fake quantize weights or activations, targeting integer dtypes up to torch.int16. torch.uint16 is not supported natively, but torch.int32 can be used as a carrier by setting quant_min and quant_max.

Parameters:

• dtype – dtype to simulate during fake quantization, e.g. torch.int8.

• granularity –

  granularity of scales and zero points, e.g. PerGroup(32). We also support the following strings:

  1. ’per_token’: equivalent to PerToken()

  2. ’per_channel’: equivalent to PerAxis(0)

  3. ’per_group’: equivalent to PerGroup(group_size), must be combined

     with separate group_size kwarg, Alternatively, just set the group_size kwarg and leave this field empty.

  4. ’per_tensor’: equivalent to PerTensor()

• mapping_type – mapping from floating-point values to integers. Supported values are MappingType.SYMMETRIC, MappingType.SYMMETRIC_NO_CLIPPING_ERR, and MappingType.ASYMMETRIC. Alternatively, set is_symmetric (bool) and leave this field empty.

• scale_precision – scale dtype (default torch.fp32)

• zero_point_precision – zero point dtype (default torch.int32)

• zero_point_domain – whether zero point is in integer (default) or float domain

• is_dynamic – whether to use dynamic (default) or static scale and zero points

• range_learning (prototype) – whether to learn scale and zero points during training (default false), not compatible with is_dynamic.

Keyword Arguments:

• group_size – size of each group in per group fake quantization, can be set instead of granularity

• is_symmetric – whether to use symmetric or asymmetric quantization. Setting this to True selects MappingType.SYMMETRIC. Use mapping_type to select MappingType.SYMMETRIC_NO_CLIPPING_ERR.

• quant_min – optional lower bound for the quantized values. Must be set together with quant_max and is supported only for torch.int32.

• quant_max – optional upper bound for the quantized values. Must be set together with quant_min and is supported only for torch.int32.

Example usage:

# Per token asymmetric quantization
IntxFakeQuantizeConfig(torch.int8, "per_token", is_symmetric=False)
IntxFakeQuantizeConfig(torch.int8, PerToken(), MappingType.ASYMMETRIC)

# Per channel symmetric quantization
IntxFakeQuantizeConfig(torch.int4, "per_channel")
IntxFakeQuantizeConfig(torch.int4, "per_channel", is_symmetric=True)
IntxFakeQuantizeConfig(torch.int4, PerAxis(0), MappingType.SYMMETRIC)

# Per group symmetric quantization
IntxFakeQuantizeConfig(torch.int4, group_size=32)
IntxFakeQuantizeConfig(torch.int4, group_size=32, is_symmetric=True)
IntxFakeQuantizeConfig(torch.int4, "per_group", group_size=32, is_symmetric=True)
IntxFakeQuantizeConfig(torch.int4, PerGroup(32), MappingType.SYMMETRIC)

# Per tensor symmetric quantization
IntxFakeQuantizeConfig(torch.int8, "per_tensor")
IntxFakeQuantizeConfig(torch.int8, PerTensor())

# Per tensor asymmetric quantization with a logical uint16 range
IntxFakeQuantizeConfig(
    torch.int32,
    PerTensor(),
    MappingType.ASYMMETRIC,
    quant_min=0,
    quant_max=2**16 - 1,
)

property group_size: int

If this is per group granularity, return the group size. Otherwise, throw an error.

property is_symmetric: bool

Return True for either symmetric mapping type.

Setting this property to True selects MappingType.SYMMETRIC. Setting it to False selects MappingType.ASYMMETRIC. Set mapping_type directly to choose MappingType.SYMMETRIC_NO_CLIPPING_ERR.