FqnToConfig

class torchao.quantization.FqnToConfig(fqn_to_config: ~typing.OrderedDict[str, ~typing.Optional[~torchao.core.config.AOBaseConfig]] = <factory>, module_fqn_to_config: ~typing.OrderedDict[str, ~typing.Optional[~torchao.core.config.AOBaseConfig]] = <factory>, version: int = 1)

Configuration class for applying different quantization configs to modules or parameters based on their fully qualified names (FQNs).

Parameters

• fqn_to_config –

  typing.OrderedDict[str, Optional[AOBaseConfig]]: an ordered dictionary from

  (1). fully qualified name (fqn) of module or parameter (2). regex of fully qualified name (in python re module regex format), should

  start with prefix “re:” or

  (3). “_default”

  to the config that we want to apply to the module/param or None

  Config key ordered by precedence:

  • fully qualified parameter name, e.g. language.layers.0.q_proj.weight

  • fully qualified module name, e.g. language.layers.0.q_proj

  • regex for parameter names, must start with re:, e.g. re:language.layers..+.q_proj.weight. The first regex that matches will be applied.

  • regex for module names, must start with re:, e.g. re:language.layers..+.q_proj, whichever regex fully matches the module fqn first will be applied (order of keys for dictionary are kept consistent since we are using OrderedDict)

  • ”_default”, fallback if no match for all previous keys (Note, when using _default, the config is applied to all modules, to apply

    it to only a subset of modules, e.g. with some types, it’s better to filter the modules that we don’t want to quantize before hand and configure them to None, e.g. {“re:.+norm.+”: None, “_default”: linear_config}) “_default” is not supported when filter_fn is not specified.

• module_fqn_to_config – typing.OrderedDict[str, Optional[AOBaseConfig]]: To maintain BC with ModuleFqnToConfig, to be deprecated later

• version – int: Version of config to use.

Note

• The order of patterns in the OrderedDict may matter as only the first matching pattern is applied

• “_default” is ignored for parameter replacement.