FqnToConfig

class torchao.quantization.FqnToConfig(fqn_to_config: ~typing.OrderedDict[str, ~torchao.core.config.AOBaseConfig | None] = <factory>, module_fqn_to_config: ~typing.OrderedDict[str, ~torchao.core.config.AOBaseConfig | None] = <factory>, version: int = 1)

Configuration class for applying different quantization configs to modules or parameters based on their fully qualified names (FQNs).

Parameters:

• fqn_to_config –

  typing.OrderedDict[str, Optional[AOBaseConfig]]: an ordered dictionary from a key to the config that we want to apply to the module/param (or None). A key is one of:

  1. fully qualified name (fqn) of module or parameter

  2. regex of fully qualified name (in python re module regex format), should start with prefix “re:” or

  3. ”_default”

  Config key ordered by precedence:

  • fully qualified parameter name, e.g. language.layers.0.q_proj.weight

  • fully qualified module name, e.g. language.layers.0.q_proj

  • regex for parameter names, must start with re:, e.g. re:language.layers..+.q_proj.weight. The first regex that matches will be applied.

  • regex for module names, must start with re:, e.g. re:language.layers..+.q_proj, whichever regex fully matches the module fqn first will be applied (order of keys for dictionary are kept consistent since we are using OrderedDict)

  • ”_default”, fallback if no match for all previous keys (Note, when using _default, the config is applied to all modules, to apply it to only a subset of modules, e.g. with some types, it’s better to filter the modules that we don’t want to quantize before hand and configure them to None, e.g. {“re:.+norm.+”: None, “_default”: linear_config}) “_default” is not supported when filter_fn is not specified.

• module_fqn_to_config – typing.OrderedDict[str, Optional[AOBaseConfig]]: To maintain BC with ModuleFqnToConfig, to be deprecated later

• version – int: Version of config to use.

Note

• The order of patterns in the OrderedDict may matter as only the first matching pattern is applied

• “_default” is ignored for parameter replacement.