torch.nn.utils.get_total_norm#
- torch.nn.utils.get_total_norm(tensors, norm_type=2.0, error_if_nonfinite=False, foreach=None, dtype=None)[source]#
Compute the norm of an iterable of tensors.
The norm is computed over the norms of the individual tensors, as if the norms of the individual tensors were concatenated into a single vector.
- Parameters:
tensors (Iterable[Tensor] or Tensor) – an iterable of Tensors or a single Tensor that will be normalized
norm_type (float) – type of the used p-norm. Can be
'inf'for infinity norm.error_if_nonfinite (bool) – if True, an error is thrown if the total norm of
tensorsisnan,inf, or-inf. Default:Falseforeach (bool) – use the faster foreach-based implementation. If
None, use the foreach implementation for CUDA and CPU native tensors and silently fall back to the slow implementation for other device types. Default:Nonedtype (torch.dtype, optional) – if set, the per-tensor norms and the norm-of-norms are accumulated in this dtype and the result is returned in it. Otherwise, inputs accumulate in their own dtype by default, which may be undesirable for lower precision inputs (e.g.
bfloat16). It must be a floating point dtype that the inputs promote to (torch.promote_types(input.dtype, dtype) == dtype) which is a constraint of passingdtypetotorch.linalg.vector_norm(). Default:None.
- Returns:
Total norm of the tensors (viewed as a single vector).
- Return type: