inference_mode#
- class torch.autograd.grad_mode.inference_mode(mode=True)[source]#
Context manager that enables or disables inference mode.
InferenceMode is analogous to
no_grad
and should be used when you are certain your operations will not interact with autograd (e.g., during data loading or model evaluation). Compared tono_grad
, it removes additional overhead by disabling view tracking and version counter bumps. It is also more restrictive, in that tensors created in this mode cannot be used in computations recorded by autograd.This context manager is thread-local; it does not affect computation in other threads.
Also functions as a decorator.
Note
Inference mode is one of several mechanisms that can locally enable or disable gradients. See Locally disabling gradient computation for a comparison. If avoiding the use of tensors created in inference mode in autograd-tracked regions is difficult, consider benchmarking your code with and without inference mode to weigh the performance benefits against the trade-offs. You can always use
no_grad
instead.Note
Unlike some other mechanisms that locally enable or disable grad, entering inference_mode also disables forward-mode AD.
- Parameters
mode (bool or function) – Either a boolean flag to enable or disable inference mode, or a Python function to decorate with inference mode enabled.
- Example::
>>> import torch >>> x = torch.ones(1, 2, 3, requires_grad=True) >>> with torch.inference_mode(): ... y = x * x >>> y.requires_grad False >>> y._version Traceback (most recent call last): File "<stdin>", line 1, in <module> RuntimeError: Inference tensors do not track version counter. >>> @torch.inference_mode() ... def func(x): ... return x * x >>> out = func(x) >>> out.requires_grad False >>> @torch.inference_mode() ... def doubler(x): ... return x * 2 >>> out = doubler(x) >>> out.requires_grad False