# torch.compiler

`torch.compiler` is a namespace through which some of the internal compiler
methods are surfaced for user consumption. The main function and the feature in
this namespace is `torch.compile`.

`torch.compile` is a PyTorch function introduced in PyTorch 2.x that aims to
solve the problem of accurate graph capturing in PyTorch and ultimately enable
software engineers to run their PyTorch programs faster. `torch.compile` is
written in Python and it marks the transition of PyTorch from C++ to Python.

`torch.compile` leverages the following underlying technologies:

- **TorchDynamo (torch._dynamo)** is an internal API that uses a CPython
feature called the Frame Evaluation API to safely capture PyTorch graphs.
Methods that are available externally for PyTorch users are surfaced
through the `torch.compiler` namespace.
- **TorchInductor** is the default `torch.compile` deep learning compiler
that generates fast code for multiple accelerators and backends. You
need to use a backend compiler to make speedups through `torch.compile`
possible. For NVIDIA, AMD and Intel GPUs, it leverages OpenAI Triton as the key
building block.
- **AOT Autograd** captures not only the user-level code, but also backpropagation,
which results in capturing the backwards pass "ahead-of-time". This enables
acceleration of both forwards and backwards pass using TorchInductor.

To better understand how `torch.compile` tracing behavior on your code, or to
learn more about the internals of `torch.compile`, please refer to the [`torch.compile` programming model](compile/programming_model.html).

Note

In some cases, the terms `torch.compile`, TorchDynamo, `torch.compiler`
might be used interchangeably in this documentation.

`torch.compiler` also includes a prototype ahead-of-time API, `torch.compiler.precompile`.
It captures a whole computation - `fn(model, x)`, with the model(s) passed as arguments -
from the caller's own calls and lowers it to a self-contained, runnable Python source
artifact plus an acceleration cache that a fresh process reloads. No weights are baked in,
so you pass the model again at runtime:

```
with torch.compiler.precompile.capture(
 lambda model, x: model(x), artifact_path="m.py", cache_path="m.cache"
) as cap:
 y = cap(model, x)

f = torch.compiler.precompile.load("m.py", "m.cache")
y = f(model, x)
```

See the [API reference](../../torch.compiler_api.html#torch-compiler-api) for details.

Warning

`torch.compile` may not support recently released major versions of Python.

If you attempt to use `@torch.compile` in an unsupported Python
environment, you may encounter an error similar to:

```
RuntimeError: torch.compile is not supported on Python 3.xx.0+
```

Please ensure that your current Python version is within the range
supported by PyTorch for `torch.compile`.

If you have installed PyTorch on a Python version that is too new,
you will need to switch to an earlier Python version in order to use `torch.compile`.

As mentioned above, to run your workflows faster, `torch.compile` through
TorchDynamo requires a backend that converts the captured graphs into a fast
machine code. Different backends can result in various optimization gains.
The default backend is called TorchInductor, also known as *inductor*,
TorchDynamo has a list of supported backends developed by our partners,
which can be seen by running `torch.compiler.list_backends()` each of which
with its optional dependencies.

Some of the most commonly used backends include:

**Training & inference backends**

| Backend | Description |
| --- | --- |
| `torch.compile(m, backend="inductor")` | Uses the TorchInductor backend. [Read more](https://dev-discuss.pytorch.org/t/torchinductor-a-pytorch-native-compiler-with-define-by-run-ir-and-symbolic-shapes/747) |
| `torch.compile(m, backend="cudagraphs")` | CUDA graphs with AOT Autograd. [Read more](https://pytorch.org/docs/stable/torch.compiler_cudagraph_trees.html) |
| `torch.compile(m, backend="ipex")` | Uses IPEX on CPU. [Read more](https://github.com/intel/intel-extension-for-pytorch) |

**Inference-only backends**

| Backend | Description |
| --- | --- |
| `torch.compile(m, backend="tensorrt")` | Uses Torch-TensorRT for inference optimizations. Requires `import torch_tensorrt` in the calling script to register backend. [Read more](https://github.com/pytorch/TensorRT) |
| `torch.compile(m, backend="ipex")` | Uses IPEX for inference on CPU. [Read more](https://github.com/intel/intel-extension-for-pytorch) |
| `torch.compile(m, backend="tvm")` | Uses Apache TVM for inference optimizations. [Read more](https://tvm.apache.org/) |
| `torch.compile(m, backend="openvino")` | Uses OpenVINO for inference optimizations. [Read more](https://docs.openvino.ai/torchcompile) |