---
myst:
  html_meta:
    description: Frequently asked questions about the PyTorch C++ API and libtorch.
    keywords: PyTorch, C++, FAQ, libtorch, troubleshooting
---

# FAQ

Listed below are a number of common issues users face with the various parts of
the C++ API.

## C++ Extensions

### Undefined symbol errors from PyTorch/ATen

**Problem**: You import your extension and get an `ImportError` stating that
some C++ symbol from PyTorch or ATen is undefined. For example:

```cpp
>>> import extension
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/user/.pyenv/versions/3.7.1/lib/python3.7/site-packages/extension.cpython-37m-x86_64-linux-gnu.so: undefined symbol: _ZN2at19UndefinedTensorImpl10_singletonE
```

**Fix**: The fix is to `import torch` before you import your extension. This will make
the symbols from the PyTorch dynamic (shared) library that your extension
depends on available, allowing them to be resolved once you import your extension.

### I created a tensor using a function from `at::` and get errors

**Problem**: You created a tensor using e.g. `at::ones` or `at::randn` or
any other tensor factory from the `at::` namespace and are getting errors.

**Fix**: Replace `at::` with `torch::` for factory function calls. You
should never use factory functions from the `at::` namespace, as they will
create tensors. The corresponding `torch::` functions will create variables,
and you should only ever deal with variables in your code.

## LibTorch

### How do I move a model to GPU?

**Problem**: You want to run your model on GPU but are unsure how to move both
the model and tensors to the correct device.

**Fix**: Use the `to()` method to move your model and tensors to a CUDA device:

```cpp
torch::Device device(torch::kCUDA);
model->to(device);
auto input = torch::randn({1, 3, 224, 224}).to(device);
auto output = model->forward(input);
```

You can also check for CUDA availability before moving:

```cpp
torch::Device device(torch::cuda::is_available() ? torch::kCUDA : torch::kCPU);
```

Make sure to compile with the TorchScript headers by including `<torch/script.h>`.

### My model runs slower in C++ than in Python

**Problem**: Your model inference is slower in C++ compared to Python.

**Fix**: There are several common causes:

1. **Enable inference mode**: Wrap your inference code with `torch::NoGradGuard`
   to disable gradient computation:

```cpp
torch::NoGradGuard no_grad;
auto output = model->forward(input);
```

2. **Enable optimizations**: For TorchScript models, use `optimize_for_inference`:

```cpp
module = torch::jit::optimize_for_inference(module);
```

3. **Warm up the model**: Run a few inference passes before benchmarking to allow
   JIT compilation and memory allocation to complete.

4. **Check thread settings**: Ensure proper thread configuration:

```cpp
at::set_num_threads(4);  // Adjust based on your hardware
```

## Neural Network Modules

### How do I register submodules in a custom module?

**Problem**: You created a custom module but the submodules are not being
recognized during `forward()` or when saving/loading the model.

**Fix**: You must register submodules in the constructor using
`register_module()`:

```cpp
struct MyModel : torch::nn::Module {
  MyModel() {
    fc1 = register_module("fc1", torch::nn::Linear(784, 128));
    fc2 = register_module("fc2", torch::nn::Linear(128, 10));
  }

  torch::Tensor forward(torch::Tensor x) {
    x = torch::relu(fc1->forward(x));
    return fc2->forward(x);
  }

  torch::nn::Linear fc1{nullptr}, fc2{nullptr};
};
```

### How do I set a module to evaluation mode?

**Problem**: Layers like Dropout and BatchNorm behave differently during training
and evaluation, and you need to switch between modes.

**Fix**: Use the `eval()` and `train()` methods:

```cpp
model->eval();  // Set to evaluation mode
// ... run inference ...
model->train(); // Set back to training mode
```

## Data Loading

### How do I create a custom dataset?

**Problem**: You want to load your own data instead of using built-in datasets.

**Fix**: Create a class that inherits from `torch::data::datasets::Dataset` and
implement the `get()` and `size()` methods:

```cpp
class CustomDataset : public torch::data::datasets::Dataset<CustomDataset> {
 public:
  explicit CustomDataset(const std::string& data_path) {
    // Load your data here
  }

  torch::data::Example<> get(size_t index) override {
    // Return a single data sample
    torch::Tensor data = /* load data at index */;
    torch::Tensor label = /* load label at index */;
    return {data, label};
  }

  torch::optional<size_t> size() const override {
    return dataset_size_;
  }

 private:
  size_t dataset_size_;
};
```

Then use it with a DataLoader:

```cpp
auto dataset = CustomDataset("path/to/data")
  .map(torch::data::transforms::Stack<>());
auto dataloader = torch::data::make_data_loader(
  std::move(dataset),
  torch::data::DataLoaderOptions().batch_size(32).workers(4));
```

## Serialization

### How do I save and load model weights?

**Problem**: You want to save trained model weights and load them later.

**Fix**: Use `torch::save()` and `torch::load()`:

```cpp
// Saving
torch::save(model, "model.pt");

// Loading
torch::load(model, "model.pt");
```

For saving only specific tensors or state:

```cpp
torch::serialize::OutputArchive archive;
model->save(archive);
archive.save_to("model_weights.pt");

// Loading
torch::serialize::InputArchive archive;
archive.load_from("model_weights.pt");
model->load(archive);
```

## Build and Compilation

### CMake cannot find Torch

**Problem**: When building your project with CMake, you get an error that
`Torch` package cannot be found.

**Fix**: You need to specify the path to the LibTorch installation using
`CMAKE_PREFIX_PATH`:

```cpp
cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch ..
```

Alternatively, set `Torch_DIR` to point to the directory containing
`TorchConfig.cmake`:

```cpp
cmake -DTorch_DIR=/path/to/libtorch/share/cmake/Torch ..
```

### Linker errors with undefined references

**Problem**: Your project compiles but you get linker errors with undefined
references to PyTorch symbols.

**Fix**: Ensure you're linking against all required libraries in your
`CMakeLists.txt`:

```cpp
find_package(Torch REQUIRED)
add_executable(my_app main.cpp)
target_link_libraries(my_app "${TORCH_LIBRARIES}")
set_property(TARGET my_app PROPERTY CXX_STANDARD 17)
```

Also ensure that the compiler flags are set correctly:

```cpp
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")
```