Skip to main content
Ctrl+K

Torch-TensorRT

  • Installation
  • User Guide
  • Advanced Usage
  • Model Zoo
  • API Reference
    • Debugging
    • Contributing
    • Legacy Frontends
  • GitHub
  • Installation
  • User Guide
  • Advanced Usage
  • Model Zoo
  • API Reference
  • Debugging
  • Contributing
  • Legacy Frontends
  • GitHub

Section Navigation

Architecture

  • System Overview
  • Compiler Phases
    • Lowering Phase
    • Partitioning Phase
    • Conversion Phase
    • Runtime Phase
  • Built-in Dynamo Lowering Passes
  • Observer / Callback System
  • Writing TorchScript Converters
  • Useful Links for Torch-TensorRT Development

Feature Design

  • Engine Caching
  • Weight Refit
  • CUDAGraphs
  • Dynamic Shape Support
  • Complex Number Support
  • Dynamic Memory Allocation
  • Autocast and Precision Management
  • Export and Serialization
  • Contributing
  • Useful...

Useful Links for Torch-TensorRT Development#

TensorRT Available Layers and Expected Dimensions#

  • https://docs.nvidia.com/deeplearning/sdk/tensorrt-support-matrix/index.html#layers-matrix

TensorRT C++ Documentation#

  • https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/index.html

TensorRT Python Documentation (Sometimes easier to read)#

  • https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/index.html

PyTorch Functional API#

  • https://pytorch.org/docs/stable/nn.functional.html

PyTorch native_ops#

  • pytorch/pytorch

PyTorch IR Documentation#

  • pytorch/pytorch

previous

Writing TorchScript Converters

next

Engine Caching

On this page
  • TensorRT Available Layers and Expected Dimensions
  • TensorRT C++ Documentation
  • TensorRT Python Documentation (Sometimes easier to read)
  • PyTorch Functional API
  • PyTorch native_ops
  • PyTorch IR Documentation
Edit on GitHub
Show Source

© Copyright 2024, NVIDIA Corporation.