Runtime & Serialization

Configure the Torch-TensorRT runtime, serialize engines for reuse, benchmark correctly, handle unsupported operators, and deploy on specialized hardware (DLA).

• Deploying Torch-TensorRT Programs
• Runtime API
• Runtime Settings (TensorRT-RTX)
• DLA
• Saving models compiled with Torch-TensorRT
• Extracting a Raw TensorRT Engine
• AOTInductor Deployment
• MutableTorchTensorRTModule
• Example: Saving and Loading Models with Dynamic Shapes
• Example: Saving Models with Dynamic Shapes - Both Methods