Plugins

Register custom CUDA and Triton kernels as TensorRT plugins — from auto-generated Python plugins to AOT-compiled C++ plugins for use in serialized engines.

• Plugin System
• Example: Auto-generate a Plugin for a Custom Kernel
• Example: Using Custom Kernels within TensorRT Engines
• Automatically Generate a TensorRT AOT Plugin
• Step 1: Define the Triton Kernel
• Step 2: Register the PyTorch op
• Step 3: Register the QDP Shape Descriptor
• Step 4: Register the AOT Implementation
• Step 5: Generate the Converter
• Step 6: Compile and Run
• Example: Custom Kernels with NVRTC in TensorRT AOT Plugins