Torch-TensorRT C++ API

Class Hierarchy

• • Namespace torch_tensorrt
    • Namespace torch_tensorrt::executorch_backend
      • Struct EngineHandle
      • Struct InputProfileBounds
      • Struct TensorRTBlobHeader
      • Struct TRTDeleter
      • Class CudaStreamGuard
      • Class TensorRTBackend
      • Class TRTLogger
    • Namespace torch_tensorrt::logging
      • Enum Level
    • Namespace torch_tensorrt::torchscript
      • Struct CompileSpec
    • Struct Device
      • Class Device::DeviceType
    • Struct GraphInputs
    • Struct Input
    • Class DataType
    • Class TensorFormat
    • Enum EngineCapability

File Hierarchy

• • Directory cpp
    • Directory include
      • Directory torch_tensorrt
        • Directory executorch
          • File TensorRTBackend.h
          • File TensorRTBindingNames.h
          • File TensorRTBlobHeader.h
        • File logging.h
        • File macros.h
        • File torch_tensorrt.h

Full API

Namespaces

• Namespace torch_tensorrt
  • Namespaces
  • Classes
  • Enums
  • Functions

• Namespace torch_tensorrt::executorch_backend
  • Namespaces
  • Classes
  • Typedefs

• Namespace torch_tensorrt::executorch_backend::detail

• Namespace torch_tensorrt::logging
  • Enums
  • Functions

• Namespace torch_tensorrt::torchscript
  • Classes
  • Functions

Classes and Structs

• Struct Device
  • Nested Relationships
    • Nested Types
  • Struct Documentation
    • torch_tensorrt::Device
    • Device()
    • device_type
    • gpu_id
    • dla_core
    • allow_gpu_fallback
    • torch_tensorrt::Device::DeviceType
    • Value
    • kGPU
    • kDLA
    • DeviceType()
    • DeviceType()
    • DeviceType()
    • operator Value()
    • operator bool()
    • operator==()
    • operator!=()

• Struct EngineHandle
  • Struct Documentation
    • torch_tensorrt::executorch_backend::EngineHandle
    • ~EngineHandle()
    • logger
    • runtime
    • engine
    • exec_ctx
    • input_binding_names
    • output_binding_names
    • input_profile_bounds
    • cached_input_ptrs
    • cached_input_sizes
    • cached_output_ptrs
    • cached_output_sizes
    • num_inputs
    • num_outputs
    • device_id
    • unified_memory
    • mu
    • inflight_event
    • inflight_pending

• Struct InputProfileBounds
  • Struct Documentation
    • torch_tensorrt::executorch_backend::InputProfileBounds
    • min
    • max

• Struct TensorRTBlobHeader
  • Struct Documentation
    • torch_tensorrt::executorch_backend::TensorRTBlobHeader
    • metadata_offset
    • metadata_size
    • engine_offset
    • engine_size
    • input_binding_names
    • output_binding_names
    • hardware_compatible
    • device_id
    • engine_data()
    • parse()

• Struct TRTDeleter
  • Struct Documentation
    • torch_tensorrt::executorch_backend::TRTDeleter
    • operator()()

• Struct GraphInputs
  • Struct Documentation
    • torch_tensorrt::GraphInputs
    • input_signature
    • inputs

• Struct Input
  • Inheritance Relationships
    • Base Type
  • Struct Documentation
    • torch_tensorrt::Input
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • Input()
    • min_shape
    • opt_shape
    • max_shape
    • shape
    • dtype
    • format
    • tensor_domain

• Struct CompileSpec
  • Struct Documentation
    • torch_tensorrt::torchscript::CompileSpec
    • CompileSpec()
    • CompileSpec()
    • CompileSpec()
    • CompileSpec()
    • graph_inputs
    • enabled_precisions
    • disable_tf32
    • sparse_weights
    • refit
    • debug
    • truncate_long_and_double
    • allow_shape_tensors
    • device
    • capability
    • num_avg_timing_iters
    • workspace_size
    • dla_sram_size
    • dla_local_dram_size
    • dla_global_dram_size
    • require_full_compilation
    • min_block_size
    • torch_executed_ops
    • torch_executed_modules

• Class DataType
  • Class Documentation
    • torch_tensorrt::DataType
    • Value
    • kLong
    • kDouble
    • kFloat
    • kHalf
    • kChar
    • kInt
    • kBool
    • kUnknown
    • DataType()
    • DataType()
    • DataType()
    • operator Value()
    • operator bool()
    • operator==()
    • operator==()
    • operator!=()
    • operator!=()

• Class Device::DeviceType
  • Nested Relationships
  • Class Documentation
    • torch_tensorrt::Device::DeviceType
    • Value
    • kGPU
    • kDLA
    • DeviceType()
    • DeviceType()
    • DeviceType()
    • operator Value()
    • operator bool()
    • operator==()
    • operator!=()

• Class CudaStreamGuard
  • Class Documentation
    • torch_tensorrt::executorch_backend::CudaStreamGuard
    • CudaStreamGuard()
    • ~CudaStreamGuard()
    • CudaStreamGuard()
    • operator=()

• Class TensorRTBackend
  • Inheritance Relationships
    • Base Type
  • Class Documentation
    • torch_tensorrt::executorch_backend::TensorRTBackend
    • is_available()
    • init()
    • execute()
    • destroy()

• Class TRTLogger
  • Inheritance Relationships
    • Base Type
  • Class Documentation
    • torch_tensorrt::executorch_backend::TRTLogger
    • log()

• Class TensorFormat
  • Class Documentation
    • torch_tensorrt::TensorFormat
    • Value
    • kContiguous
    • kChannelsLast
    • kUnknown
    • TensorFormat()
    • TensorFormat()
    • TensorFormat()
    • operator Value()
    • operator bool()
    • operator==()
    • operator==()
    • operator!=()
    • operator!=()

Enums

• Enum EngineCapability
  • Enum Documentation
    • EngineCapability
    • kSTANDARD
    • kSAFETY
    • kDLA_STANDALONE

• Enum Level
  • Enum Documentation
    • Level
    • kINTERNAL_ERROR
    • kERROR
    • kWARNING
    • kINFO
    • kDEBUG
    • kGRAPH

Functions

• Function torch_tensorrt::dump_build_info
  • Function Documentation
    • dump_build_info()

• Function torch_tensorrt::get_build_info
  • Function Documentation
    • get_build_info()

• Function torch_tensorrt::logging::get_is_colored_output_on
  • Function Documentation
    • get_is_colored_output_on()

• Function torch_tensorrt::logging::get_logging_prefix
  • Function Documentation
    • get_logging_prefix()

• Function torch_tensorrt::logging::get_reportable_log_level
  • Function Documentation
    • get_reportable_log_level()

• Function torch_tensorrt::logging::log
  • Function Documentation
    • log()

• Function torch_tensorrt::logging::set_is_colored_output_on
  • Function Documentation
    • set_is_colored_output_on()

• Function torch_tensorrt::logging::set_logging_prefix
  • Function Documentation
    • set_logging_prefix()

• Function torch_tensorrt::logging::set_reportable_log_level
  • Function Documentation
    • set_reportable_log_level()

• Function torch_tensorrt::set_device
  • Function Documentation
    • set_device()

• Function torch_tensorrt::torchscript::check_method_operator_support
  • Function Documentation
    • check_method_operator_support()

• Function torch_tensorrt::torchscript::compile
  • Function Documentation
    • compile()

• Function torch_tensorrt::torchscript::convert_method_to_trt_engine
  • Function Documentation
    • convert_method_to_trt_engine()

• Function torch_tensorrt::torchscript::embed_engine_in_new_module
  • Function Documentation
    • embed_engine_in_new_module()

Defines

• Define STR
  • Define Documentation
    • STR

• Define TORCH_TENSORRT_MAJOR_VERSION
  • Define Documentation
    • TORCH_TENSORRT_MAJOR_VERSION

• Define TORCH_TENSORRT_MINOR_VERSION
  • Define Documentation
    • TORCH_TENSORRT_MINOR_VERSION

• Define TORCH_TENSORRT_PATCH_VERSION
  • Define Documentation
    • TORCH_TENSORRT_PATCH_VERSION

• Define TORCH_TENSORRT_VERSION
  • Define Documentation
    • TORCH_TENSORRT_VERSION

• Define TORCHTRT_API
  • Define Documentation
    • TORCHTRT_API

• Define TORCHTRT_HIDDEN
  • Define Documentation
    • TORCHTRT_HIDDEN

• Define XSTR
  • Define Documentation
    • XSTR

Typedefs

• Typedef torch_tensorrt::executorch_backend::TRTUniquePtr
  • Typedef Documentation
    • TRTUniquePtr