Rate this Page

LibTorch Stable ABI#

Created On: Mar 17, 2025 | Last Updated: Apr 24, 2025

This note will eventually contain more details on how to use the APIs in torch/csrc/stable. For the moment, it contains a table of internal representations:

  1. type in custom extension: type used within the end user custom library.

  2. StableIValue representation: a stable conversion of the type to liaison between the user model vs libtorch.so in an ABI-stable manner.

  3. type in libtorch: type used within libtorch.so (or any code binary locked with libtorch).

  4. Schema Type: type as described by the schema, which we hail as the source of truth for both ATen ops in native_functions.yaml and for user defined custom operators registered to the dispatcher via TORCH_LIBRARY or torch.library.

type in custom extension

StableIValue representation

type in libtorch

Schema Type

std::optional<S>

if there is a value, raw bitwise copy into leading bytes of uint64_t of pointer to a new StableIValue representing S. if there is no value, nullptr.

std::optional<T>

Type?

RAIIATH

raw bitwise copy of underlying AtenTensorHandle into leading bytes of uint64_t

at::Tensor

Tensor

int32_t

raw bitwise copy into leading bytes of uint64_t

at::ScalarType

ScalarType

int32_t

raw bitwise copy into leading bytes of uint64_t

at::Layout

Layout

int32_t

raw bitwise copy into leading bytes of uint64_t

at::MemoryFormat

MemoryFormat

bool

raw bitwise copy into leading bytes of uint64_t

bool

bool

int64_t

raw bitwise copy into leading bytes of uint64_t

int64_t

int

double

raw bitwise copy into leading bytes of uint64_t

double

float

?

?

c10::Device

Device

?

?

c10::Stream

Stream

?

?

c10::complex

complex

?

?

at::Scalar

Scalar

?

?

std::string/const char*/ivalue::ConstantString

str

?

?

at::Storage

Storage

?

?

at::Generator

Generator

?

?

c10::List<T>

Type[]

?

?

ivalue::Tuple<T>

(Type, …)

?

?

c10::SymInt

SymInt

?

?

c10::SymFloat

SymFloat

?

?

c10::SymBool

SymBool

?

?

at::QScheme

QScheme

Our confidently supported types are the ones in the table that have completed rows. For a limited set of use cases, we also implicitly support any literal type that is representable within 64 bits as StableIValues, as the default reinterpret_cast will succeed. You can work with StableIValue abstractions in your custom kernel for types such as c10::Device even if there is no standard defined representation of device in custom extensions. For example, a custom operator can take as argument a StableIValue device and directly pass it through to an aten operator with aoti_torch_call_dispatcher.