Storage Backends

TorchRL provides various storage backends for replay buffers, each optimized for different use cases.

CompressedListStorage(max_size, *[, ...])	A storage that compresses and decompresses data.
CompressedListStorageCheckpointer()	A storage checkpointer for CompressedListStorage.
FlatStorageCheckpointer([done_keys, reward_keys])	Saves the storage in a compact form, saving space on the TED format.
H5StorageCheckpointer(*[, checkpoint_file, ...])	Saves the storage in a compact form, saving space on the TED format and using H5 format to save the data.
ImmutableDatasetWriter([compilable])	A blocking writer for immutable datasets.
LazyMemmapStorage(max_size, *[, ...])	A memory-mapped storage for tensors and tensordicts.
LazyTensorStorage(max_size, *[, device, ...])	A pre-allocated tensor storage for tensors and tensordicts.
ListStorage([max_size, compilable, device])	A storage stored in a list.
LazyStackStorage([max_size, compilable, ...])	A ListStorage that returns LazyStackTensorDict instances.
ListStorageCheckpointer()	A storage checkpointer for ListStoage.
NestedStorageCheckpointer([done_keys, ...])	Saves the storage in a compact form, saving space on the TED format and using memory-mapped nested tensors.
Storage(max_size[, checkpointer, compilable])	A Storage is the container of a replay buffer.
StorageCheckpointerBase()	Public base class for storage checkpointers.
StorageEnsemble(*storages[, transforms])	An ensemble of storages.
StorageEnsembleCheckpointer()	Checkpointer for ensemble storages.
TensorStorage(storage[, max_size, device, ...])	A storage for tensors and tensordicts.
TensorStorageCheckpointer()	A storage checkpointer for TensorStorages.

Storage Performance

Storage choice is very influential on replay buffer sampling latency, especially in distributed reinforcement learning settings with larger data volumes. LazyMemmapStorage is highly advised in distributed settings with shared storage due to the lower serialization cost of MemoryMappedTensors as well as the ability to specify file storage locations for improved node failure recovery.