reference/generated/tensordict.TensorDictBase.make_memmap_from_storage

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

TensorDictBase.make_memmap_from_storage

TensorDictBase.make_memmap_from_storage(key: NestedKey, storage: UntypedStorage, shape: Size | Tensor, *, dtype: dtype | None = None, robust_key: bool | None = True) → MemoryMappedTensor

Adds a memory-mapped tensor at key that reads its data from storage, and returns it.

The tensordict must be memory-mapped (see memmap()).

The entry is written even though a memory-mapped tensordict is locked. Other tensordicts that share its files, for example in other processes, see it only after memmap_refresh_().

If the storage has a file, it must be the file of the new entry in the directory of the tensordict, or the call raises a RuntimeError. A storage without a file is used as it is: its data is not written to the directory. If the tensordict has a directory and the storage is not empty, load_memmap() and memmap_refresh_() drop the entry, and the call emits a FutureWarning: starting with TensorDict 0.17, the default will change to raising a RuntimeError. Use make_memmap_from_tensor() to copy the data into the directory.

Parameters:

• key (NestedKey) – the key of the new entry. A tuple key creates the missing nested tensordicts.

• storage (UntypedStorage) – the storage of the new tensor. It must be in physical memory.

• shape (Size or Tensor) – the shape of the new tensor, or a tensor of shapes for a nested tensor.

Keyword Arguments:

• dtype (dtype, optional) – the dtype of the new tensor. Defaults to None, which uses torch.get_default_dtype().

• robust_key (bool, optional) – if True, keys are percent-encoded in file names, so that every file stays inside the directory of the tensordict. If False, keys are used as file names unchanged. None is the same as True. Defaults to True.

Returns:

The new MemoryMappedTensor.

Raises:

RuntimeError – if the tensordict is not memory-mapped, if the key exists, or if the file of the storage is not the file of the new entry.

Every tensordict class and every tensorclass has this method. For the other methods, see Methods by task.