reference/generated/torchrl.modules.ValueTransform

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

ValueTransform

class torchrl.modules.ValueTransform(*args: Any, **kwargs: Any)

Abstract base class for invertible scalar value transforms.

A value transform maps raw rewards or returns to a numerically convenient prediction space. inverse() maps predictions back to the raw value space before they are used for bootstrapping.

Subclasses implement forward() and inverse() as element-wise tensor operations.

Examples

>>> import torch
>>> from torchrl.modules import ValueTransform
>>> class ScaleValueTransform(ValueTransform):
...     def forward(self, value):
...         return value * 2
...     def inverse(self, value):
...         return value / 2
>>> transform = ScaleValueTransform()
>>> value = torch.tensor([-2.0, 0.0, 2.0])
>>> transformed = transform(value)
>>> transformed
tensor([-4.,  0.,  4.])
>>> transform.inverse(transformed)
tensor([-2.,  0.,  2.])

See also

IdentityValueTransform, SymLogValueTransform, SignedHyperbolicValueTransform, and ComposeValueTransform.

abstract forward(value: Tensor) → Tensor

Map a raw value tensor to the transformed prediction space.

abstract inverse(value: Tensor) → Tensor

Map a transformed value tensor back to raw value space.