reference/objectives_other

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Other Loss Modules

Additional loss modules for specialized algorithms.

ACTLoss(*args, **kwargs)	Loss module for Action Chunking with Transformers (ACT).
BCLoss(*args, **kwargs)	Behavior Cloning Loss Module.
DiffusionBCLoss(*args, **kwargs)	Behavioural Cloning loss for diffusion-based policies.
GAILLoss(*args, **kwargs)	TorchRL implementation of the Generative Adversarial Imitation Learning (GAIL) loss.
DTLoss(*args, **kwargs)	TorchRL implementation of the Online Decision Transformer loss.
OnlineDTLoss(*args, **kwargs)	TorchRL implementation of the Online Decision Transformer loss.
DreamerActorLoss(*args, **kwargs)	Dreamer Actor Loss.
DreamerModelLoss(*args, **kwargs)	Dreamer Model Loss.
DreamerValueLoss(*args, **kwargs)	Dreamer Value Loss.
WorldModelLoss(*args, **kwargs)	A general loss module for model-based world models.
ExponentialQuadraticCost(*args, **kwargs)	Computes the expected saturating cost for a Gaussian-distributed state.
RNDLoss(*args, **kwargs)	Loss module for training the predictor network in Random Network Distillation.

DreamerV3

Loss modules for DreamerV3 (Mastering Diverse Domains in World Models, Hafner et al. 2023). Key differences from V1: discrete categorical latent state, KL balancing, symlog transforms, and two-hot value distributions.

DreamerV3ActorLoss(*args, **kwargs)	DreamerV3 Actor Loss.
DreamerV3ModelLoss(*args, **kwargs)	DreamerV3 World Model Loss.
DreamerV3ValueLoss(*args, **kwargs)	DreamerV3 Value Loss.

DreamerV3 Utilities

symlog(value)	Apply the element-wise symmetric logarithm transform.
symexp(value)	Apply the inverse symmetric exponential transform element-wise.
two_hot_encode(x, bins)	Encode raw scalar values on a sorted two-hot support.
two_hot_decode(logits, bins)	Decode logits over a raw-value support to their scalar expectation.
two_hot_cross_entropy(logits, target, bins)	Return two-hot cross entropy for raw scalar targets.
categorical_kl_terms(posterior_logits, ...)	Return DreamerV3 dynamics and representation KL losses.