reference/modules_critics

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Value Networks and Critics

Value networks estimate the value of states or state-action pairs.

ValueOperator(*args, **kwargs)	General class for value functions in RL.
ValueNorm(*[, shape, epsilon, device])	Abstract base class for value normalisers.
PopArtValueNorm(*[, shape, beta, epsilon, ...])	PopArt-style EMA value normaliser.
RunningValueNorm(*[, shape, epsilon, device])	Exact running mean / variance (Welford's online algorithm).
DuelingCnnDQNet(out_features[, ...])	Dueling CNN Q-network.
DistributionalDQNnet(*args, **kwargs)	Distributional Deep Q-Network softmax layer.
ConvNet(in_features, depth, num_cells, ...)	A convolutional neural network.
CrossCriticGroupSpec(obs_dim, n_agents, ...)	Specification for one agent group used by CrossGroupCritic.
CrossGroupCritic(*args, **kwargs)	Centralised critic that conditions on observations from multiple agent groups.
MLP(in_features, out_features, depth, ...)	A multi-layer perceptron.
DdpgCnnActor(action_dim[, conv_net_kwargs, ...])	DDPG Convolutional Actor class.
DdpgCnnQNet([conv_net_kwargs, ...])	DDPG Convolutional Q-value class.
DdpgMlpActor(action_dim[, mlp_net_kwargs, ...])	DDPG Actor class.
DdpgMlpQNet([mlp_net_kwargs_net1, ...])	DDPG Q-value MLP class.
LSTMModule(*args, **kwargs)	An embedder for an LSTM module.
GRUModule(*args, **kwargs)	An embedder for an GRU module.
canonicalize_rnn_subset(data, modules, *[, ...])	Canonicalize only the union of RNN keys used by modules.
set_recurrent_mode([mode])	Context manager for setting RNNs recurrent mode.
OnlineDTActor(state_dim, action_dim[, ...])	Online Decision Transformer Actor class.
DTActor(state_dim, action_dim[, ...])	Decision Transformer Actor class.
DecisionTransformer(state_dim, action_dim[, ...])	Online Decision Transformer.

Value transforms

Value transforms compress large reward and return scales into a numerically convenient prediction space and provide the inverse mapping needed before bootstrapping. They can be composed to build custom invertible mappings.

ValueTransform(*args, **kwargs)	Abstract base class for invertible scalar value transforms.
IdentityValueTransform(*args, **kwargs)	Leave scalar values unchanged.
SymLogValueTransform(*args, **kwargs)	Symmetric-log value transform used by DreamerV3.
SignedHyperbolicValueTransform([epsilon])	Signed-hyperbolic transform for large-magnitude value targets.
ComposeValueTransform(*transforms)	Compose value transforms while preserving the inverse mapping.
functional.symlog(value)	Apply the element-wise symmetric logarithm transform.
functional.symexp(value)	Apply the inverse symmetric exponential transform element-wise.
functional.signed_hyperbolic(value[, epsilon])	Apply the signed hyperbolic value transform.
functional.signed_parabolic(value[, epsilon])	Apply the inverse of signed_hyperbolic() element-wise.