reference/modules_exploration

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Exploration Strategies

Exploration modules add noise to actions to enable exploration during training. NoisyLinear instead injects learnable noise in parameter space. The forward pass does not resample; callers must call reset_noise() or module.apply(reset_noise). make_trainer() registers such a resample on the pre_optim_steps hook when cfg.noisy is set.

AdditiveGaussianModule(*args, **kwargs)	Additive Gaussian PO module.
ConsistentDropoutModule(*args, **kwargs)	A TensorDictModule wrapper for ConsistentDropout.
EGreedyModule(*args, **kwargs)	Epsilon-Greedy exploration module.
NoisyLazyLinear(out_features[, bias, ...])	Noisy Lazy Linear Layer.
NoisyLinear(in_features, out_features[, ...])	Noisy Linear Layer.
OrnsteinUhlenbeckProcessModule(*args, **kwargs)	Ornstein-Uhlenbeck exploration policy module.

Helpers

reset_noise(layer)	Resets the noise of noisy layers.
set_exploration_modules_spec_from_env(...)	Sets exploration module specs from an environment action spec.