Aliases in torch.optim

Created On: Jul 18, 2025 | Last Updated On: Aug 27, 2026

Most optimizer classes are available both from torch.optim and from the nested module in which they are defined. For example, torch.optim.Adam and torch.optim.adam.Adam refer to the same class. While some functional optimizer APIs are exposed in their respective nested modules, all functional optimizer APIs are available in the torch.optim.functional namespace; see Functional optimizer API for usage guidance.

Adadelta	Implements Adadelta algorithm.
adadelta	Functional API that performs Adadelta algorithm computation.

Adagrad	Implements Adagrad algorithm.
adagrad	Functional API that performs Adagrad algorithm computation.

Adam	Implements Adam algorithm.
adam	Functional API that performs Adam algorithm computation.

Adamax	Implements Adamax algorithm (a variant of Adam based on infinity norm).
adamax	Functional API that performs Adamax algorithm computation.

AdamW	Implements AdamW algorithm, where weight decay does not accumulate in the momentum nor variance.
adamw	Functional API that performs AdamW algorithm computation.

ASGD	Implements Averaged Stochastic Gradient Descent.
asgd	Functional API that performs ASGD algorithm computation.

LBFGS

Implements L-BFGS algorithm.

Implementation for the NAdam algorithm.

NAdam	Implements NAdam algorithm.
nadam	Functional API that performs NAdam algorithm computation.

Implementation for the RAdam algorithm.

RAdam	Implements RAdam algorithm.
radam	Functional API that performs RAdam algorithm computation.

Implementation for the RMSprop algorithm.

RMSprop	Implements RMSprop algorithm.
rmsprop	Functional API that performs RMSprop algorithm computation.

Implementation for the Resilient backpropagation.

Rprop	Implements the resilient backpropagation algorithm.
rprop	Functional API that performs Rprop algorithm computation.

Implementation for Stochastic Gradient Descent optimizer.

SGD	Implements stochastic gradient descent (optionally with momentum).
sgd	Functional API that performs SGD algorithm computation.

SparseAdam

SparseAdam implements a masked version of the Adam algorithm suitable for sparse gradients.