Class LayerNormImpl

• Defined in File normalization.h

Inheritance Relationships

Base Type

• public torch::nn::Cloneable< LayerNormImpl > (Template Class Cloneable)

Class Documentation

class LayerNormImpl : public torch::nn::Cloneable<LayerNormImpl>

Applies Layer Normalization over a mini-batch of inputs as described in the paper Layer Normalization_ .

See https://pytorch.org/docs/main/nn.html#torch.nn.LayerNorm to learn about the exact behavior of this module.

See the documentation for torch::nn::LayerNormOptions class to learn what constructor arguments are supported for this module.

Example:

LayerNorm model(LayerNormOptions({2,
2}).elementwise_affine(false).eps(2e-5));

Public Functions

inline LayerNormImpl(std::vector<int64_t> normalized_shape)

explicit LayerNormImpl(LayerNormOptions options_)

virtual void reset() override

reset() must perform initialization of all members with reference semantics, most importantly parameters, buffers and submodules.

void reset_parameters()

virtual void pretty_print(std::ostream &stream) const override

Pretty prints the LayerNorm module into the given stream.

Tensor forward(const Tensor &input)

Applies layer normalization over a mini-batch of inputs as described in the paper Layer Normalization_ .

The mean and standard-deviation are calculated separately over the last certain number dimensions which have to be of the shape specified by input normalized_shape.

Layer Normalization: https://arxiv.org/abs/1607.06450

Public Members

LayerNormOptions options

The options with which this module was constructed.

Tensor weight

The learned weight.

Initialized to ones if the elementwise_affine option is set to true upon construction.

Tensor bias

The learned bias.

Initialized to zeros elementwise_affine option is set to true upon construction.