Rate this Page

torch.nn.attention.bias#

Created On: Dec 05, 2023 | Last Updated: Jan 24, 2024

Defines bias subclasses that work with scaled_dot_product_attention

CausalBias#

CausalBias

A bias representing causal attention patterns.

causal_lower_right

Creates a lower-right triangular causal bias.

causal_upper_left

Creates an upper-left triangular causal bias.

CausalVariant

Enum for causal variants used in attention mechanisms.