torch.nn.attention.bias

Created On: Dec 05, 2023 | Last Updated On: Jan 24, 2024

Defines bias subclasses that work with scaled_dot_product_attention

CausalBias

CausalBias

A bias representing causal attention patterns.

causal_lower_right	Creates a lower-right triangular causal bias.
causal_upper_left	Creates an upper-left triangular causal bias.
CausalVariant	Enum for causal variants used in attention mechanisms.