torch.nn.attention.bias#
Created On: Dec 05, 2023 | Last Updated: Jan 24, 2024
Defines bias subclasses that work with scaled_dot_product_attention
CausalBias#
CausalBias |
A bias representing causal attention patterns. |
causal_lower_right |
Creates a lower-right triangular causal bias. |
causal_upper_left |
Creates an upper-left triangular causal bias. |
CausalVariant |
Enum for causal variants used in attention mechanisms. |