torch.foreach

Created On: Aug 20, 2026 | Last Updated On: Aug 20, 2026

Operations over lists of tensors.

Warning

torch.foreach is a beta API. Its signatures may change based on user feedback. Existing private torch._foreach_* functions remain available for compatibility during migration.

Each function applies the corresponding ordinary PyTorch operation to every position in one or more tensor lists.

The functions will use an accelerated multi-tensor implementation when their inputs meet its requirements. Otherwise they use a semantically equivalent per-tensor fallback. Calling a function in this module does not guarantee a single or fused kernel.

torch.foreach applies familiar PyTorch operations across lists of tensors. For example, torch.foreach.add(inputs, other) is semantically equivalent to a Python loop that applies torch.add() at every list position. When available on an accelerator and certain conditions are met, a foreach operation will use a horizontally fused multi-tensor kernel to improve runtime. On CUDA, common eligibility requirements include strided, non-overlapping dense tensors on the same device, compatible dtypes, and matching sizes and strides for corresponding tensors.

inputs = [torch.ones(2), torch.ones(3)]
result = torch.foreach.add(inputs, 2)
# Equivalent to tuple(torch.add(tensor, 2) for tensor in inputs)

torch.foreach.mul_(inputs, 3)
# Mutates each tensor and returns `inputs`.

API Coverage

A foreach API lifts an ordinary tensor operation over a list of inputs. This creates a combinatorial space of possible signatures. Depending on the operation, a tensor argument could be shared as a Tensor or supplied elementwise as a TensorList, while a scalar argument could be a Scalar, a ScalarList, a shared 0-D Tensor, or a packed 1-D CPU Tensor.

You can then imagine that one operation may take on various forms such as TensorList/TensorList, TensorList/Tensor, TensorList/ScalarList, TensorList/Scalar, etc. Operations with more parameters would have more combinations. The public foreach APIs support a subset of these combinations based on usage. If you would like to see an implementation of a missing combination, please file an issue!

Across the supported signatures, we maintain constraints that TensorList and ScalarList arguments must be non-empty, and corresponding tensor and scalar lists must have the same length.

Only signatures that explicitly list Tensor include it in the supported typed surface. A 0-D Tensor that does not require gradients may sometimes be accepted for a Scalar parameter through implicit scalar conversion. Converting an accelerator Tensor this way reads its value on the host, which may be expensive. On CUDA, this synchronizes eager execution and is unsupported during CUDA graph capture, so it should not be relied upon as a Tensor overload.

The public foreach API also does not support out= variants and may have a higher memory footprint than looping through the non-foreach original API, as multiple intermediates can be alive simultaneously.

Migrating from the private API

You may be familiar with the private spellings of foreach APIs, e.g., for torch.add():

torch._foreach_add(inputs, other)  # Private spelling
torch.foreach.add(inputs, other)   # Public beta spelling

The private spellings remain available with unchanged signatures for backward compatibility. Public functions call the same ATen operators but improve API consistency in two ways:

1. All required operands are positional-only, and all optional parameters are keyword-only.

2. Parameter names align with the corresponding ordinary operation.

The primary tensor-list argument that the foreach API applies over is named inputs, and other arguments retain the ordinary operation’s logical name, even when the currently supported form requires a list. This keeps signatures descriptive as operand forms evolve.

Unary operations

abs	Applies torch.abs() to each tensor in inputs.
abs_	Applies torch.abs() to each tensor in inputs in-place.
acos	Applies torch.acos() to each tensor in inputs.
acos_	Applies torch.acos() to each tensor in inputs in-place.
asin	Applies torch.asin() to each tensor in inputs.
asin_	Applies torch.asin() to each tensor in inputs in-place.
atan	Applies torch.atan() to each tensor in inputs.
atan_	Applies torch.atan() to each tensor in inputs in-place.
ceil	Applies torch.ceil() to each tensor in inputs.
ceil_	Applies torch.ceil() to each tensor in inputs in-place.
clone	Clones every tensor in inputs.
cos	Applies torch.cos() to each tensor in inputs.
cos_	Applies torch.cos() to each tensor in inputs in-place.
cosh	Applies torch.cosh() to each tensor in inputs.
cosh_	Applies torch.cosh() to each tensor in inputs in-place.
erf	Applies torch.erf() to each tensor in inputs.
erf_	Applies torch.erf() to each tensor in inputs in-place.
erfc	Applies torch.erfc() to each tensor in inputs.
erfc_	Applies torch.erfc() to each tensor in inputs in-place.
exp	Applies torch.exp() to each tensor in inputs.
exp_	Applies torch.exp() to each tensor in inputs in-place.
expm1	Applies torch.expm1() to each tensor in inputs.
expm1_	Applies torch.expm1() to each tensor in inputs in-place.
floor	Applies torch.floor() to each tensor in inputs.
floor_	Applies torch.floor() to each tensor in inputs in-place.
frac	Applies torch.frac() to each tensor in inputs.
frac_	Applies torch.frac() to each tensor in inputs in-place.
lgamma	Applies torch.lgamma() to each tensor in inputs.
lgamma_	Applies torch.lgamma() to each tensor in inputs in-place.
log	Applies torch.log() to each tensor in inputs.
log_	Applies torch.log() to each tensor in inputs in-place.
log10	Applies torch.log10() to each tensor in inputs.
log10_	Applies torch.log10() to each tensor in inputs in-place.
log1p	Applies torch.log1p() to each tensor in inputs.
log1p_	Applies torch.log1p() to each tensor in inputs in-place.
log2	Applies torch.log2() to each tensor in inputs.
log2_	Applies torch.log2() to each tensor in inputs in-place.
neg	Applies torch.neg() to each tensor in inputs.
neg_	Applies torch.neg() to each tensor in inputs in-place.
reciprocal	Applies torch.reciprocal() to each tensor in inputs.
reciprocal_	Applies torch.reciprocal() to each tensor in inputs in-place.
round	Applies torch.round() to each tensor in inputs.
round_	Applies torch.round() to each tensor in inputs in-place.
rsqrt	Applies torch.rsqrt() to each tensor in inputs.
rsqrt_	Applies torch.rsqrt() to each tensor in inputs in-place.
sigmoid	Applies torch.sigmoid() to each tensor in inputs.
sigmoid_	Applies torch.sigmoid() to each tensor in inputs in-place.
sign	Applies torch.sign() to each tensor in inputs.
sign_	Applies torch.sign() to each tensor in inputs in-place.
sin	Applies torch.sin() to each tensor in inputs.
sin_	Applies torch.sin() to each tensor in inputs in-place.
sinh	Applies torch.sinh() to each tensor in inputs.
sinh_	Applies torch.sinh() to each tensor in inputs in-place.
sqrt	Applies torch.sqrt() to each tensor in inputs.
sqrt_	Applies torch.sqrt() to each tensor in inputs in-place.
tan	Applies torch.tan() to each tensor in inputs.
tan_	Applies torch.tan() to each tensor in inputs in-place.
tanh	Applies torch.tanh() to each tensor in inputs.
tanh_	Applies torch.tanh() to each tensor in inputs in-place.
trunc	Applies torch.trunc() to each tensor in inputs.
trunc_	Applies torch.trunc() to each tensor in inputs in-place.
zero_	Fills every tensor in inputs with zero.

Binary operations

add	Applies torch.add() to every tensor in inputs.
add_	Applies torch.add() to every tensor in inputs.
sub	Applies torch.sub() to every tensor in inputs.
sub_	Applies torch.sub() to every tensor in inputs.
mul	Applies torch.mul() to every tensor in inputs.
mul_	Applies torch.mul() to every tensor in inputs.
div	Applies torch.div() to every tensor in inputs.
div_	Applies torch.div() to every tensor in inputs.
clamp_min	Applies torch.clamp() to every tensor in inputs.
clamp_min_	Applies torch.clamp() to every tensor in inputs.
clamp_max	Applies torch.clamp() to every tensor in inputs.
clamp_max_	Applies torch.clamp() to every tensor in inputs.
minimum	Applies torch.minimum() to every tensor in inputs.
minimum_	Applies torch.minimum() to every tensor in inputs.
maximum	Applies torch.maximum() to every tensor in inputs.
maximum_	Applies torch.maximum() to every tensor in inputs.
pow	Applies torch.pow() at each list position.
pow_	In-place version of torch.foreach.pow().
copy_	Copies each tensor in src into the corresponding tensor in inputs, following torch.Tensor.copy_().

Pointwise operations

addcmul	Applies torch.addcmul() to corresponding tensors from the three input lists.
addcmul_	Applies torch.addcmul() to corresponding tensors from the three input lists.
addcdiv	Applies torch.addcdiv() to corresponding tensors from the three input lists.
addcdiv_	Applies torch.addcdiv() to corresponding tensors from the three input lists.
lerp	Applies torch.lerp() to corresponding tensors in inputs and end.
lerp_	In-place version of torch.foreach.lerp().

Reductions and matrix operations

max	Returns the maximum value of each tensor in inputs.
norm	Returns the vector norm of each tensor in inputs.
mm	Multiplies corresponding matrices from inputs and mat2 using torch.mm().