torch.cuda.memory.max_memory_reserved

torch.cuda.memory.max_memory_reserved(device=None)

Return the maximum GPU memory managed by the caching allocator in bytes for a given device.

By default, this returns the peak cached memory since the beginning of this program. reset_peak_memory_stats() can be used to reset the starting point in tracking this metric. For example, these two functions can measure the peak cached memory amount of each iteration in a training loop.

Parameters:

device (torch.device or int, optional) – selected device. Returns statistic for the current device, given by current_device(), if device is None (default).

Return type:

int

Note

See Memory management for more details about GPU memory management.

Note

Under PYTORCH_CUDA_ALLOC_CONF=backend:cudaMallocAsync, the peak is computed by summing the high-water marks of the default mempool and the device graph-memory pool (CUDA graph captures reserve backing in the latter). Because those two high-water marks need not occur at the same instant, the reported peak is a conservative upper bound on the true simultaneous peak. The current value (memory_reserved()) is exact.