XPU Streams

XPU streams provide a mechanism for asynchronous execution of operations on Intel GPUs. Like CUDA streams, operations queued to the same stream execute in order, while operations on different streams can execute concurrently.

XPUStream

class XPUStream

Public Types

enum Unchecked

Values:

enumerator UNCHECKED

Public Functions

inline explicit XPUStream(Stream stream)

Construct a XPUStream from a Stream.

This construction is checked, and will raise an error if the Stream is not, in fact, a XPU stream.

inline explicit XPUStream(Unchecked, Stream stream)

Construct a XPUStream from a Stream with no error checking.

inline bool operator==(const XPUStream &other) const noexcept

inline bool operator!=(const XPUStream &other) const noexcept

inline operator sycl::queue&() const

Implicit conversion to sycl::queue&.

inline operator sycl::queue*() const

Implicit conversion to sycl::queue*.

inline operator Stream() const

Implicit conversion to Stream (a.k.a., forget that the stream is a XPU stream).

inline DeviceType device_type() const

Get the XPU device type that this stream is associated with.

inline DeviceIndex device_index() const

Get the XPU device index that this stream is associated with.

inline Device device() const

Get the full Device that this stream is associated with.

The Device is guaranteed to be a XPU device.

inline StreamId id() const

Return the stream ID corresponding to this particular stream.

StreamId is a int64_t representation generated by its type and index.

inline bool query() const

Return true if all enqueued tasks in this stream have been completed, otherwise return false.

inline void synchronize() const

Performs a blocking wait for the completion of all enqueued tasks in this stream.

int priority() const

Return the priority that this stream is associated with.

Lower numbers represent higher priority.

sycl::queue &queue() const

Explicit conversion to sycl::queue&.

inline Stream unwrap() const

Explicit conversion to Stream.

inline struct c10::StreamData3 pack3() const

Reversibly pack a XPUStream into a struct representation.

The XPUStream can be unpacked using unpack3().

Public Static Functions

static inline XPUStream unpack3(StreamId stream_id, DeviceIndex device_index, DeviceType device_type)

Unpack a XPUStream from the 3 fields generated by pack3().

static inline std::tuple<int, int> priority_range()

Return the range of priority supported by PyTorch.

Example:

#include <c10/xpu/XPUStream.h>

// Get the current XPU stream
auto stream = c10::xpu::getCurrentXPUStream();

// Create a new stream from the pool
auto new_stream = c10::xpu::getStreamFromPool();

// Synchronize
stream.synchronize();

Acquiring XPU Streams

XPUStream c10::xpu::getCurrentXPUStream(DeviceIndex device = -1)

Get the current XPU stream, for the passed XPU device, or for the current device if no device index is passed.

void c10::xpu::setCurrentXPUStream(XPUStream stream)

Set the current stream on the device of the passed in stream to be the passed in stream.

XPUStream c10::xpu::getStreamFromPool(const bool isHighPriority = false, DeviceIndex device = -1)

Get a stream from the pool in a round-robin fashion.

You can request a stream from the highest priority pool by setting isHighPriority to true for a specific device.

Stream Synchronization

void c10::xpu::syncStreamsOnDevice(DeviceIndex device = -1)

Block all reserved SYCL queues in the stream pools on the device, and wait for their synchronizations.