reference/data_samplers

Run in Google Colab

Colab

Download Notebook

Notebook

View on GitHub

GitHub

Sampling Strategies

Samplers control how data is retrieved from the replay buffer storage.

See also

The trajectory-aware samplers (SliceSampler and its variants) recover episode boundaries from the stored data at sampling time. The conventions they rely on — trajectory ids, end flags, circular-storage wraparound and the write cursor — are documented in Trajectory boundaries.

PrioritizedSampler(*args, **kwargs)	Prioritized sampler for replay buffer.
PrioritizedSliceSampler(*args, **kwargs)	Samples slices of data along the first dimension, given start and stop signals, using prioritized sampling.
PromptGroupSampler(*args, **kwargs)	A sampler that draws complete groups of items sharing a common key.
ConsumingSampler(*args, **kwargs)	A random sampler that consumes entries after they have been sampled.
RandomSampler(*args, **kwargs)	A uniformly random sampler for composable replay buffers.
Sampler(*args, **kwargs)	A generic sampler base class for composable Replay Buffers.
SamplerEnsemble(*args, **kwargs)	An ensemble of samplers.
SamplerWithoutReplacement(*args, **kwargs)	A data-consuming sampler that ensures that the same sample is not present in consecutive batches.
SliceSampler(*args, **kwargs)	Samples slices of data along the first dimension, given start and stop signals.
SliceSamplerWithoutReplacement(*args, **kwargs)	Samples slices of data along the first dimension, given start and stop signals, without replacement.
StalenessAwareSampler(*args, **kwargs)	A sampler that weights entries by freshness and filters stale entries.

Writers

Writers control how data is written to the storage. Writers that reuse storage slots can stamp each slot with a reuse counter so consumers holding an index can detect that it was overwritten – see Detecting overwritten slots.

RoundRobinWriter([compilable, track_generations])	A RoundRobin Writer class for composable replay buffers.
TensorDictMaxValueWriter([rank_key, reduction])	A Writer class for composable replay buffers that keeps the top elements based on some ranking key.
TensorDictRoundRobinWriter([compilable, ...])	A RoundRobin Writer class for composable, tensordict-based replay buffers.
Writer([compilable])	A ReplayBuffer base Writer class.
WriterEnsemble(*writers)	An ensemble of writers.