API Reference#
- Process Groups
ErrorSwallowingProcessGroupWrapperFakeProcessGroupWrapperManagedProcessGroupProcessGroupProcessGroupBabyProcessGroupBabyGlooProcessGroupBabyNCCLProcessGroupDummyProcessGroupGlooProcessGroupNCCLProcessGroupWrappercreate_store_client()- Manager
ExceptionWithTracebackManagerWorldSizeMode- Optimizers
OptimizerWrapper- Distributed Data Parallel
DistributedDataParallelPureDistributedDataParallel- LocalSGD
DiLoCoLocalSGDextract_local_tensor()- Data
DistributedSampler- Checkpointing
CheckpointTransportHTTPTransport- Parameter Servers
ParameterServer- Coordination (Low Level API)
LighthouseClientLighthouseServerManagerClientManagerServerQuorumQuorumMember