API 参考#
- 进程组
ErrorSwallowingProcessGroupWrapperFakeProcessGroupWrapperManagedProcessGroupProcessGroupProcessGroupBabyProcessGroupBabyGlooProcessGroupBabyNCCLProcessGroupDummyProcessGroupGlooProcessGroupNCCLProcessGroupWrappercreate_store_client()- 管理器
ExceptionWithTraceback管理器WorldSizeModeget_timeout()- 优化器
OptimizerWrapper- 分布式数据并行
DistributedDataParallelPureDistributedDataParallel- LocalSGD
DiLoCoLocalSGDextract_local_tensor()- 数据
DistributedSampler- 检查点
CheckpointTransportHTTPTransport- 参数服务器
ParameterServer- 协调（低级 API）
LighthouseClientLighthouseServerManagerClientManagerServerQuorumQuorumMember
