Skill UI

Guides engineers through DeepSpeed distributed training and NVMe I/O optimizations, covering ZeRO, pipeline parallelism, mixed precision, and DeepNVMe handles for synchronous and asynchronous tensor storage.

HuggingFace Accelerate adds distributed support to any PyTorch script with just four lines, providing a unified API for DDP, DeepSpeed, FSDP, and Megatron, plus automatic device placement, mixed precision, and simple launch/config commands for fast prototyping.

Automates mixed precision ML training workflows by surfacing best practices, generating runnable code/configurations, and validating setups across data prep, model tuning, and experiment tracking.

Guides adding PyTorch FSDP2’s fully_shard integration to training loops, covering correct distributed init, bottom-up sharding, mixed precision/offload policies, DTensor-aware optimizers, and distributed checkpointing so oversized models or DeviceMesh sharding goals work reliably.