Download

Skill UI

Browse and discover 5019+ curated skills

All Development Artificial Intelligence Design & Creative Product & Business Data Science Marketing Soft Skills Productivity Engineering Languages

Search FSDP , found 7 results

Default Newest Most Downloaded

Axolotl Fine-Tuning Guide

Orchestra-Research/AI-Research-SKILLs

Guides developers through Axolotl-based LLM fine-tuning, covering YAML config patterns, LoRA/QLoRA and DPO/KTO/ORPO/GRPO workflows, FSDP tips, compressed checkpoints, and debugging best practices for reliable multimodal training.

Axolotl Fine-Tuning Guide

Orchestra-Research/AI-Research-SKILLs

Guides developers through Axolotl-based LLM fine-tuning, covering YAML config patterns, LoRA/QLoRA and DPO/KTO/ORPO/GRPO workflows, FSDP tips, compressed checkpoints, and debugging best practices for reliable multimodal training.

TorchTitan LLM Pretraining

distributed-llm-pretraining-torchtitan

Orchestra-Research/AI-Research-SKILLs

TorchTitan delivers PyTorch-native distributed LLM pretraining with composable 4D parallelism (FSDP2, TP, PP, CP), Float8, torch.compile, and checkpointing so you can scale Llama 3.1, DeepSeek V3, or custom models across 8 to 512+ GPUs.

Accelerate Distributed Training

huggingface-accelerate

Orchestra-Research/AI-Research-SKILLs

Accelerate wraps PyTorch scripts with a four-line integration so you can launch DDP, DeepSpeed, FSDP, Megatron, or mixed-precision training on any hardware via a single command, handling device placement, sharding, and automatic config generation.

PyTorch FSDP2 Training

Orchestra-Research/AI-Research-SKILLs

Adds PyTorch FSDP2 setup to training scripts so you can initialize torchrun, bottom-up shard modules, configure mixed precision/offload, and checkpoint via DCP when models outgrow a single GPU.

Lightning Training Suite

pytorch-lightning

Orchestra-Research/AI-Research-SKILLs

High-level PyTorch Lightning streamlines training loops with automatic distributed support (DDP/FSDP/DeepSpeed), callbacks, checkpointing, logging, and device management so you can scale from a laptop to multi-node clusters without boilerplate.

Verl RL Training

verl-rl-training

Orchestra-Research/AI-Research-SKILLs

Guides RLHF-style LLM fine-tuning with verl's HybridFlow stack, scaling PPO/GRPO/DAPO across FSDP/Megatron/vLLM backends for math reasoning, agentic multiturn, and vision-language workflows.

1

Language