Skill UI

Orchestrates PyTorch-native TorchTitan 4D parallelism (FSDP2, TP, PP, CP) for large-scale LLM pretraining, scaling from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing to boot Llama 3.1, DeepSeek V3, or custom models.

Guides automated distributed training setups with step-by-step practices, production-ready configs, and validation checks for ML training workflows.

High-performance RLHF framework that combines Ray, vLLM, and ZeRO-3 to accelerate PPO/GRPO/RLOO/DPO training for 7B-70B+ models on distributed GPU clusters, providing a single-stack workflow for reward models and policy optimization.

PyTorch Lightning is a high-level framework built on PyTorch that significantly reduces boilerplate code in deep learning training loops. It automatically handles device management, distributed training (DDP, FSDP, DeepSpeed), mixed precision, and advanced callbacks. It allows researchers to scale models from local machines to supercomputers with minimal code changes, enabling clean, scalable, and best-practice deep learning implementations.

torchforge is Meta's advanced PyTorch-native library designed for agentic Reinforcement Learning (RL). It fundamentally separates core RL algorithms from complex distributed infrastructure concerns. This allows researchers to rapidly prototype and experiment with novel RL methods (like GRPO) using clean, PyTorch-native abstractions, while automatically handling large-scale tasks like distributed training across thousands of GPUs using Monarch and TorchTitan.

Guides training large language models with Volcano Engine's verl library, covering PPO/GRPO workflows, distributed backends, multi-turn rollouts, reward configs, critic-enabled PPO, and production deployments using Ray workers.

PyTorch Lightning is a deep learning framework that organizes PyTorch code to eliminate boilerplate while maintaining flexibility. It automates training workflows, supports multi-GPU/TPU orchestration, and implements best practices for neural network training. Features include LightningModules, Trainers, data pipelines, callbacks, logging (W&B, TensorBoard), and distributed training strategies like DDP and FSDP for scalable training.

This skill covers setting up infrastructure for fine-tuning Large Language Models (LLMs) using techniques like QLoRA, LoRA, and full fine-tuning. It includes configuring distributed training with DeepSpeed or FSDP, utilizing frameworks such as Hugging Face TRL and Axolotl, and implementing alignment methods like RLHF and DPO. Suitable for domain-specific model adaptation and production deployment.

Train and fine-tune transformer language models with Hugging Face TRL via CLI. Covers SFT, DPO, GRPO, RLOO, KTO and reward model training, plus LoRA adapters, YAML configs and distributed multi-GPU runs.