Skill UI

High-performance RLHF framework that combines Ray, vLLM, and ZeRO-3 to accelerate PPO/GRPO/RLOO/DPO training for 7B-70B+ models on distributed GPU clusters, providing a single-stack workflow for reward models and policy optimization.

Quantizes HuggingFace LLMs to 8-bit or 4-bit with bitsandbytes, cutting memory by up to 75% while keeping accuracy, and supports QLoRA fine-tuning plus 8-bit optimizers for faster, memory-efficient training.

SimPO (Simple Preference Optimization) is a state-of-the-art, reference-free method designed for aligning Large Language Models (LLMs) using human preference data. It is an efficient alternative to DPO and PPO, notably outperforming DPO without requiring a separate reference model. It is ideal for practitioners who need faster, simpler, and more resource-efficient fine-tuning for preference alignment.

Guides post-training of GLM-class LLMs with Megatron-LM and SGLang, focusing on GRPO rollout generation, custom data buffers, and tight RL scaling workflows for research and production use.

Provides production-ready Megatron-Core workflows for training 2B–462B parameter LLMs with tensor/pipeline/expert/context parallelism, maximizing GPU efficiency on H100 clusters.

Guides training large language models with Volcano Engine's verl library, covering PPO/GRPO workflows, distributed backends, multi-turn rollouts, reward configs, critic-enabled PPO, and production deployments using Ray workers.

A comprehensive skill for fine-tuning and training large language and vision models using advanced techniques like TRL (SFT, DPO, GRPO) or Unsloth. This process runs entirely on managed Hugging Face cloud GPUs, eliminating the need for local GPU setup. It supports modern training workflows, model conversion (GGUF for local deployment), and ensures seamless artifact storage to the Hugging Face Hub.

This workflow guides users through the complete lifecycle of customizing large language models (LLMs). It covers uploading training data, initiating fine-tuning jobs, monitoring training progress, managing model versions, and finally deploying the optimized model to a dedicated, autoscaling inference endpoint using the Together AI API. Ideal for production-level customization.

This skill provides expert capabilities for training and fine-tuning transformer language models using the TRL (Transformers Reinforcement Learning) library. It supports state-of-the-art post-training methods, including Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), Group Relative Policy Optimization (GRPO), and Reward Model training. Use this skill to align and customize foundation models for advanced tasks via CLI commands.