Skill UI

Automated guidance for model explainability tooling within ML training workflows, giving step-by-step best practices, production-ready code, and validation for experiments whenever users mention explainability tool needs.

Combines fine-tuned models with Mergekit to blend math, coding, and chat expertise without retraining, covering SLERP, TIES, DARE, Task Arithmetic, and linear merges for rapid experimentation and deployment.

Compress large language models quickly using Wanda, SparseGPT, and N:M pruning to enforce hardware-friendly sparsity, enabling faster inference and deployment without retraining.

This skill provides comprehensive guidance on training Mixture of Experts (MoE) models, including architectures like Mixtral and DeepSeek-V3. It details techniques for achieving large-scale model capacity with significantly reduced computational costs (e.g., 5x reduction vs. dense models). Topics covered include top-k routing, load balancing, expert parallelism, and integration with frameworks like DeepSpeed and HuggingFace for efficient, resource-constrained training.

nanoGPT is a simplified, minimalist implementation designed specifically for educational purposes. It reproduces the core architecture of GPT-2 (124M) using clean, hackable code, allowing users to understand the entire transformer pipeline from scratch. The framework supports full workflows, including data preparation on various datasets (e.g., Shakespeare, OpenWebText), model training, and text generation, making it ideal for students and researchers learning NLP and deep learning principles.

High-performance RLHF framework that combines Ray, vLLM, and ZeRO-3 to accelerate PPO/GRPO/RLOO/DPO training for 7B-70B+ models on distributed GPU clusters, providing a single-stack workflow for reward models and policy optimization.

Flash Attention optimizer delivers 2-4x speedup and 10-20x memory savings for transformer attention, ideal when training or running long-context models that hit GPU memory or latency limits; supports PyTorch native scaled_dot_product_attention, the flash-attn library, H100 FP8 precision, and sliding-window attention.

Auto-activating ML training skill that kicks in whenever you mention optuna study creator, then guides you through data prep, model training, hyperparameter tuning, and experiment tracking with production-ready code and validations.

Guides adding PyTorch FSDP2’s fully_shard integration to training loops, covering correct distributed init, bottom-up sharding, mixed precision/offload policies, DTensor-aware optimizers, and distributed checkpointing so oversized models or DeviceMesh sharding goals work reliably.

Automates PyTorch training workflows by guiding data preparation, model configuration, hyperparameter tuning, and experiment tracking while following industry best practices for production readiness.

Pyvene is a declarative framework designed for performing causal interventions on PyTorch neural networks. It allows researchers to conduct advanced experiments such as activation patching, causal tracing (ROME-style), and interchange intervention training (IIT). Use this library when you need to test causal hypotheses about model behavior, deeply interpret model components, or ensure reproducibility in advanced AI research.

Quantizes HuggingFace LLMs to 8-bit or 4-bit with bitsandbytes, cutting memory by up to 75% while keeping accuracy, and supports QLoRA fine-tuning plus 8-bit optimizers for faster, memory-efficient training.