Skill UI

Activation-aware weight quantization that compresses 7B-70B models to 4-bit with minimal accuracy loss, delivering ~3x inference speedups and memory reductions for constrained GPU deployments while integrating with vLLM and Marlin kernels.

Orchestrates PyTorch-native TorchTitan 4D parallelism (FSDP2, TP, PP, CP) for large-scale LLM pretraining, scaling from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing to boot Llama 3.1, DeepSeek V3, or custom models.

FAISS (Facebook AI Similarity Search) is a library designed for efficient similarity search and clustering of dense vectors. It supports indexing and searching on billions of vectors at scale, featuring GPU acceleration and multiple index types (Flat, IVF, HNSW, PQ) to balance speed and accuracy. It is essential for building high-performance, large-scale applications requiring fast k-NN retrieval or pure vector similarity matching, especially in fields like RAG and recommendation systems.

GPTQ enables post-training 4-bit quantization for large LLMs, delivering up to 4× memory reduction and 3-4× faster inference with under 2% perplexity loss on consumer GPUs through AutoGPTQ, PEFT, and Transformer integrations.

Automates GPU resource optimization for ML deployments by guiding best practices, generating production-ready configurations, and validating setups for model serving, MLOps pipelines, and monitoring.

Execute complex and compute-intensive workloads directly on fully managed Hugging Face cloud infrastructure. This skill eliminates the need for local setup, allowing users to run jobs using cloud CPUs, GPUs, or TPUs. Ideal for large-scale data processing, batch inference, reproducible model training, and scheduled automation tasks, while persisting results to the Hugging Face Hub.

Playbook for running ML training and inference on Lambda Labs’ on-demand GPU cloud, covering when to choose it, console/SSH/API workflows, available GPUs, pricing, and persistent storage with Lambda Stack pre-installed.

llama.cpp is a pure C/C++ library designed for efficient LLM inference, optimized to run on resource-constrained environments. It excels in scenarios where high-end NVIDIA GPUs (CUDA) are unavailable, making it ideal for Apple Silicon Macs, AMD/Intel GPUs, or edge deployment devices like Raspberry Pi. It supports GGUF quantization formats, enabling significant memory reduction and speed improvements for local, cross-platform AI applications.

Modal Serverless GPU provides on‑demand, pay‑per‑second access to T4 through H200 GPUs, letting you deploy ML models as auto‑scaling APIs, run batch jobs, and prototype without managing infrastructure.

NVIDIA’s NeMo Guardrails adds runtime safety rails to LLM apps, covering jailbreak detection, hallucination and fact-checking, PII filtering, and toxicity checks via Colang 2.0 rules and integrations for production use on T4 GPUs.

Guides researchers through nnsight’s tracing and activation APIs so you can inspect or modify PyTorch internals locally or via NDIF-powered remote runs on 70B+ foundation models without local GPUs.

High-performance RLHF framework that combines Ray, vLLM, and ZeRO-3 to accelerate PPO/GRPO/RLOO/DPO training for 7B-70B+ models on distributed GPU clusters, providing a single-stack workflow for reward models and policy optimization.