Skill UI

This comprehensive workflow guides the end-to-end implementation of Retrieval-Augmented Generation (RAG) systems. It covers critical stages, including selecting optimal embedding models, setting up vector databases, designing chunking strategies, implementing hybrid retrieval, integrating Large Language Models (LLMs), and performing rigorous evaluation. Use this when building knowledge-grounded AI applications or advanced semantic search systems.

A comprehensive, model-agnostic toolkit for advanced LLM development. It enables iterative prompt optimization through static analysis (token usage, clarity), measures critical RAG metrics (relevance, faithfulness, coverage), and validates complex agent workflows and configurations. Essential for rigorous LLM quality assurance and cost management.

Automates splitting datasets into training, validation, and test subsets by generating and running Python code so you can quickly prepare data for machine learning model training and evaluation.

Scaffolds production-ready generative AI agents on Vertex AI with Gemini models, multi-modal tools, RAG, and Cloud Run/Endpoint deployment, including observability and evaluation for enterprise scenarios.

Guides developers through scikit-learn workflows for classical supervised and unsupervised tasks, data preprocessing, evaluation schemes, hyperparameter tuning, and end-to-end pipelines when building interpretable production models.

Toolkit for Python-based survival and time-to-event modeling with scikit-survival, supporting Cox models, ensemble learners, SVMs, and evaluation metrics for censored data.

A comprehensive guide to building production-grade evaluation systems for Large Language Models (LLMs). This skill covers advanced methodologies, including Direct Scoring and Pairwise Comparison, alongside critical techniques for mitigating systematic biases (e.g., Position Bias, Length Bias). Learn how to select appropriate metrics and structure prompts to ensure reliable, consistent, and objective quality assessment across various AI applications.

This comprehensive guide outlines a rigorous, human-in-the-loop engineering workflow for leveraging advanced AI agents to complete complex implementation tasks. It mandates defining clear completion criteria, decomposing work into manageable units, and strategically routing models (Haiku, Sonnet, Opus) based on task complexity. The workflow emphasizes continuous evaluation, regression testing, security review of invariants and edge cases, and strict cost discipline to ensure robust and reliable software development using AI assistance.

This skill provides comprehensive tools for documenting and managing model performance on Hugging Face. It supports multiple methods for adding structured evaluation data, including extracting tables from README content, importing benchmark scores via APIs (like Artificial Analysis), and executing high-performance custom evaluations using advanced backends such as vLLM or lighteval. It ensures that all metadata is structured and compatible with model leaderboards and industry standards.

This framework analyzes foundational design principles—such as affordances, signifiers, and conceptual models—to create products that are highly intuitive and discoverable. It guides analysis by bridging the 'Gulf of Execution' and the 'Gulf of Evaluation.' Use this when diagnosing confusing user interfaces, poor usability, or needing to ensure a product is robustly human-centered and easy to understand.

A lightweight CLI tool for systematically comparing various coding AI agents (such as Claude Code, Aider, and Codex) head-to-head. It measures crucial metrics—pass rate, API cost, execution time, and consistency—on reproducible tasks defined via YAML manifests. Use this to objectively select the best agent for your codebase or to validate model updates before production deployment.

This skill is designed for performing local, on-premise evaluations and benchmarking of models hosted on the Hugging Face Hub. It facilitates running specialized evaluation frameworks like `inspect-ai` and `lighteval` against local hardware. Users can select optimal inference backends, including `vllm` for high throughput, or fall back to Hugging Face Transformers or `accelerate`, ensuring comprehensive smoke testing and task selection for LLMs. It focuses purely on the local execution process, separate from remote jobs or publishing workflows.