Skill UI

Provides comprehensive guidance on building robust evaluation frameworks for complex agent systems. It addresses the inherent challenges of non-determinism and multiple valid paths, enabling systematic performance testing, validating context engineering choices, and measuring continuous improvements across multiple quality dimensions (e.g., factual accuracy, tool efficiency).

A comprehensive skill set for building, deploying, and managing production-grade Machine Learning systems. Expertise spans modern frameworks (PyTorch, TensorFlow), model serving using Kubernetes, advanced feature engineering (Feast, Tecton), and robust MLOps practices (monitoring, A/B testing, CI/CD). Focuses on delivering scalable, reliable, and business-value-driven AI solutions.

This skill specializes in designing, optimizing, and evaluating prompts for Large Language Models (LLMs). It allows users to generate optimized prompt templates, structured output schemas (e.g., JSON mode, function calling), and comprehensive test suites. Use this skill when building new LLM applications, implementing complex reasoning patterns like Chain-of-Thought, defining system guardrails, or building robust evaluation frameworks to measure and improve model performance.

This skill is designed for running comprehensive evaluations of Hugging Face Hub models on local hardware. It supports multiple evaluation frameworks, including `inspect-ai` and `lighteval`, and provides flexibility in backend selection (e.g., `vllm`, Transformers, or `accelerate`) to optimize for performance and compatibility. Use this for local GPU benchmarking, smoke testing, and backend selection, but note that it does not handle remote job orchestration or result publishing.

This skill enables the creation of robust evaluation frameworks for AI agent systems. It is used for systematically testing agent performance, validating context engineering choices, and measuring improvements over time. The approach accounts for non-determinism and multiple valid paths, utilizing multi-dimensional rubrics, LLM-as-judge methods, and human evaluation to ensure quality gates and catch regressions before deployment.

This skill guides users on running evaluations against models on the Hugging Face Hub using local hardware. It covers using inspect-ai and lighteval frameworks with local inference backends like vllm, Transformers, or accelerate. It includes task selection, smoke testing, and hardware guidance for different model sizes. It does not cover remote orchestration or publishing results to the community workflow.

This skill enables building production-ready machine learning systems using modern frameworks like PyTorch and TensorFlow. It covers model serving, feature engineering, A/B testing, monitoring, and MLOps practices. Focuses on scalable, reliable, and efficient ML infrastructure for delivering business value in production environments.