Skill UI

A systematic workflow designed to significantly improve the reliability and performance of existing AI agents. It covers detailed performance analysis, including gathering metrics (e.g., task success rate, latency), classifying failure modes, and advanced prompt engineering techniques (like Chain-of-Thought and Constitutional AI). The process culminates in structured A/B testing and controlled deployment, ensuring measurable, robust improvements.

Provides comprehensive guidance on building robust evaluation frameworks for complex agent systems. It addresses the inherent challenges of non-determinism and multiple valid paths, enabling systematic performance testing, validating context engineering choices, and measuring continuous improvements across multiple quality dimensions (e.g., factual accuracy, tool efficiency).

A comprehensive checklist for ensuring LangChain applications are robust, secure, and scalable for production environments. It covers critical areas including configuration management, advanced error handling, observability setup (e.g., LangSmith), performance optimization, security best practices (like prompt injection prevention and PII handling), rigorous testing, and zero-downtime deployment strategies. Use this guide when preparing for a major launch or auditing existing LLM systems.

Master comprehensive strategies for evaluating Large Language Model (LLM) applications. This skill covers automated metrics (BLEU, ROUGE, BERTScore), structured human evaluation dimensions, and advanced LLM-as-Judge techniques. Use it to systematically measure model performance, compare prompts, detect regressions, and build confidence in production systems.

This skill provides comprehensive guidance for building production-grade MLOps pipelines. It covers the entire machine learning lifecycle, from initial data ingestion and rigorous feature engineering to model training, advanced validation (A/B testing, performance drift), and robust deployment strategies (Canary, Blue-green). It enables the design and orchestration of scalable, reproducible, and observable ML systems using tools like Airflow and Kubeflow.

This skill specializes in designing, optimizing, and evaluating prompts for Large Language Models (LLMs). It allows users to generate optimized prompt templates, structured output schemas (e.g., JSON mode, function calling), and comprehensive test suites. Use this skill when building new LLM applications, implementing complex reasoning patterns like Chain-of-Thought, defining system guardrails, or building robust evaluation frameworks to measure and improve model performance.

A comprehensive collection of battle-tested, structured prompts designed to elevate interactions with Large Language Models (LLMs). Covers advanced use cases including expert coding reviews, system architecture design, performance analysis, technical writing, and creative brainstorming, ensuring high-quality, actionable AI output.

This skill is designed for running comprehensive evaluations of Hugging Face Hub models on local hardware. It supports multiple evaluation frameworks, including `inspect-ai` and `lighteval`, and provides flexibility in backend selection (e.g., `vllm`, Transformers, or `accelerate`) to optimize for performance and compatibility. Use this for local GPU benchmarking, smoke testing, and backend selection, but note that it does not handle remote job orchestration or result publishing.

A comprehensive skill for the entire lifecycle of AI capability development. Use this tool to draft new skills from scratch, iteratively improve existing ones, and rigorously measure their performance. The process includes capturing user intent, defining test cases, running quantitative benchmarks, evaluating results, and optimizing the skill description for maximum triggering accuracy.

A comprehensive evaluation suite designed to benchmark the performance of Large Language Models (LLM) and Vision-Language Models (VLM) specifically for home security applications. It covers 143 diverse tests across 16 critical domains, including tool use, prompt injection resistance, contextual reasoning, alert triage, and detailed scene analysis, ensuring robustness for smart home AI systems.

This harness provides comprehensive, reproducible evaluation pipelines for complex LLM chains and agents (LangChain/LangGraph 1.0). It integrates golden dataset management, LangSmith evaluation, RAGAS metrics, deepeval LLM-as-judge, and structured agent trajectory analysis. Use it when establishing quality benchmarks for new chains, diagnosing performance regressions after model switches, or implementing CI/CD gates to prevent quality drops.

This comprehensive skill guides users through the entire lifecycle of developing AI agent skills. From initial concept (Understand) to structured design, writing the core skill logic, rigorous testing, and final performance optimization. It ensures skills are robust, accurate, and highly triggerable across any domain, including engineering, content creation, and research.