Skill UI

A systematic workflow designed to significantly improve the reliability and performance of existing AI agents. It covers detailed performance analysis, including gathering metrics (e.g., task success rate, latency), classifying failure modes, and advanced prompt engineering techniques (like Chain-of-Thought and Constitutional AI). The process culminates in structured A/B testing and controlled deployment, ensuring measurable, robust improvements.

Provides comprehensive guidance on building robust evaluation frameworks for complex agent systems. It addresses the inherent challenges of non-determinism and multiple valid paths, enabling systematic performance testing, validating context engineering choices, and measuring continuous improvements across multiple quality dimensions (e.g., factual accuracy, tool efficiency).

A comprehensive checklist for ensuring LangChain applications are robust, secure, and scalable for production environments. It covers critical areas including configuration management, advanced error handling, observability setup (e.g., LangSmith), performance optimization, security best practices (like prompt injection prevention and PII handling), rigorous testing, and zero-downtime deployment strategies. Use this guide when preparing for a major launch or auditing existing LLM systems.

Master comprehensive strategies for evaluating Large Language Model (LLM) applications. This skill covers automated metrics (BLEU, ROUGE, BERTScore), structured human evaluation dimensions, and advanced LLM-as-Judge techniques. Use it to systematically measure model performance, compare prompts, detect regressions, and build confidence in production systems.

This skill provides comprehensive guidance for building production-grade MLOps pipelines. It covers the entire machine learning lifecycle, from initial data ingestion and rigorous feature engineering to model training, advanced validation (A/B testing, performance drift), and robust deployment strategies (Canary, Blue-green). It enables the design and orchestration of scalable, reproducible, and observable ML systems using tools like Airflow and Kubeflow.

This skill is designed for running comprehensive evaluations of Hugging Face Hub models on local hardware. It supports multiple evaluation frameworks, including `inspect-ai` and `lighteval`, and provides flexibility in backend selection (e.g., `vllm`, Transformers, or `accelerate`) to optimize for performance and compatibility. Use this for local GPU benchmarking, smoke testing, and backend selection, but note that it does not handle remote job orchestration or result publishing.

Phoenix Evals enables developers to build and run evaluators for AI and LLM applications. It supports both code-based and LLM-based evaluation methods, including RAG systems. Users can create datasets, run experiments, validate evaluator accuracy, and implement production guardrails. Available in Python and TypeScript, it helps ensure model performance and reliability through systematic testing and human validation.

This harness provides comprehensive, reproducible evaluation pipelines for complex LLM chains and agents (LangChain/LangGraph 1.0). It integrates golden dataset management, LangSmith evaluation, RAGAS metrics, deepeval LLM-as-judge, and structured agent trajectory analysis. Use it when establishing quality benchmarks for new chains, diagnosing performance regressions after model switches, or implementing CI/CD gates to prevent quality drops.

This comprehensive skill guides users through the entire lifecycle of developing AI agent skills. From initial concept (Understand) to structured design, writing the core skill logic, rigorous testing, and final performance optimization. It ensures skills are robust, accurate, and highly triggerable across any domain, including engineering, content creation, and research.

This comprehensive benchmark suite evaluates the performance of Vision-Language Models (VLM) on complex video anomaly detection tasks using the SmartHome-Bench dataset. It covers a wide range of real-world smart home scenarios, including security breaches, elderly fall detection, and unusual pet behavior. Unlike single-frame analysis, this benchmark requires deep multi-frame video understanding, making it a robust tool for testing the advanced AI capabilities of video monitoring systems.

This skill enables systematic improvement of existing AI agents by analyzing performance metrics, identifying failure modes, and applying prompt engineering techniques. It involves establishing baselines, optimizing prompts through chain-of-thought and few-shot examples, refining role definitions, and validating changes through structured testing and A/B comparisons to ensure reliability and efficiency.

Provides comprehensive guidance on building robust evaluation frameworks for complex agent systems. It addresses the inherent challenges of non-determinism and multiple valid paths, enabling systematic performance testing, validating context engineering choices, and measuring continuous improvements across multiple quality dimensions (e.g., factual accuracy, tool efficiency).