Skill UI

This skill provides comprehensive tools for documenting and managing model performance on Hugging Face. It supports multiple methods for adding structured evaluation data, including extracting tables from README content, importing benchmark scores via APIs (like Artificial Analysis), and executing high-performance custom evaluations using advanced backends such as vLLM or lighteval. It ensures that all metadata is structured and compatible with model leaderboards and industry standards.

This skill is designed for performing local, on-premise evaluations and benchmarking of models hosted on the Hugging Face Hub. It facilitates running specialized evaluation frameworks like `inspect-ai` and `lighteval` against local hardware. Users can select optimal inference backends, including `vllm` for high throughput, or fall back to Hugging Face Transformers or `accelerate`, ensuring comprehensive smoke testing and task selection for LLMs. It focuses purely on the local execution process, separate from remote jobs or publishing workflows.

This skill provides comprehensive management capabilities for AI Integrations within the Arize platform. Users can create, list, retrieve, and update credentials for various Large Language Model (LLM) providers, including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, and Vertex AI. These integrations are crucial for enabling Arize's advanced monitoring and evaluation features that require invoking external LLMs.

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

This skill provides a comprehensive framework for optimizing, improving, and debugging Large Language Model (LLM) prompts. It leverages production trace data, system evaluations, and human annotations to extract prompts from spans. Users can run a data-driven optimization loop, ensuring the AI responses meet desired quality standards. Ideal for prompt tuning, system prompt refinement, and improving overall AI output quality.

A comprehensive evaluation suite designed to benchmark the performance of Large Language Models (LLM) and Vision-Language Models (VLM) specifically for home security applications. It covers 143 diverse tests across 16 critical domains, including tool use, prompt injection resistance, contextual reasoning, alert triage, and detailed scene analysis, ensuring robustness for smart home AI systems.

This harness provides comprehensive, reproducible evaluation pipelines for complex LLM chains and agents (LangChain/LangGraph 1.0). It integrates golden dataset management, LangSmith evaluation, RAGAS metrics, deepeval LLM-as-judge, and structured agent trajectory analysis. Use it when establishing quality benchmarks for new chains, diagnosing performance regressions after model switches, or implementing CI/CD gates to prevent quality drops.

A rigorous, Chief AI Officer-level interrogation tool designed to pressure-test any plan involving artificial intelligence. Before deployment, it forces comprehensive review across crucial dimensions: evaluation metrics, hallucination risk, EU AI Act compliance, model architecture selection (API vs. build), cost economics, and required organizational roles. Essential for maintaining AI product safety and compliance.