Skill UI

Provides patterns and techniques for building robust evaluation loops for AI agents. This skill guides the implementation of self-critique, evaluator-optimizer pipelines, and test-driven refinement workflows. Use it to significantly enhance the accuracy and quality of complex outputs like code, reports, and analyses by implementing structured, iterative improvement cycles (Generate → Evaluate → Critique → Refine).

This skill provides comprehensive management capabilities for AI Integrations within the Arize platform. Users can create, list, retrieve, and update credentials for various Large Language Model (LLM) providers, including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, and Vertex AI. These integrations are crucial for enabling Arize's advanced monitoring and evaluation features that require invoking external LLMs.

This skill provides comprehensive commands to manage the entire AI model evaluation lifecycle using Arize. Users can list, retrieve metadata, and export results from experiments. It is ideal for comparing multiple model runs, benchmarking performance against a dataset, and analyzing model accuracy using structured evaluation workflows (CRUD operations).

Conducts comprehensive AI prompt safety reviews that analyze bias, misinformation, injection, and privacy risks while delivering effectiveness evaluations, security guidance, and best-practice recommendations for prompt engineers.

This skill provides comprehensive commands for creating, managing, versioning, and querying structured datasets within the Arize platform. It covers dataset CRUD operations, appending examples, and exporting data in various formats (JSON, CSV, Parquet) using the `ax` CLI. Ideal for preparing test data, generating evaluation examples, or retrieving golden datasets for LLM development and model testing.

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

This skill provides a comprehensive framework for optimizing, improving, and debugging Large Language Model (LLM) prompts. It leverages production trace data, system evaluations, and human annotations to extract prompts from spans. Users can run a data-driven optimization loop, ensuring the AI responses meet desired quality standards. Ideal for prompt tuning, system prompt refinement, and improving overall AI output quality.

This skill guides the end-to-end process of building automated evaluation pipelines for Python applications powered by Large Language Models (LLMs). It teaches how to define evaluation criteria, instrument the application, build golden datasets, and run real-world evaluations using tools like pixie, ensuring the app's logic (not just the LLM) is rigorously tested. Ideal for QA, benchmarking, and improving LLM-based services.

The Phoenix CLI is a powerful tool for debugging complex LLM and agent workflows. It allows users to fetch detailed traces, spans, and sessions, analyze errors, and structure findings using open and axial coding techniques. It is essential for investigating complex LLM failures, building robust failure taxonomies, and guiding the next steps in application evaluation after initial instrumentation.

A specialized toolkit designed for generating and validating research-only artifacts for Clinical Decision Support (CDS) systems. It supports evidence-profile generation, cohort analysis, survival modeling, biomarker evaluation, and governance documentation. IMPORTANT: This skill is strictly for aggregate research and documentation; it must never be used for patient care, diagnosis, or live clinical operations.

Polars is a lightning-fast DataFrame library built for Python and Rust, leveraging Apache Arrow. It provides an expression-based API and lazy evaluation framework for highly efficient data manipulation. It is ideal for complex data pipelines, large-scale ETL tasks, and migrating from Pandas due to its superior performance, parallel execution capabilities, and optimized query planning.

This guide provides comprehensive, version-aware documentation for using PufferLib in reinforcement learning pipelines. It details best practices for adapting Gymnasium/PettingZoo environments, managing different PufferLib versions (3.0.0 vs 4.0 source), and executing safe, reproducible training and evaluation routines. Key focus areas include environment contract validation, sandboxing native builds, and maintaining data provenance.