Skill UI

Use Clarvia AEO (Agent Experience Optimization) to score any MCP server, API, or CLI tool for agent-readiness. It evaluates 15,400+ AI tools across four dimensions—API accessibility, data structuring, agent compatibility, and trust signals—ensuring the components are robust enough for advanced AI agent workflows and dynamic selection processes.

This skill is designed for running comprehensive evaluations of Hugging Face Hub models on local hardware. It supports multiple evaluation frameworks, including `inspect-ai` and `lighteval`, and provides flexibility in backend selection (e.g., `vllm`, Transformers, or `accelerate`) to optimize for performance and compatibility. Use this for local GPU benchmarking, smoke testing, and backend selection, but note that it does not handle remote job orchestration or result publishing.

This skill interactively monitors AI training sessions by periodically checking WandB or fallback logs for anomalies like NaN, divergence, or plateaus. It analyzes loss trends and evaluation metrics to determine whether to continue, wait, or stop the training run, ensuring training quality and resource efficiency.

Generates structured peer-review quality critiques of academic papers, covering summaries, methodology scrutiny, contribution evaluation, literature positioning, and recommendations across domains whenever a user requests reviews, analyses, or summaries of research publications.

Provides patterns and techniques for building robust evaluation loops for AI agents. This skill guides the implementation of self-critique, evaluator-optimizer pipelines, and test-driven refinement workflows. Use it to significantly enhance the accuracy and quality of complex outputs like code, reports, and analyses by implementing structured, iterative improvement cycles (Generate → Evaluate → Critique → Refine).

This skill provides comprehensive management capabilities for AI Integrations within the Arize platform. Users can create, list, retrieve, and update credentials for various Large Language Model (LLM) providers, including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, and Vertex AI. These integrations are crucial for enabling Arize's advanced monitoring and evaluation features that require invoking external LLMs.

Conducts comprehensive AI prompt safety reviews that analyze bias, misinformation, injection, and privacy risks while delivering effectiveness evaluations, security guidance, and best-practice recommendations for prompt engineers.

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

This skill provides a comprehensive framework for optimizing, improving, and debugging Large Language Model (LLM) prompts. It leverages production trace data, system evaluations, and human annotations to extract prompts from spans. Users can run a data-driven optimization loop, ensuring the AI responses meet desired quality standards. Ideal for prompt tuning, system prompt refinement, and improving overall AI output quality.

The Phoenix CLI is a powerful tool for debugging complex LLM and agent workflows. It allows users to fetch detailed traces, spans, and sessions, analyze errors, and structure findings using open and axial coding techniques. It is essential for investigating complex LLM failures, building robust failure taxonomies, and guiding the next steps in application evaluation after initial instrumentation.

Phoenix Evals enables developers to build and run evaluators for AI and LLM applications. It supports both code-based and LLM-based evaluation methods, including RAG systems. Users can create datasets, run experiments, validate evaluator accuracy, and implement production guardrails. Available in Python and TypeScript, it helps ensure model performance and reliability through systematic testing and human validation.

Applies the DHDNA framework to analyze reasoning and writing patterns in text. It provides a rubric for cognitive-style reflection and thinking-pattern profiles without validating psychological traits. Useful for evaluating textual evidence across twelve dimensions like analytical depth and ethical reasoning.