Skill UI

This skill provides interactive, periodic monitoring of deep learning training runs. It continuously checks critical metrics—such as loss trends, evaluation scores, and gradient norms—against defined baselines to detect signs of divergence, NaN values, plateaus, or general instability. It delivers a structured report and recommends whether to continue, wait, or stop the training process, ensuring computational resources are not wasted on failing models.

This skill provides comprehensive management capabilities for AI Integrations within the Arize platform. Users can create, list, retrieve, and update credentials for various Large Language Model (LLM) providers, including OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, and Vertex AI. These integrations are crucial for enabling Arize's advanced monitoring and evaluation features that require invoking external LLMs.

This skill provides comprehensive commands to manage the entire AI model evaluation lifecycle using Arize. Users can list, retrieve metadata, and export results from experiments. It is ideal for comparing multiple model runs, benchmarking performance against a dataset, and analyzing model accuracy using structured evaluation workflows (CRUD operations).

This skill provides comprehensive commands for creating, managing, versioning, and querying structured datasets within the Arize platform. It covers dataset CRUD operations, appending examples, and exporting data in various formats (JSON, CSV, Parquet) using the `ax` CLI. Ideal for preparing test data, generating evaluation examples, or retrieving golden datasets for LLM development and model testing.

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

This skill provides a comprehensive framework for optimizing, improving, and debugging Large Language Model (LLM) prompts. It leverages production trace data, system evaluations, and human annotations to extract prompts from spans. Users can run a data-driven optimization loop, ensuring the AI responses meet desired quality standards. Ideal for prompt tuning, system prompt refinement, and improving overall AI output quality.

This skill guides the end-to-end process of building automated evaluation pipelines for Python applications powered by Large Language Models (LLMs). It teaches how to define evaluation criteria, instrument the application, build golden datasets, and run real-world evaluations using tools like pixie, ensuring the app's logic (not just the LLM) is rigorously tested. Ideal for QA, benchmarking, and improving LLM-based services.

A comprehensive resource detailing the use of scikit-learn, the industry-standard Python library for classical machine learning. Use this skill for the entire ML lifecycle, including supervised tasks (classification, regression), unsupervised tasks (clustering, dimensionality reduction), data preprocessing, model evaluation, and building production-ready pipelines.

A comprehensive evaluation suite designed to benchmark the performance of Large Language Models (LLM) and Vision-Language Models (VLM) specifically for home security applications. It covers 143 diverse tests across 16 critical domains, including tool use, prompt injection resistance, contextual reasoning, alert triage, and detailed scene analysis, ensuring robustness for smart home AI systems.

This harness provides comprehensive, reproducible evaluation pipelines for complex LLM chains and agents (LangChain/LangGraph 1.0). It integrates golden dataset management, LangSmith evaluation, RAGAS metrics, deepeval LLM-as-judge, and structured agent trajectory analysis. Use it when establishing quality benchmarks for new chains, diagnosing performance regressions after model switches, or implementing CI/CD gates to prevent quality drops.

This tool provides a local development loop for integrating with Together AI's OpenAI-compatible API. It mocks key functionalities like chat completions, embeddings, and model listing, allowing developers to build and test AI applications rapidly without consuming live API credits. Developers can toggle between mock mode for rapid iteration and live mode for final model evaluation, ensuring a seamless workflow.

This comprehensive canvas guides teams through developing robust AI-powered features. It ensures product decisions are rigorous by covering problem definition, model selection (LLM, RAG, etc.), data governance, evaluation metrics, user experience design (including fallbacks), and responsible AI compliance. Ideal for assessing AI readiness and preventing common product failures.