Skill UI

A comprehensive framework for rigorously testing and benchmarking LLM agents. It moves beyond simple pass/fail checks to assess complex behaviors, reliability metrics, and capability consistency across multiple runs. Ideal for production monitoring and identifying subtle failure modes in advanced AI agents.

This is a high-level Software Development Kit (SDK) designed for managing the full lifecycle of AI projects within Azure AI Foundry. Built for Java, it provides robust client interfaces for core AI operations, including managing connections to Azure resources, uploading and managing datasets, creating and updating search indexes, running model evaluations, and handling deployments. It streamlines complex AI workflow orchestration.

This is a high-level Software Development Kit (SDK) designed for managing the full lifecycle of AI projects within Azure AI Foundry. Built for Java, it provides robust client interfaces for core AI operations, including managing connections to Azure resources, uploading and managing datasets, creating and updating search indexes, running model evaluations, and handling deployments. It streamlines complex AI workflow orchestration.

This SDK provides a comprehensive Python client library for interacting with Azure AI Speech-to-Text services. It supports both real-time streaming transcription and robust batch processing of audio files. Users can leverage advanced features such as speaker diarization (identifying multiple speakers) and capturing precise timestamps, making it invaluable for automated meeting minute generation, podcast content processing, and live captioning solutions.

This SDK provides a comprehensive Python client library for interacting with Azure AI Speech-to-Text services. It supports both real-time streaming transcription and robust batch processing of audio files. Users can leverage advanced features such as speaker diarization (identifying multiple speakers) and capturing precise timestamps, making it invaluable for automated meeting minute generation, podcast content processing, and live captioning solutions.

This is the .NET SDK for managing Arize AI Observability and Evaluation resources within Azure Resource Manager (ARM). It provides programmatic access to create, read, update, and delete AI organization resources. It is essential for MLOps professionals and developers who need to monitor, track, and evaluate the performance of AI models deployed on Azure.

Automatically evaluates Claude Code sessions at the Stop hook, identifies reusable patterns, and stores them as learned skills for future reuse, helping you capture troubleshooting steps, debugging techniques, and project conventions without manual effort.

Formal evaluation framework for Claude Code sessions that applies evaluation-driven development to define capability and regression checks, run deterministic and model graders, and track pass@k/pass^k reliability metrics before shipping.

A comprehensive toolkit for evaluating the performance of code generation models. It supports benchmarking against industry standards like HumanEval, MBPP, and MultiPL-E across multiple languages. Use this tool to quantitatively compare different LLM coding abilities and measure code synthesis quality using pass@k metrics.

Evaluates large language models across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) with standardized prompts and metrics, making it easy to benchmark HuggingFace or API models, compare releases, and track training progress.

Provides comprehensive guidance on building robust evaluation frameworks for complex agent systems. It addresses the inherent challenges of non-determinism and multiple valid paths, enabling systematic performance testing, validating context engineering choices, and measuring continuous improvements across multiple quality dimensions (e.g., factual accuracy, tool efficiency).

Track and improve production LLMs with Langfuse observability: trace API calls, manage prompts, score evaluations, integrate with LangChain/LlamaIndex/OpenAI, and keep dashboards on cost, latency, and quality.