Skill UI

Automates creation of A/B test configurations for ML deployment, guiding users through best practices, code samples, and validation steps so production models are served with monitoring and rollout control.

A comprehensive framework for rigorously testing and benchmarking LLM agents. It moves beyond simple pass/fail checks to assess complex behaviors, reliability metrics, and capability consistency across multiple runs. Ideal for production monitoring and identifying subtle failure modes in advanced AI agents.

This is the .NET SDK for managing Arize AI Observability and Evaluation resources within Azure Resource Manager (ARM). It provides programmatic access to create, read, update, and delete AI organization resources. It is essential for MLOps professionals and developers who need to monitor, track, and evaluate the performance of AI models deployed on Azure.

Provides automated guidance for setting up canary deployments in ML production environments, covering model serving, MLOps pipelines, monitoring, and configuration validation.

Comprehensive guide for configuring Bring Your Own Key (BYOK) for major AI providers, including OpenAI, Anthropic, Google Gemini, Azure, and custom endpoints. This allows users to bypass default quotas, manage costs directly with the source provider, and utilize a wide range of specialized models within the Cursor IDE. It covers setup steps, security best practices, and cost monitoring strategies.

Deploying Machine Learning Models automates serving trained models to production by analyzing requirements, generating API and infrastructure code, and orchestrating deployment workflows with validation, error handling, and monitoring baked in.

Generates complete, production-ready Google Cloud Platform (GCP) code examples and infrastructure templates. It spans various modern AI patterns, including ADK agents, Genkit RAG flows, and multimodal analysis using Vertex AI and Gemini. The output includes runnable code, deployment configuration (Terraform/IaC), security best practices (IAM, VPC-SC), and monitoring setup, ensuring enterprise-grade reliability and cost optimization.

Automates GPU resource optimization for ML deployments by guiding best practices, generating production-ready configurations, and validating setups for model serving, MLOps pipelines, and monitoring.

Automates inference latency profiler tasks in ML deployment scenarios, offering step-by-step guidance on model serving, MLOps pipelines, monitoring, and production optimization, generating production-ready code and validating outputs against best practices.

Implement comprehensive end-to-end tracing for Large Language Model (LLM) calls, chains, and complex agents using Langfuse. This workflow demonstrates techniques like OpenAI drop-in wrappers, manual span creation for RAG pipelines, tracking streamed responses, and integrating with various LLM providers (OpenAI, Anthropic). Use this to monitor latency, track token usage, and debug complex AI features in production.

Learn how to monitor, analyze, and optimize the expenditure of Large Language Models (LLMs) using Langfuse's advanced analytics suite. This guide covers techniques like automatic token tracking, programmatic cost reporting via the Metrics API, and implementing smart model routing to automatically select the most cost-effective model based on task complexity and input length. Essential for any developer managing AI budgets and ensuring operational efficiency.

This skill guides the design and implementation of a complete, production-grade Machine Learning pipeline. It employs a multi-agent orchestration workflow, systematically covering all stages of the ML lifecycle: data ingestion and quality assurance, feature engineering, model training, optimization, and robust deployment (MLOps). It ensures built-in reproducibility, monitoring, and scalability.