Skill UI

A comprehensive framework for rigorously testing and benchmarking LLM agents. It moves beyond simple pass/fail checks to assess complex behaviors, reliability metrics, and capability consistency across multiple runs. Ideal for production monitoring and identifying subtle failure modes in advanced AI agents.

This skill provides a comprehensive solution for implementing observability for Customer.io integrations. It covers setting up Prometheus metrics (latency, request counts, bounces), structured JSON logging with PII redaction, and OpenTelemetry tracing. Use it when building robust, production-grade monitoring dashboards and alerting systems for Customer.io APIs.

Implements a comprehensive observability stack for Deepgram integrations. It utilizes Prometheus metrics for tracking request rates, latency, and costs, OpenTelemetry for distributed tracing, Pino for structured logging, and AlertManager rules for proactive alerting. Essential for production monitoring, dashboard setup, and performance analysis of Deepgram API usage.

A comprehensive checklist and guide for deploying Deepgram integrations into a production environment. It covers critical areas including robust authentication practices, system resilience design (e.g., backoff strategies, circuit breakers), performance optimization (singleton clients, audio preprocessing), security measures (PII redaction), and mandatory monitoring setup (Prometheus metrics and alert rules). Essential for any go-live phase.

This skill provides a comprehensive framework for implementing observability for Documenso integrations. It wraps API calls using proxies to track key metrics (request counts, latency, error rates) and sets up structured logging and health checks. Use this when setting up monitoring stacks (like Prometheus/Grafana) or debugging production issues, ensuring detailed operational metrics are available.

Provides a comprehensive framework for implementing observability in Evernote integrations. It covers metrics collection (Prometheus), structured JSON logging, distributed tracing (OpenTelemetry), and defining health/readiness endpoints. Use this when setting up production monitoring, alerting, and performance tracking for mission-critical Evernote APIs.

A comprehensive guide and code implementation for setting up observability, metrics, and alerting for Exa search integrations. This skill tracks crucial KPIs such as search latency (P50/P95), error rates, result usage, and cache hit ratios. It is essential for monitoring search quality, debugging performance bottlenecks, and managing API resource usage in production environments.

A comprehensive solution for implementing robust observability for Gamma API integrations. This pattern provides structured logging, detailed metrics tracking (including request counts, error rates, latency, and credit consumption), and health check endpoints. It is essential for monitoring the reliability and performance of your application when using the Gamma API in a production environment.

This skill guides you through designing and implementing production-grade Grafana dashboards for comprehensive system observability. Learn to visualize critical metrics, track key performance indicators (KPIs), monitor infrastructure health (using principles like RED/USE), and build actionable alerts using Prometheus data sources. Ideal for DevOps engineers and Site Reliability Engineers (SREs).

This skill provides comprehensive observability tools for Groq API integrations. It instruments the client to track critical performance metrics such as Time to First Token (TTFT), total latency, token throughput, and estimated cost. Furthermore, it sets up Prometheus exporters and alert rules to monitor rate limit status and detect performance degradation in real-time, ensuring robust production monitoring.

This runbook provides a structured guide for responding to critical Guidewire production incidents. It covers the entire lifecycle of incident management, including initial triage (checking monitoring, API errors, and batch logs), diagnosing performance degradation (JVM metrics and query performance), implementing mitigation steps, and proper escalation procedures to Guidewire Support.

A comprehensive guide to setting up production-grade observability for LangChain applications. It integrates LangSmith tracing for zero-code debugging, custom callbacks for detailed metric collection (latency, token usage, error rate), and Prometheus exporters for robust alerting. This allows developers to monitor LLM pipelines, track performance metrics, and manage operational health in real-time.