Skill UI

A comprehensive workflow for implementing robust LLM evaluation using Langfuse. This skill guides users through key processes including scoring traces (numeric, categorical, boolean), collecting user feedback, managing versioned prompts, creating structured datasets, running automated experiments, and implementing LLM-as-a-Judge patterns for advanced quality assurance and system benchmarking.

This guide details how to establish comprehensive observability for LLM operations using Langfuse. It covers setting up Prometheus metrics (tracking traces, generation duration, and token usage) and visualizing them in Grafana dashboards. Ideal for monitoring application health, optimizing costs, and tracking performance over time.

Fetch execution traces from LangSmith Studio to inspect LangChain/LangGraph agent runs, analyze errors, tool calls, memory usage, and performance, and export sessions for deeper diagnostics using the langsmith-fetch CLI.

This skill provides comprehensive tools to download, export, and deeply inspect existing Arize traces and spans. It is essential for understanding the full execution flow of complex LLM applications, debugging runtime errors, and performing root-cause analysis. Users can retrieve detailed data filtered by Trace ID, Span ID, or Session ID, enabling thorough analysis of conversational or multi-step agent behaviors.

The Phoenix CLI is a powerful tool for debugging complex LLM and agent workflows. It allows users to fetch detailed traces, spans, and sessions, analyze errors, and structure findings using open and axial coding techniques. It is essential for investigating complex LLM failures, building robust failure taxonomies, and guiding the next steps in application evaluation after initial instrumentation.

Utilize advanced media intelligence to analyze audio, image, video, and text for synthetic manipulation and AI-generated content. This skill detects deepfakes, traces the original AI synthesis source, applies and detects watermarks, verifies speaker identity, and provides comprehensive media provenance analysis, ensuring media authenticity claims are rigorously supported.