Skill UI

A comprehensive guide to setting up production-grade observability for LangChain applications. It integrates LangSmith tracing for zero-code debugging, custom callbacks for detailed metric collection (latency, token usage, error rate), and Prometheus exporters for robust alerting. This allows developers to monitor LLM pipelines, track performance metrics, and manage operational health in real-time.

Track and improve production LLMs with Langfuse observability: trace API calls, manage prompts, score evaluations, integrate with LangChain/LlamaIndex/OpenAI, and keep dashboards on cost, latency, and quality.

This guide details how to establish comprehensive observability for LLM operations using Langfuse. It covers setting up Prometheus metrics (tracking traces, generation duration, and token usage) and visualizing them in Grafana dashboards. Ideal for monitoring application health, optimizing costs, and tracking performance over time.

Weights & Biases (W&B) automates experiment logging, real-time dashboards, hyperparameter sweeps, artifact tracking, and model registry/versioning for ML teams using PyTorch, TensorFlow, or HuggingFace workflows.

Hierarchical Agent Memory (HAM) scopes Claude Code agent context by generating per-directory CLAUDE.md files, tracking token savings, and surfacing analytics via a dashboard so multi-folder projects avoid repeated full scans.

Automate Datadog monitoring tasks via Rube MCP’s Composio toolkit—query metrics, search logs, manage monitors/dashboards, and create events or downtimes while respecting live tool schemas.

ccboard is a comprehensive TUI and Web dashboard designed for monitoring and managing Claude Code usage. It provides unified visibility into token consumption, detailed cost tracking, session history, and the real-time status of Model Context Protocol (MCP) servers. Use it to analyze usage patterns, manage configurations, and ensure cost efficiency across all projects.

This MCP server provides a persistent and searchable memory bank for AI agents. It enables reading, writing, and searching long-term knowledge, architecture patterns, and decisions. The system syncs with project documentation and includes a standalone dashboard for visualization. Ideal for managing agent context and knowledge retention across complex workflows.

Automate Datadog monitoring tasks via Rube MCP’s Composio toolkit—query metrics, search logs, manage monitors/dashboards, and create events or downtimes while respecting live tool schemas.

Reduces token costs by scoping context into directory-level CLAUDE.md files. Sets up hierarchical memory, tracks savings via dashboard, routes agents to sub-contexts, and provides audit and insights commands.

Trackio is a library for logging and visualizing machine learning training metrics. It supports Python API for logging metrics and firing alerts during training, and CLI for retrieving data. Metrics sync to Hugging Face Spaces for real-time dashboards. Ideal for autonomous ML workflows and experiment management.