Skill UI

Assesses AI/ML systems for prompt injection, jailbreak, model inversion, data poisoning, and agent tool abuse using MITRE ATLAS mapping, signature detection, and scoring metrics before deployment.

This skill acts as an advanced extension layer, implementing a self-evolution protocol, maintaining session state continuity (Compaction), and providing KPI-style performance summaries. It allows users to simulate continuous professional development, track metrics, and master complex role-playing and interaction skills across multiple sessions, ensuring performance baselines are continuously adjusted and internalized.

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

Automate Datadog monitoring tasks via Rube MCP’s Composio toolkit—query metrics, search logs, manage monitors/dashboards, and create events or downtimes while respecting live tool schemas.

Analyzes and generates a self-contained, explorable HTML report detailing usage metrics for Claude Code sessions. It tracks token consumption (input/output), subagent usage, skill calls, and cache efficiency across specified time ranges. This tool helps developers visualize resource waste and pinpoint areas for workflow optimization, ensuring cost-effective and efficient AI development cycles.

PyHealth is a modular Python toolkit designed for developing deep learning pipelines in clinical and healthcare settings. It provides a standardized 5-stage framework (Dataset → Task → Model → Trainer → Metrics) to handle complex, multi-modal data, including EHR records, physiological signals, and medical imaging. Use cases include predicting mortality, readmission risks, recommending drugs, or performing medical code mapping (ICD/ATC).

A logging wrapper designed to instrument and monitor calls to the Claude API. It automatically tracks critical metrics including token usage, API latency, estimated cost, and error details. Essential for implementing robust observability patterns and setting up proactive alerts in production environments.

This module provides comprehensive observability for Anthropic's Claude API integrations. It enables the tracking of critical operational metrics including latency, token usage (input/output), estimated API costs, and error rates. Use it to set up structured logging and Prometheus metrics for robust monitoring and alerting, ensuring reliable and cost-effective AI application deployments.

This harness provides comprehensive, reproducible evaluation pipelines for complex LLM chains and agents (LangChain/LangGraph 1.0). It integrates golden dataset management, LangSmith evaluation, RAGAS metrics, deepeval LLM-as-judge, and structured agent trajectory analysis. Use it when establishing quality benchmarks for new chains, diagnosing performance regressions after model switches, or implementing CI/CD gates to prevent quality drops.

This skill wraps the `getCompactContext` functionality to efficiently retrieve a compacted ReasoningBank context, specifically optimized for cost-analysis queries. It generates a detailed report summarizing critical metrics such as tokens saved, memories retrieved, and agentic-flow availability. It is essential for monitoring and optimizing resource usage in complex AI workflows, providing deep visibility into the efficiency and cost of the context retrieval mechanism.

This tool enables the creation and management of Dynamic Agentic Architecture (DAA) agents. Unlike static agents, DAA agents possess the ability to learn, adapt their internal behavior based on real-time performance metrics, and continuously evolve through interactions. It facilitates knowledge sharing among multiple agents, ensuring systemic, continuous improvement in complex AI systems. Ideal for scenarios requiring high adaptability and evolving intelligence.

This tool executes the advanced `metaharness/darwin` security benchmark (ADR-155). It rigorously evaluates the security detection capabilities of AI systems against a controlled ground-truth corpus, providing critical metrics like TPR, FPR, and patch-pass rates. It compares the champion model against multiple baselines, offering empirical data crucial for iterative security improvement and drift detection.