Skill UI

This skill runs comprehensive benchmarks against structural and adversarial corpora. It measures the performance and cost efficiency of a booster model compared to leading LLM baselines (Gemini, Anthropic). Results are persisted in verifiable JSON reports, crucial for auditing claims and verifying booster win rates before publishing.

This skill consumes federation_spend events to aggregate per-peer spending across rolling time windows (1h, 24h, 7d). It implements a crucial circuit breaker mechanism by monitoring if any peer's spending exceeds a predefined threshold, ensuring the stability and resource management of the federation. Used to establish spending baselines and verify budget constraints.

This skill runs an automated evolution process designed to systematically optimize the seven core policy surfaces of an AI agent harness (e.g., planner, contextBuilder, reviewer). By mutating these policies and scoring the variants in a sandbox, it identifies crucial performance improvements without requiring expensive foundation model retraining. It is ideal for empirical configuration tuning and establishing optimized baselines.

This tool executes the advanced `metaharness/darwin` security benchmark (ADR-155). It rigorously evaluates the security detection capabilities of AI systems against a controlled ground-truth corpus, providing critical metrics like TPR, FPR, and patch-pass rates. It compares the champion model against multiple baselines, offering empirical data crucial for iterative security improvement and drift detection.

A comprehensive tool for aggregating and displaying system metrics (counters, gauges, histograms) from the observability namespace. It computes baselines and flags anomalies by detecting deviations exceeding standard deviations, providing a clear snapshot of system health, error rates, and performance degradation trends.

A comprehensive tool to scan all connected digital marketing platforms (e.g., Google Ads, Meta, TikTok) for statistically significant deviations from established performance baselines. It detects performance drops (traffic, conversions, ROAS) or unexpected growth opportunities, classifying anomalies by severity (Critical, Warning, Info) and providing probable root causes and immediate action plans. Essential for proactive campaign management and cost control.

Generates comprehensive quality intelligence reports by analyzing historical evaluation scores. This tool tracks content performance over time, identifies improving or declining content types, detects significant quality regressions against baselines, and provides prioritized, actionable recommendations for improving brand quality across various content dimensions.

This powerful tool monitors target keyword rankings across major search engines (e.g., Google) and tracks the presence and ownership of various SERP features (such as AI Overviews, Featured Snippets, and Local Packs). It establishes performance baselines and generates critical alerts when rankings drop significantly, or when the structure of search results changes, providing comprehensive visibility into organic search health.

A comprehensive guide for positioning AI/ML manuscripts for top-tier general AI conferences like AAAI. It helps authors refine their scope, frame their contribution, provide necessary evidence (e.g., ablations, baselines), and strategically compare their work against related research and similar venues (ICML, IJCAI). Ideal for pre-submission manuscript refinement.

Provides comprehensive guidelines for designing, auditing, and documenting empirical evidence for high-level AI research submissions (e.g., AAAI). It covers essential methodologies including strong baselines, single-factor ablations, robustness testing (shift, seed, prompt), statistical significance, and ensuring perfect alignment with reproducibility checklists and ethical reporting. Use this guide before submission to ensure empirical evidence strongly supports all claimed AI contributions.

A comprehensive guide for designing and presenting rigorous experiments and simulations for top-tier journals in Control Theory, Robotics, and Smart Systems. Covers RQ-driven design, establishing fair baselines, providing theoretical proofs of stability/convergence, performing ablation studies, and ensuring statistical significance, helping authors build an undeniable evidence chain for double-blind external review.

A comprehensive guide for researchers designing and auditing NLP experiments for top-tier conferences. It covers establishing defensible baselines, ensuring evaluation breadth, performing rigorous statistical tests (significance, variance), controlling for data contamination, and structuring detailed functional error analysis. Essential reading for publishing reliable AI research.