Skill UI

Design task-local harnesses, evaluation gates, and reusable skill extraction for adaptive agent workflows. This skill helps build structured systems for one-off work, shared skills, and observable checkpoints. It defines input boundaries, tool bindings, and safety checks to ensure reliable automation without manual intervention.

A formal framework designed for evaluating AI-assisted code generation sessions. It implements Development Oriented Evals (EDD) principles, allowing users to define capability and regression criteria. The system supports multiple evaluation types—code-based, model-based, and human review—and tracks reliability using advanced metrics like pass@k, ensuring robust quality control for AI agents.

This skill provides a detailed, side-by-side comparison of advanced LLM agent evaluation frameworks, such as ruflo and HAL. It helps researchers and developers identify capability gaps—including real code execution, full web browsing, and structured file parsing—and prioritize architectural improvements for robust agent benchmarking and performance tracking.

A sophisticated multi-agent development harness inspired by GANs. It separates product planning, code generation, and rigorous evaluation into three distinct agents (Planner, Generator, Evaluator). This creates an adversarial feedback loop, allowing projects to iteratively reach professional, production-grade quality far beyond what a single-agent workflow can achieve. Ideal for complex, full-stack applications.

This guide provides comprehensive instructions for packaging multiagent system (MARL) and game theory evidence for academic submissions, particularly for conferences like AAMAS. It emphasizes that the artifact must be inspectable, containing not just a trained model, but the game definition, environment code, opponent sets, and interaction protocols, ensuring rigorous reproducibility and reviewability.

This guide provides a detailed walkthrough of the AAMAS peer review process. It covers everything from initial submissions and technical evaluation (correctness, significance, rigor) to the rebuttal phase, area-chair synthesis, and final decisions. Use this when planning academic submissions or understanding the rigorous criteria for publishing multi-agent systems research.

This skill enables the execution of OpenSquilla agents against complex, real-world software engineering benchmarks (SWE-bench). It runs the entire process—from pulling the necessary Docker image to collecting the patch and running official evaluations—in a containerized environment. Use it to test an agent's ability to solve specific coding issues, benchmark performance across datasets (e.g., verified, multilingual), or verify if a generated patch successfully resolves an instance.

This workflow implements a bounded, iterative refinement loop for AI agents. It utilizes a required markdown rubric to grade the agent’s output in an isolated context. Based on the grader's verdict, it recommends the next move (sharpen, re-run, or promote). Once the agent passes the rubric, held-back evaluation cases are run in parallel to ensure generalization before final deployment.

A comprehensive, stack-agnostic tool for deeply evaluating agent harnesses. It checks for broken code paths, redundant instructions, and the overall usefulness of agent logic using a structured, multi-stage dual-judge protocol. The process involves high-priority questionnaires (Q1, Q2) before running three defined tracks: A (Correctness), B (Redundancy), and C (Usefulness). Ideal for auditing complex agent systems before deployment or major refactoring.

This tool initializes a dedicated AgentHub session for multi-agent collaborations. It requires defining the core task, specifying the number of parallel agents, and configuring detailed evaluation criteria (including the evaluation command, target metric, and measurement direction). Upon execution, it generates a unique session ID and prepares the entire environment for subsequent agent spawning and competitive evaluation, optionally capturing a performance baseline.

Build automated evaluation suites for AI agents with golden datasets, rubrics, and regression gates. Covers unit, tool-calling, end-to-end, and safety evals to validate prompt changes and gate deployments on quality.

Evaluate AI agent behavior using versioned cases and explicit verifiers. This skill helps compare agent or prompt changes, reproduce failures, and run regression tests. It ensures reliability on declared task distributions without inferring product readiness from generic scores. Includes methods for uncertainty reporting and safety verification.