Skill UI

This comprehensive canvas guides teams through developing robust AI-powered features. It ensures product decisions are rigorous by covering problem definition, model selection (LLM, RAG, etc.), data governance, evaluation metrics, user experience design (including fallbacks), and responsible AI compliance. Ideal for assessing AI readiness and preventing common product failures.

This skill executes the DeepSWE benchmark, a comprehensive coding-agent evaluation suite. It uses OpenRouter and mini-swe-agent to score various Large Language Models (LLMs) on 113 tasks, providing reproducible results for model comparison, development assessment, and research benchmarking. Use this when a standardized, deep evaluation of LLM coding capabilities is required.

A lightweight CLI tool for systematically comparing various coding AI agents (such as Claude Code, Aider, and Codex) head-to-head. It measures crucial metrics—pass rate, API cost, execution time, and consistency—on reproducible tasks defined via YAML manifests. Use this to objectively select the best agent for your codebase or to validate model updates before production deployment.

This comprehensive guide outlines a rigorous, human-in-the-loop engineering workflow for leveraging advanced AI agents to complete complex implementation tasks. It mandates defining clear completion criteria, decomposing work into manageable units, and strategically routing models (Haiku, Sonnet, Opus) based on task complexity. The workflow emphasizes continuous evaluation, regression testing, security review of invariants and edge cases, and strict cost discipline to ensure robust and reliable software development using AI assistance.

A formal framework designed for evaluating AI-assisted code generation sessions. It implements Development Oriented Evals (EDD) principles, allowing users to define capability and regression criteria. The system supports multiple evaluation types—code-based, model-based, and human review—and tracks reliability using advanced metrics like pass@k, ensuring robust quality control for AI agents.

Guides users through the entire end-to-end process of running the GAIA benchmark. It covers environment validation, cost estimation, model execution, and generating a signed, HAL-compatible submission package for leaderboards. This ensures high integrity and reproducibility for model performance evaluation.

GateGuard is a mandatory pre-tool-use hook designed to significantly enhance the reliability and safety of LLM-generated code and actions. Instead of relying on self-evaluation, it forces the model to perform concrete, verifiable investigations—such as listing all imported files, checking data schemas, or detailing rollback procedures—before any write or destructive command execution. It establishes a three-stage gate (Deny, Force, Allow) to prevent hallucination and ensure context awareness in complex coding environments.

A comprehensive guide detailing the entire model lifecycle for RuView. It supports multiple advanced tracking methods including camera-free pose estimation (WiFlow), camera-supervised methods using MediaPipe, robust contrastive embeddings (RuVector), and domain generalization (MERIDIAN). Covers training, evaluation, and deployment across local GPU and cloud platforms.

This guide provides comprehensive instructions for packaging multiagent system (MARL) and game theory evidence for academic submissions, particularly for conferences like AAMAS. It emphasizes that the artifact must be inspectable, containing not just a trained model, but the game definition, environment code, opponent sets, and interaction protocols, ensuring rigorous reproducibility and reviewability.

A comprehensive guide for packaging and submitting evidence packages (artifacts) for NLP research papers, such as code, datasets, prompts, and model outputs. It details requirements for reproducibility, adhering to the Responsible NLP checklist, ensuring proper anonymization, and preparing for post-acceptance public release in academic settings like ACL.

A comprehensive guide for structuring research artifacts (code, models, datasets, media) for ACM Multimedia conferences. It details the distinct requirements for anonymous review packages versus permanent public releases, offering guidelines on ethics, licensing, reproducibility, and track-specific documentation.

This skill guides researchers on the mandatory, rigorous evaluation of complex systems models (e.g., AgSy). It details best practices for model validation, emphasizing the comparison of observed vs. simulated data using independent, held-out datasets. Key components covered include standard fit statistics (RMSE, NSE), sensitivity analysis (identifying core drivers), uncertainty propagation, and multi-objective trade-off/scenario analysis to ensure scientific credibility.