Skill UI

Autonomous full research pipeline that takes a general research direction through idea discovery, experiments, review, and writing to a polished submission-ready paper PDF. Use for complete AI/ML research lifecycle workflows from ideation to submission, including resume support and cross-model quality review.

Get a multi-round adversarial critical review of your research ideas, papers, or experimental results from an external LLM reviewer backend (Codex or manual). Compiles a briefing, runs ultra-reasoning review rounds, and returns a verdict with logical gaps, missing experiments, and narrative weaknesses.

Designs systematic ablation studies for ML paper submissions. Helps isolate component contributions, test hyperparameters, and answer reviewer questions. Codex plans experiments while CC implements them, ensuring rigorous validation before submission.

A workflow that bridges experiment plans to execution. Reads an experiment plan, implements training and evaluation code, performs cross-model code review, deploys runs to GPU clusters, collects initial metrics, and prepares outputs for automated review loops.

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

Phoenix Evals enables developers to build and run evaluators for AI and LLM applications. It supports both code-based and LLM-based evaluation methods, including RAG systems. Users can create datasets, run experiments, validate evaluator accuracy, and implement production guardrails. Available in Python and TypeScript, it helps ensure model performance and reliability through systematic testing and human validation.

A persistent, graph-based knowledge base designed to accumulate structured knowledge throughout the entire research lifecycle. It stores and connects diverse entities—including published papers, novel ideas, conducted experiments, and testable claims. Inspired by LLM Wiki patterns, it ensures knowledge compounding, meaning the wiki becomes progressively smarter with every piece of input, rather than requiring re-derivation for every query.

SSH job queue for multi-seed/multi-config ML experiments with OOM-aware retry, stale-screen cleanup, and wave-transition race prevention. Use for batch experiments, grid runs, multi-seed sweeps, or auto-chain experiments when /run-experiment cannot handle 10+ jobs that need orchestration.

Audit experiment integrity before claiming results. Uses an independent cross-model reviewer to detect fake ground truth, score normalization fraud, phantom results, and insufficient experimental scope. Invoke after experiments complete or whenever experimental claims need honest verification.

This powerful compiler ingests heterogeneous research inputs, including PDF papers, code repositories, experiment logs, and raw notes. It transforms this diverse data into a comprehensive, structured Agent-Native Research Artifact (ARA). The system extracts core knowledge atoms, builds cognitive maps (claims, concepts, heuristics), and generates a verifiable, executable knowledge package for advanced AI consumption and research iteration.

This skill autonomously optimizes artifacts like code or models using Hypothesis Tree Refinement. It runs iterative experiments with dev/test evaluators to prevent overfitting, coordinating subagents in isolated worktrees. Ideal for improving training recipes, agents, or pipelines without manual supervision.

Provides comprehensive guidelines for designing, auditing, and documenting empirical evidence for high-level AI research submissions (e.g., AAAI). It covers essential methodologies including strong baselines, single-factor ablations, robustness testing (shift, seed, prompt), statistical significance, and ensuring perfect alignment with reproducibility checklists and ethical reporting. Use this guide before submission to ensure empirical evidence strongly supports all claimed AI contributions.