Skill UI

Pyvene is a declarative framework designed for performing causal interventions on PyTorch neural networks. It allows researchers to conduct advanced experiments such as activation patching, causal tracing (ROME-style), and interchange intervention training (IIT). Use this library when you need to test causal hypotheses about model behavior, deeply interpret model components, or ensure reproducibility in advanced AI research.

torchforge is Meta's advanced PyTorch-native library designed for agentic Reinforcement Learning (RL). It fundamentally separates core RL algorithms from complex distributed infrastructure concerns. This allows researchers to rapidly prototype and experiment with novel RL methods (like GRPO) using clean, PyTorch-native abstractions, while automatically handling large-scale tasks like distributed training across thousands of GPUs using Monarch and TorchTitan.

Designs systematic ablation studies for ML paper submissions. Helps isolate component contributions, test hyperparameters, and answer reviewer questions. Codex plans experiments while CC implements them, ensuring rigorous validation before submission.

A persistent, graph-based knowledge base designed to accumulate structured knowledge throughout the entire research lifecycle. It stores and connects diverse entities—including published papers, novel ideas, conducted experiments, and testable claims. Inspired by LLM Wiki patterns, it ensures knowledge compounding, meaning the wiki becomes progressively smarter with every piece of input, rather than requiring re-derivation for every query.

Provides comprehensive guidelines for designing, auditing, and documenting empirical evidence for high-level AI research submissions (e.g., AAAI). It covers essential methodologies including strong baselines, single-factor ablations, robustness testing (shift, seed, prompt), statistical significance, and ensuring perfect alignment with reproducibility checklists and ethical reporting. Use this guide before submission to ensure empirical evidence strongly supports all claimed AI contributions.

Provides comprehensive guidelines for designing, auditing, and reporting high-rigor multi-agent experiments, especially in game theory and self-play systems. It emphasizes testing the *interaction* claim—such as convergence to equilibrium, truthful behavior, or emergent cooperation—against held-out opponents, populations, and strategic deviations, moving beyond simple single-agent leaderboard scores.

A comprehensive guide for researchers designing and auditing NLP experiments for top-tier conferences. It covers establishing defensible baselines, ensuring evaluation breadth, performing rigorous statistical tests (significance, variance), controlling for data contamination, and structuring detailed functional error analysis. Essential reading for publishing reliable AI research.

This comprehensive guide provides stringent methodological standards for authors submitting papers to ACM Multimedia (ACM MM). It details how to rigorously validate cross-modal claims, ensuring that evidence supports the multimedia contributions. Key areas covered include designing ablations to isolate fusion mechanisms, conducting robust user studies, reporting performance against matched baselines, and maintaining transparency regarding datasets, media licenses, and compute costs.

A comprehensive guide detailing best practices for designing and auditing empirical evaluations of Large Language Models (LLMs). It addresses critical pitfalls such as data contamination, baseline unfairness, model version drift, and decoding parameter sensitivity, ensuring that published results are robust, reproducible, and scientifically sound for academic publication.

A detailed guide for authors submitting theoretical machine learning papers (like COLT) regarding the appropriate inclusion and design of numerical simulations. It emphasizes that numerics should only visualize proved bounds, phase transitions, or separations, maintaining the focus on theoretical proofs rather than empirical data. Provides rules for plotting, reporting variability, and maintaining academic honesty.

A comprehensive guideline for researchers developing robot-learning papers. It details best practices for experimental design, emphasizing statistical rigor, handling stochasticity (training seeds, initial states), bridging the sim-to-real gap, and ensuring fairness across baseline policy families (BC/RL/VLA). This resource helps ensure claims are supported by credible evidence.

A comprehensive guide for designing, auditing, and executing rigorous experimental programs for top-tier computer vision papers (e.g., CVPR). It covers essential scientific components, including establishing fair benchmarks, structured ablation studies, documenting failure modes, quantifying computational efficiency, and demonstrating cross-dataset generalization, ensuring the research is reproducible and robust.