Skill UI

This skill guides the implementation of security chaos engineering experiments. It involves deliberately degrading or disabling security controls (like WAF, firewalls, or log pipelines) in a controlled environment to rigorously verify the detection, alerting, and response capabilities of the Security Operations Center (SOC). Users learn to use tools like boto3 and Python to simulate failure scenarios and assess system resilience and compliance posture.

Provides a detailed, multi-platform workflow to monitor long-running computational experiments. It checks the status of jobs across various environments (SSH, vast.ai, Modal), collects screen logs, fetches structured JSON results, and pulls detailed training metrics (loss curves, accuracy) from Weights & Biases (W&B). Essential for MLOps and automated result review.

Autonomous full research pipeline that takes a general research direction through idea discovery, experiments, review, and writing to a polished submission-ready paper PDF. Use for complete AI/ML research lifecycle workflows from ideation to submission, including resume support and cross-model quality review.

Get a multi-round adversarial critical review of your research ideas, papers, or experimental results from an external LLM reviewer backend (Codex or manual). Compiles a briefing, runs ultra-reasoning review rounds, and returns a verdict with logical gaps, missing experiments, and narrative weaknesses.

This comprehensive tool manages the complete lifecycle of ML experiments, providing robust deployment capabilities across diverse environments. It supports running training jobs on local GPUs, dedicated remote servers, Vast.ai, and serverless platforms like Modal. The workflow handles environment detection, code synchronization (rsync/git), dependency pinning, and integrates advanced logging tools (e.g., W&B), ensuring reliable, scalable, and reproducible ML training runs.

Designs systematic ablation studies for ML paper submissions. Helps isolate component contributions, test hyperparameters, and answer reviewer questions. Codex plans experiments while CC implements them, ensuring rigorous validation before submission.

Trackio is an essential library for logging, visualizing, and analyzing machine learning training experiments. It provides multi-interface support: a Python API for real-time metric logging and alert firing during training; a Command Line Interface (CLI) for retrieving historical metrics and alerts; and synchronization capabilities to Hugging Face Spaces for persistent dashboards. This tool is crucial for debugging, diagnosing training issues (like loss spikes), and automating ML workflow iteration.

This skill acts as a critical gatekeeper after a set of experiments are completed. It systematically evaluates raw experimental metrics and outcomes against predefined research claims. The goal is to determine whether the results adequately support a claim, contradict it, or if critical evidence is missing. It guides the research team towards the next decisive action: pivoting the hypothesis, supplementing the data, or confirming the findings. Use this before writing final reports.

Utilize Modal, a serverless GPU cloud, for running complex machine learning workloads, including training, fine-tuning, and inference. This tool provides a zero-config environment, eliminating the need for SSH setup or Docker management. It features automatic scale-to-zero billing and local-first execution, making high-powered GPU compute accessible for debugging and small-scale experiments.

This skill provides comprehensive commands to manage the entire AI model evaluation lifecycle using Arize. Users can list, retrieve metadata, and export results from experiments. It is ideal for comparing multiple model runs, benchmarking performance against a dataset, and analyzing model accuracy using structured evaluation workflows (CRUD operations).

This skill guides users on setting up and running LLM-as-judge evaluations using Arize. It covers defining evaluators (including prompts, classification choices, and models) and executing tasks against real data spans or experiments. It supports key metrics like hallucination, faithfulness, and correctness, and facilitates continuous monitoring for comprehensive LLM performance analysis.

Guides teams through testing messaging and word choice to find defensible market positioning, using a phased Crawl-Walk-Run rollout plus word-change experiments to accelerate enterprise deals.