Skill UI

Provides automated guidance for ab test analyzer workflows, covering SQL queries, visualization, statistical analysis, and BI validation so teams can run reliable experiments within data analytics contexts.

Automated guidance for configuring and applying early stopping callbacks across ML training workflows, covering best practices, code generation, and validation for PyTorch, TensorFlow, and scikit-learn experiments.

Track experiments, catalog models, compare versions, and deploy trained artifacts with MLflow’s framework-agnostic lifecycle tools across PyTorch, TensorFlow, and other stacks.

Guides senior data scientists through designing controlled experiments, engineering structured features, training ML models, and translating results into decisions using Python, R, SQL, and tools like Scikit-learn, XGBoost, and MLflow.

Provides a detailed, multi-platform workflow to monitor long-running computational experiments. It checks the status of jobs across various environments (SSH, vast.ai, Modal), collects screen logs, fetches structured JSON results, and pulls detailed training metrics (loss curves, accuracy) from Weights & Biases (W&B). Essential for MLOps and automated result review.

This comprehensive tool manages the complete lifecycle of ML experiments, providing robust deployment capabilities across diverse environments. It supports running training jobs on local GPUs, dedicated remote servers, Vast.ai, and serverless platforms like Modal. The workflow handles environment detection, code synchronization (rsync/git), dependency pinning, and integrates advanced logging tools (e.g., W&B), ensuring reliable, scalable, and reproducible ML training runs.

Trackio is an essential library for logging, visualizing, and analyzing machine learning training experiments. It provides multi-interface support: a Python API for real-time metric logging and alert firing during training; a Command Line Interface (CLI) for retrieving historical metrics and alerts; and synchronization capabilities to Hugging Face Spaces for persistent dashboards. This tool is crucial for debugging, diagnosing training issues (like loss spikes), and automating ML workflow iteration.

This skill acts as a critical gatekeeper after a set of experiments are completed. It systematically evaluates raw experimental metrics and outcomes against predefined research claims. The goal is to determine whether the results adequately support a claim, contradict it, or if critical evidence is missing. It guides the research team towards the next decisive action: pivoting the hypothesis, supplementing the data, or confirming the findings. Use this before writing final reports.

Utilize Modal, a serverless GPU cloud, for running complex machine learning workloads, including training, fine-tuning, and inference. This tool provides a zero-config environment, eliminating the need for SSH setup or Docker management. It features automatic scale-to-zero billing and local-first execution, making high-powered GPU compute accessible for debugging and small-scale experiments.

This skill provides comprehensive commands to manage the entire AI model evaluation lifecycle using Arize. Users can list, retrieve metadata, and export results from experiments. It is ideal for comparing multiple model runs, benchmarking performance against a dataset, and analyzing model accuracy using structured evaluation workflows (CRUD operations).

DeepTools is a comprehensive Python command-line toolkit for processing and analyzing high-throughput sequencing data. It supports quality control, normalization, sample comparison, and generating publication-quality visualizations for ChIP-seq, RNA-seq, and ATAC-seq experiments. Key features include converting BAM to bigWig, assessing sequencing depth, creating heatmaps around genomic features, and performing correlation analysis. Ideal for bioinformatics workflows requiring robust genomic data processing and visualization.

Plan experiments before data collection: choose designs, randomize, block, and generate reproducible DOE layouts to avoid confounding and pseudoreplication.