Skill UI

Provides automated guidance for ab test analyzer workflows, covering SQL queries, visualization, statistical analysis, and BI validation so teams can run reliable experiments within data analytics contexts.

Automated guidance for configuring and applying early stopping callbacks across ML training workflows, covering best practices, code generation, and validation for PyTorch, TensorFlow, and scikit-learn experiments.

Track experiments, catalog models, compare versions, and deploy trained artifacts with MLflow’s framework-agnostic lifecycle tools across PyTorch, TensorFlow, and other stacks.

Guides senior data scientists through designing controlled experiments, engineering structured features, training ML models, and translating results into decisions using Python, R, SQL, and tools like Scikit-learn, XGBoost, and MLflow.

Provides a detailed, multi-platform workflow to monitor long-running computational experiments. It checks the status of jobs across various environments (SSH, vast.ai, Modal), collects screen logs, fetches structured JSON results, and pulls detailed training metrics (loss curves, accuracy) from Weights & Biases (W&B). Essential for MLOps and automated result review.

This comprehensive tool manages the complete lifecycle of ML experiments, providing robust deployment capabilities across diverse environments. It supports running training jobs on local GPUs, dedicated remote servers, Vast.ai, and serverless platforms like Modal. The workflow handles environment detection, code synchronization (rsync/git), dependency pinning, and integrates advanced logging tools (e.g., W&B), ensuring reliable, scalable, and reproducible ML training runs.

Trackio is an essential library for logging, visualizing, and analyzing machine learning training experiments. It provides multi-interface support: a Python API for real-time metric logging and alert firing during training; a Command Line Interface (CLI) for retrieving historical metrics and alerts; and synchronization capabilities to Hugging Face Spaces for persistent dashboards. This tool is crucial for debugging, diagnosing training issues (like loss spikes), and automating ML workflow iteration.

This skill evaluates experimental results against intended claims after experiments complete. It collects data from sources like W&B and logs, verifies evidence existence, and uses AI models to judge claim support. It routes to next actions like pivoting, supplementing, or confirming findings before writing papers or running ablations.

Utilize Modal, a serverless GPU cloud, for running complex machine learning workloads, including training, fine-tuning, and inference. This tool provides a zero-config environment, eliminating the need for SSH setup or Docker management. It features automatic scale-to-zero billing and local-first execution, making high-powered GPU compute accessible for debugging and small-scale experiments.

This skill provides comprehensive commands to manage the entire AI model evaluation lifecycle using Arize. Users can list, retrieve metadata, and export results from experiments. It is ideal for comparing multiple model runs, benchmarking performance against a dataset, and analyzing model accuracy using structured evaluation workflows (CRUD operations).

Searches BGPT scientific papers by topic or DOI and retrieves claim-level evidence extracted from full text, including experiments, reported statistics, scope, limitations, and provenance. Supports literature reviews and evidence synthesis.

Plan experiments before data is collected: choose a design, randomize and block units, avoid pseudoreplication, and generate reproducible layouts for factorial, screening, response-surface, crossover, split-plot, and cluster-randomized studies.