Skill UI

A comprehensive framework for rigorously testing and benchmarking LLM agents. It moves beyond simple pass/fail checks to assess complex behaviors, reliability metrics, and capability consistency across multiple runs. Ideal for production monitoring and identifying subtle failure modes in advanced AI agents.

A comprehensive toolkit for evaluating the performance of code generation models. It supports benchmarking against industry standards like HumanEval, MBPP, and MultiPL-E across multiple languages. Use this tool to quantitatively compare different LLM coding abilities and measure code synthesis quality using pass@k metrics.

Evaluates large language models across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) with standardized prompts and metrics, making it easy to benchmark HuggingFace or API models, compare releases, and track training progress.

A comprehensive workflow for implementing robust LLM evaluation using Langfuse. This skill guides users through key processes including scoring traces (numeric, categorical, boolean), collecting user feedback, managing versioned prompts, creating structured datasets, running automated experiments, and implementing LLM-as-a-Judge patterns for advanced quality assurance and system benchmarking.

Provides AI-ready drug discovery datasets, splits, and benchmarks covering ADME, toxicity, DTI, DDI, and molecular generation so researchers can train, compare, and optimize therapeutic ML models with curated splits and oracles.

This skill provides comprehensive tools for documenting and managing model performance on Hugging Face. It supports multiple methods for adding structured evaluation data, including extracting tables from README content, importing benchmark scores via APIs (like Artificial Analysis), and executing high-performance custom evaluations using advanced backends such as vLLM or lighteval. It ensures that all metadata is structured and compatible with model leaderboards and industry standards.

This skill automates the discovery of robotics research ideas by orchestrating a literature survey, idea generation, novelty check, and critical review workflow. It focuses on embodied AI, sim2real, and benchmark-grounded concepts for manipulation, locomotion, and navigation. The pipeline ensures ideas are feasible, falsifiable, and aligned with robotics constraints like simulation-first approaches and hardware availability.

Guides fine-tuning and evaluation of OpenVLA-OFT/OFT+ for robot action generation, covering LoRA adaptation, continuous action heads, FiLM conditioning, and LIBERO/ALOHA benchmarks plus deployment considerations.

This skill is designed for running comprehensive evaluations of Hugging Face Hub models on local hardware. It supports multiple evaluation frameworks, including `inspect-ai` and `lighteval`, and provides flexibility in backend selection (e.g., `vllm`, Transformers, or `accelerate`) to optimize for performance and compatibility. Use this for local GPU benchmarking, smoke testing, and backend selection, but note that it does not handle remote job orchestration or result publishing.

PyTDC enables discovery of therapeutic machine learning tasks, access to approved datasets, task-aware data splits, evaluation metrics, benchmark groups, and molecular oracle workflows. Designed for biomedical research and drug discovery applications using Python.

A universal infrastructure for automatically evolving and optimizing AI agents across any domain. Using LLM-driven mutation and iterative solve-observe-evolve cycles, it enhances critical components like prompts, skills, and memory by benchmarking performance against measurable datasets, enabling the creation of self-improving AI systems.

A comprehensive evaluation suite designed to benchmark the performance of Large Language Models (LLM) and Vision-Language Models (VLM) specifically for home security applications. It covers 143 diverse tests across 16 critical domains, including tool use, prompt injection resistance, contextual reasoning, alert triage, and detailed scene analysis, ensuring robustness for smart home AI systems.