Skill UI

A comprehensive framework for rigorously testing and benchmarking LLM agents. It moves beyond simple pass/fail checks to assess complex behaviors, reliability metrics, and capability consistency across multiple runs. Ideal for production monitoring and identifying subtle failure modes in advanced AI agents.

A comprehensive toolkit for evaluating the performance of code generation models. It supports benchmarking against industry standards like HumanEval, MBPP, and MultiPL-E across multiple languages. Use this tool to quantitatively compare different LLM coding abilities and measure code synthesis quality using pass@k metrics.

Evaluates large language models across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) with standardized prompts and metrics, making it easy to benchmark HuggingFace or API models, compare releases, and track training progress.

A comprehensive workflow for implementing robust LLM evaluation using Langfuse. This skill guides users through key processes including scoring traces (numeric, categorical, boolean), collecting user feedback, managing versioned prompts, creating structured datasets, running automated experiments, and implementing LLM-as-a-Judge patterns for advanced quality assurance and system benchmarking.

Provides AI-ready drug discovery datasets, splits, and benchmarks covering ADME, toxicity, DTI, DDI, and molecular generation so researchers can train, compare, and optimize therapeutic ML models with curated splits and oracles.

This skill provides comprehensive tools for documenting and managing model performance on Hugging Face. It supports multiple methods for adding structured evaluation data, including extracting tables from README content, importing benchmark scores via APIs (like Artificial Analysis), and executing high-performance custom evaluations using advanced backends such as vLLM or lighteval. It ensures that all metadata is structured and compatible with model leaderboards and industry standards.

This comprehensive skill automates the entire process of generating novel, robust, and benchmark-grounded research ideas in robotics and embodied AI. It guides users from a broad initial concept to concrete, actionable project proposals by chaining rigorous literature surveys, novelty verification, and critical reviews. The workflow enforces robotics constraints, prioritizing simulation-first, falsifiable, and feasible real-world applications.

Guides fine-tuning and evaluation of OpenVLA-OFT/OFT+ for robot action generation, covering LoRA adaptation, continuous action heads, FiLM conditioning, and LIBERO/ALOHA benchmarks plus deployment considerations.

This skill is designed for performing local, on-premise evaluations and benchmarking of models hosted on the Hugging Face Hub. It facilitates running specialized evaluation frameworks like `inspect-ai` and `lighteval` against local hardware. Users can select optimal inference backends, including `vllm` for high throughput, or fall back to Hugging Face Transformers or `accelerate`, ensuring comprehensive smoke testing and task selection for LLMs. It focuses purely on the local execution process, separate from remote jobs or publishing workflows.

A universal infrastructure for automatically evolving and optimizing AI agents across any domain. Using LLM-driven mutation and iterative solve-observe-evolve cycles, it enhances critical components like prompts, skills, and memory by benchmarking performance against measurable datasets, enabling the creation of self-improving AI systems.

A comprehensive evaluation suite designed to benchmark the performance of Large Language Models (LLM) and Vision-Language Models (VLM) specifically for home security applications. It covers 143 diverse tests across 16 critical domains, including tool use, prompt injection resistance, contextual reasoning, alert triage, and detailed scene analysis, ensuring robustness for smart home AI systems.

This benchmark evaluates Vision-Language Models (VLMs) on detecting potential safety hazards within indoor environments. It uses static camera frames, simulating real-world fixed security camera monitoring. The test covers 40 scenarios across 5 critical categories, including fire/smoke, electrical risks, trips/falls, child safety, and falling objects, providing a rigorous measure of VLM's practical safety inspection capability.