Skill UI

A comprehensive toolkit for evaluating the performance of code generation models. It supports benchmarking against industry standards like HumanEval, MBPP, and MultiPL-E across multiple languages. Use this tool to quantitatively compare different LLM coding abilities and measure code synthesis quality using pass@k metrics.

This guide details advanced methods for merging multiple specialized LLMs (e.g., combining math, coding, and chat capabilities) without the need for expensive retraining. Learn various techniques such as Linear Merging, SLERP, Task Arithmetic, TIES-Merging, and DARE. Use mergekit to create high-performing, specialized model variants quickly, significantly improving performance and reducing computational costs.

A comprehensive collection of battle-tested, structured prompts designed to elevate interactions with Large Language Models (LLMs). Covers advanced use cases including expert coding reviews, system architecture design, performance analysis, technical writing, and creative brainstorming, ensuring high-quality, actionable AI output.

Speeds up LLM inference by combining speculative decoding with Medusa multiple heads and lookahead techniques so chatbots, code assistants, and other real-time workloads can run with 1.5-3.6× lower latency while preserving target-model quality.

This tool assesses whether coding agents adhere to defined skills, rules, or agent definitions. It automatically generates scenarios with varying prompt strictness, executes the agent, and generates a comprehensive report detailing the compliance rate of tool calls against predefined workflow specifications using advanced LLM classification.

The Phoenix CLI is a powerful tool for debugging complex LLM and agent workflows. It allows users to fetch detailed traces, spans, and sessions, analyze errors, and structure findings using open and axial coding techniques. It is essential for investigating complex LLM failures, building robust failure taxonomies, and guiding the next steps in application evaluation after initial instrumentation.

This skill executes the DeepSWE benchmark, a comprehensive coding-agent evaluation suite. It uses OpenRouter and mini-swe-agent to score various Large Language Models (LLMs) on 113 tasks, providing reproducible results for model comparison, development assessment, and research benchmarking. Use this when a standardized, deep evaluation of LLM coding capabilities is required.

This tool assesses whether coding agents adhere to defined skills, rules, or agent definitions. It automatically generates scenarios with varying prompt strictness, executes the agent, and generates a comprehensive report detailing the compliance rate of tool calls against predefined workflow specifications using advanced LLM classification.

A comprehensive guide detailing best practices for designing and auditing empirical evaluations of Large Language Models (LLMs). It addresses critical pitfalls such as data contamination, baseline unfairness, model version drift, and decoding parameter sensitivity, ensuring that published results are robust, reproducible, and scientifically sound for academic publication.