Skill UI

A comprehensive guide to building production-grade evaluation systems for Large Language Models (LLMs). This skill covers advanced methodologies, including Direct Scoring and Pairwise Comparison, alongside critical techniques for mitigating systematic biases (e.g., Position Bias, Length Bias). Learn how to select appropriate metrics and structure prompts to ensure reliable, consistent, and objective quality assessment across various AI applications.

Performs a full GEO audit of any website by discovering business type, crawling prioritized pages, and delegating parallel subagents to evaluate AI visibility, platform readiness, technical infrastructure, EEAT content, and schema markup, then outputs a composite GEO score with prioritized actions.

The Phoenix CLI is a powerful tool for debugging complex LLM and agent workflows. It allows users to fetch detailed traces, spans, and sessions, analyze errors, and structure findings using open and axial coding techniques. It is essential for investigating complex LLM failures, building robust failure taxonomies, and guiding the next steps in application evaluation after initial instrumentation.

This guide provides a comprehensive framework for debugging and analyzing AgentCore runtime sessions using CloudWatch Logs Insights. It details advanced techniques like OpenTelemetry (OTEL) span correlation, resolving session IDs to trace IDs, and structuring complex query queries. Use this when investigating agent execution flows, diagnosing tool call failures, analyzing latency, or building detailed behavioral timelines.

This skill integrates LangChain 1.0 and LangGraph 1.0 into standard OpenTelemetry (OTEL) backends (e.g., Jaeger, Honeycomb, Tempo). It solves critical observability gaps by ensuring correct prompt content capture, propagating spans across complex subgraphs, and providing five LLM-specific Service Level Objectives (SLOs) like latency and cost. Ideal for multi-tenant, compliance-sensitive, or existing OTEL stack environments where LangSmith is unsuitable.

This guide details how to effectively delegate complex, multi-step tasks to various specialized AI agents (Codex, Pi, Claude, Hermes). It covers best practices for maintaining conversation context, handling TUI prompts (e.g., avoiding newlines, using quotes), techniques for reliable polling, and choosing the optimal agent for coding, design, or general purpose work. Essential for advanced AI workflow automation.

This guide provides a detailed walkthrough of the AAMAS peer review process. It covers everything from initial submissions and technical evaluation (correctness, significance, rigor) to the rebuttal phase, area-chair synthesis, and final decisions. Use this when planning academic submissions or understanding the rigorous criteria for publishing multi-agent systems research.

A comprehensive guide to modeling the entire academic review cycle for major NLP conferences like ACL. It details the stages from initial submission (ARR cycle intake) through peer review, author response, meta-review synthesis, commitment decision, and final venue decision (Main Conference, Findings, or Reject). It also outlines the roles of reviewers, area chairs, and program chairs, providing strategic advice for authors on navigating the complex publication landscape.

This guide details the rigorous, mathematically focused peer-review pipeline for top ML conferences like COLT (Conference on Learning Theory). It explains the double-anonymous review system, how reviewers assess correctness, significance, and novelty, and provides actionable advice on structuring rebuttals and interpreting final decisions for researchers aiming for top-tier academic publication.

A comprehensive project management skill designed for authors preparing papers for the EACL conference. It maps out the entire single-cycle submission process, covering backward planning from the ARR deadline, rebuttal strategies, commitment decisions, and final camera-ready stages. Essential for navigating highly structured academic publication timelines and minimizing risk.

This skill facilitates the loading and application of relevant historical experience memories (OpenViking memories) before executing task actions. It guides the user through a rigorous process: querying memories, applying strict applicability and exclusion checks (gating), and prioritizing real-time facts over loaded guidance. Essential for complex, multi-intent conversational flows.

Provides a comprehensive guide for planning strategy within major academic conference review systems, such as OpenReview. It details the double-blind review lifecycle, author response strategies, managing public vs. private discussions, and understanding how permanent visibility impacts decision-making. Ideal for authors and reviewers preparing for high-stakes academic submissions.