Skill UI

This comprehensive guide outlines a rigorous, human-in-the-loop engineering workflow for leveraging advanced AI agents to complete complex implementation tasks. It mandates defining clear completion criteria, decomposing work into manageable units, and strategically routing models (Haiku, Sonnet, Opus) based on task complexity. The workflow emphasizes continuous evaluation, regression testing, security review of invariants and edge cases, and strict cost discipline to ensure robust and reliable software development using AI assistance.

A lightweight CLI tool for systematically comparing various coding AI agents (such as Claude Code, Aider, and Codex) head-to-head. It measures crucial metrics—pass rate, API cost, execution time, and consistency—on reproducible tasks defined via YAML manifests. Use this to objectively select the best agent for your codebase or to validate model updates before production deployment.

This skill guides the end-to-end process of building automated evaluation pipelines for Python applications powered by Large Language Models (LLMs). It teaches how to define evaluation criteria, instrument the application, build golden datasets, and run real-world evaluations using tools like pixie, ensuring the app's logic (not just the LLM) is rigorously tested. Ideal for QA, benchmarking, and improving LLM-based services.

This tool provides a local development loop for integrating with Together AI's OpenAI-compatible API. It mocks key functionalities like chat completions, embeddings, and model listing, allowing developers to build and test AI applications rapidly without consuming live API credits. Developers can toggle between mock mode for rapid iteration and live mode for final model evaluation, ensuring a seamless workflow.