Skill UI

A comprehensive framework for rigorously testing and benchmarking LLM agents. It moves beyond simple pass/fail checks to assess complex behaviors, reliability metrics, and capability consistency across multiple runs. Ideal for production monitoring and identifying subtle failure modes in advanced AI agents.

A comprehensive CI/CD pipeline designed for LangChain applications. It automates quality assurance through fast, mocked unit tests and slower, more accurate integration tests utilizing real LLMs (e.g., OpenAI). It specifically validates the entire RAG pipeline and integrates tracing with LangSmith, ensuring robust deployment quality.

A comprehensive guide to configuring a structured, local development environment for LangChain applications. It covers setting up project architecture, managing dev dependencies, and implementing a two-tiered testing strategy: isolated unit tests using mocked LLMs (for fast, API-free testing) and robust integration tests using real API providers. Ideal for establishing a reliable, rapid iteration workflow.

A comprehensive checklist for ensuring LangChain applications are robust, secure, and scalable for production environments. It covers critical areas including configuration management, advanced error handling, observability setup (e.g., LangSmith), performance optimization, security best practices (like prompt injection prevention and PII handling), rigorous testing, and zero-downtime deployment strategies. Use this guide when preparing for a major launch or auditing existing LLM systems.

Integrate Langfuse into CI/CD pipelines (e.g., GitHub Actions) to automate the testing and validation of Large Language Models (LLMs). This guide covers prompt regression testing, tracing validation, and setting up experiment-driven quality gates, ensuring reliable and high-performing AI applications throughout the development lifecycle.

A minimal example demonstrating how to implement basic tracing using the Langfuse SDK. It illustrates the hierarchy of traces, spans, and generations (LLM calls), providing code examples in TypeScript and Python for easy setup and integration testing.

This guide provides a comprehensive local development setup for LLM applications using Langfuse. It enables fast iteration cycles, real-time trace debugging, and hot reloading, making it ideal for developers building and testing complex AI workflows locally. Instructions cover environment setup, advanced SDK usage, and optional self-hosted Docker deployment.

This guide demonstrates the basic workflow for building an AI agent using a webhook trigger. The agent receives structured data (payload), utilizes an LLM to extract key information and generate a concise summary (summarization step), and finally executes an action by posting the summary to Slack or sending an email. Ideal for beginners learning about AI agent architecture, testing data flow, and implementing simple automated system notifications.

Master comprehensive strategies for evaluating Large Language Model (LLM) applications. This skill covers automated metrics (BLEU, ROUGE, BERTScore), structured human evaluation dimensions, and advanced LLM-as-Judge techniques. Use it to systematically measure model performance, compare prompts, detect regressions, and build confidence in production systems.

This comprehensive guide provides minimal working examples for integrating with the Mistral AI API. It demonstrates core LLM functionalities including basic chat completion, streaming responses, multi-turn conversations, and generating structured JSON output. Use this guide when starting a new Mistral integration, testing API connections, or learning advanced AI patterns using TypeScript or Python.

This guide provides a comprehensive quickstart for integrating with OpenRouter, a unified endpoint for accessing multiple large language models (LLMs). It includes examples in cURL, Python, and TypeScript, demonstrating how to send chat completion requests, handle complex JSON responses, manage model parameters, and calculate usage cost across various providers (e.g., Llama, GPT-4o, Claude). Ideal for developers testing API connectivity or building multi-model applications.

A comprehensive guide for developing, optimizing, and standardizing skills for AI agents. This resource provides a decision tree for choosing the right architecture (Tier 1-3), incorporating best practices like CSO (LLM SEO) and Anti-Rationalization, and ensuring compliance through rigorous testing and checklists. Essential for robust AI system building.