Skill UI

Implement comprehensive end-to-end tracing for Large Language Model (LLM) calls, chains, and complex agents using Langfuse. This workflow demonstrates techniques like OpenAI drop-in wrappers, manual span creation for RAG pipelines, tracking streamed responses, and integrating with various LLM providers (OpenAI, Anthropic). Use this to monitor latency, track token usage, and debug complex AI features in production.

A comprehensive workflow for implementing robust LLM evaluation using Langfuse. This skill guides users through key processes including scoring traces (numeric, categorical, boolean), collecting user feedback, managing versioned prompts, creating structured datasets, running automated experiments, and implementing LLM-as-a-Judge patterns for advanced quality assurance and system benchmarking.

Learn how to monitor, analyze, and optimize the expenditure of Large Language Models (LLMs) using Langfuse's advanced analytics suite. This guide covers techniques like automatic token tracking, programmatic cost reporting via the Metrics API, and implementing smart model routing to automatically select the most cost-effective model based on task complexity and input length. Essential for any developer managing AI budgets and ensuring operational efficiency.

This guide details how to establish comprehensive observability for LLM operations using Langfuse. It covers setting up Prometheus metrics (tracking traces, generation duration, and token usage) and visualizing them in Grafana dashboards. Ideal for monitoring application health, optimizing costs, and tracking performance over time.

LangSmith centralizes tracing, evaluation, and monitoring for LLM applications, letting teams debug prompts, run regression tests, collect latency/cost metrics, and integrate with OpenAI, Anthropic, LangChain, or LlamaIndex to keep production systems observable.

This guide provides comprehensive strategies for optimizing the execution speed, reliability, and cost efficiency of AI agents. Learn how to minimize latency, right-size models (e.g., Flash vs. Opus), consolidate multiple LLM steps into single prompts, and optimize workflow triggers. Ideal for use when agents are slow, consuming excessive credits, or producing inconsistent results.

llama.cpp is a pure C/C++ library designed for efficient LLM inference, optimized to run on resource-constrained environments. It excels in scenarios where high-end NVIDIA GPUs (CUDA) are unavailable, making it ideal for Apple Silicon Macs, AMD/Intel GPUs, or edge deployment devices like Raspberry Pi. It supports GGUF quantization formats, enabling significant memory reduction and speed improvements for local, cross-platform AI applications.

Provides expert guidance for using llama-factory to fine-tune LLMs through its WebUI, covering QLoRA/LoRA quantized setups, multimodal models, and automation tips for developers exploring models like Llama, Qwen, or Gemma.

LlamaGuard is Meta’s 7–8B safety-specialized LLM that filters both prompts and responses by classifying six threat categories, enabling fast inference via vLLM/SageMaker and integration into NeMo Guardrails for end-to-end moderation.

LlamaIndex is a data-centric framework for building LLM applications, offering 300+ connectors, vector indices, query engines, agents, and multimodal support to power document Q&A, chatbots, knowledge retrieval, and RAG pipelines over private data.

This comprehensive guide provides production-ready patterns for building advanced LLM applications. It details the entire RAG pipeline, covering document ingestion, various chunking strategies, and advanced retrieval methods (Hybrid, Multi-query). Furthermore, it covers sophisticated agent architectures, such as ReAct, enabling complex reasoning and tool use. Ideal for developers designing enterprise-grade AI systems.

This skill guides users through the entire lifecycle of developing intelligent AI assistants, chatbots, and conversational interfaces. It specializes in establishing robust natural language understanding, managing multi-turn context, and defining best practices for creating seamless, production-ready AI applications with real-world value.