Skill UI

llama.cpp is a pure C/C++ library designed for efficient LLM inference, optimized to run on resource-constrained environments. It excels in scenarios where high-end NVIDIA GPUs (CUDA) are unavailable, making it ideal for Apple Silicon Macs, AMD/Intel GPUs, or edge deployment devices like Raspberry Pi. It supports GGUF quantization formats, enabling significant memory reduction and speed improvements for local, cross-platform AI applications.

Outlines guarantees valid JSON/XML/code outputs by applying CFG/FSM-constrained token filtering, Pydantic schemas, and grammar-level sampling while supporting local Transformers, llama.cpp, and vLLM backends for fast inference.

An expert guide for deploying and optimizing Large Language Models (LLMs) entirely on local hardware. Mastering the entire local AI ecosystem, including inference engines like Ollama, vLLM, and llama.cpp. Provides deep knowledge on quantization formats (GGUF, EXL2, AWQ), VRAM calculation, and ensuring privacy-first, offline deployment for state-of-the-art models.

Build with and use Pi, a minimal terminal coding harness. Covers install, provider and model config, TypeScript extensions, skills, packages, themes, SDK embedding, RPC and JSON event streams, session parsing, local llama.cpp models, custom providers, TUI components, and ecosystem packages like pi-subagents and pi-mcp-adapter.

A comprehensive guide and skill for deploying large language models (LLMs) locally using the llama.cpp framework and GGUF quantization format. It covers searching the Hugging Face Hub, selecting optimal quantizations (e.g., Q4_K_M), running local servers, and achieving OpenAI-compatible API endpoints. Supports various hardware backends including CPU, Mac Metal, CUDA, and ROCm. Ideal for developers needing offline, private LLM inference.

Format chat messages into local model templates (llama3, mistral, chatml, phi, gemma, or a model id) and route queries through a small WASM HNSW pattern index. Useful for preparing prompts for llama.cpp, Ollama, and other local runtimes.

This skill helps you select and run large language models locally using llama.cpp and GGUF formats. It covers finding compatible models on the Hugging Face Hub, choosing quantization levels for different hardware like CPU, Mac Metal, CUDA, or ROCm, and launching servers via llama-cli or llama-server. It also includes instructions for converting weights and verifying OpenAI-compatible local serving endpoints.

An expert guide for deploying and optimizing Large Language Models (LLMs) entirely on local hardware. Mastering the entire local AI ecosystem, including inference engines like Ollama, vLLM, and llama.cpp. Provides deep knowledge on quantization formats (GGUF, EXL2, AWQ), VRAM calculation, and ensuring privacy-first, offline deployment for state-of-the-art models.

Calculate the cost per million tokens for running AI models locally on your own hardware like Ollama or llama.cpp. It factors in electricity prices, hardware costs, power consumption, and generation speed to determine the break-even point against hosted API models. Useful for deciding whether to deploy models locally or use cloud services based on actual usage and financial constraints.