Skill UI

This guide details the use of the GGUF format and quantization techniques for running large language models (LLMs) efficiently. GGUF enables optimal inference on consumer hardware, Apple Silicon, and systems requiring CPU-only operation. By compressing models using various K-quant methods (like Q4_K_M), developers can significantly reduce memory footprint and hardware requirements while maintaining high performance.

llama.cpp is a pure C/C++ library designed for efficient LLM inference, optimized to run on resource-constrained environments. It excels in scenarios where high-end NVIDIA GPUs (CUDA) are unavailable, making it ideal for Apple Silicon Macs, AMD/Intel GPUs, or edge deployment devices like Raspberry Pi. It supports GGUF quantization formats, enabling significant memory reduction and speed improvements for local, cross-platform AI applications.

A comprehensive skill for fine-tuning and training large language and vision models using advanced techniques like TRL (SFT, DPO, GRPO) or Unsloth. This process runs entirely on managed Hugging Face cloud GPUs, eliminating the need for local GPU setup. It supports modern training workflows, model conversion (GGUF for local deployment), and ensures seamless artifact storage to the Hugging Face Hub.

An expert guide for deploying and optimizing Large Language Models (LLMs) entirely on local hardware. Mastering the entire local AI ecosystem, including inference engines like Ollama, vLLM, and llama.cpp. Provides deep knowledge on quantization formats (GGUF, EXL2, AWQ), VRAM calculation, and ensuring privacy-first, offline deployment for state-of-the-art models.

The `hf-mem` skill estimates the necessary VRAM or system memory required to load Safetensors or GGUF model weights for inference directly from the Hugging Face Hub. It uses efficient HTTP Range requests, avoiding local downloads. Users can determine if a model fits on their GPU or instance, including optional estimations for the KV cache and setting limits like maximum sequence length.

A comprehensive guide and skill for deploying large language models (LLMs) locally using the llama.cpp framework and GGUF quantization format. It covers searching the Hugging Face Hub, selecting optimal quantizations (e.g., Q4_K_M), running local servers, and achieving OpenAI-compatible API endpoints. Supports various hardware backends including CPU, Mac Metal, CUDA, and ROCm. Ideal for developers needing offline, private LLM inference.

Estimate the required memory for inference of Safetensors or GGUF models from the Hugging Face Hub using HTTP Range requests, without downloading weights. Supports optional KV cache estimation for LLMs and VLMs.

Train or fine-tune language and vision models using TRL or Unsloth on Hugging Face Jobs infrastructure. Supports SFT, DPO, GRPO methods and GGUF conversion for local deployment. Requires Pro account and HF_TOKEN for Hub saving. Optimized for cloud GPU training without local setup.

This skill helps you select and run large language models locally using llama.cpp and GGUF formats. It covers finding compatible models on the Hugging Face Hub, choosing quantization levels for different hardware like CPU, Mac Metal, CUDA, or ROCm, and launching servers via llama-cli or llama-server. It also includes instructions for converting weights and verifying OpenAI-compatible local serving endpoints.

An expert guide for deploying and optimizing Large Language Models (LLMs) entirely on local hardware. Mastering the entire local AI ecosystem, including inference engines like Ollama, vLLM, and llama.cpp. Provides deep knowledge on quantization formats (GGUF, EXL2, AWQ), VRAM calculation, and ensuring privacy-first, offline deployment for state-of-the-art models.

Fine-tune LLMs on a single consumer GPU with Unsloth Core. Covers VRAM sizing, LoRA/QLoRA, GRPO/DPO post-training, chat-template correctness, and GGUF export.