Skill UI

llama.cpp is a pure C/C++ library designed for efficient LLM inference, optimized to run on resource-constrained environments. It excels in scenarios where high-end NVIDIA GPUs (CUDA) are unavailable, making it ideal for Apple Silicon Macs, AMD/Intel GPUs, or edge deployment devices like Raspberry Pi. It supports GGUF quantization formats, enabling significant memory reduction and speed improvements for local, cross-platform AI applications.

LlamaGuard is Meta’s 7–8B safety-specialized LLM that filters both prompts and responses by classifying six threat categories, enabling fast inference via vLLM/SageMaker and integration into NeMo Guardrails for end-to-end moderation.

Mamba is a selective state-space model architecture delivering O(n) inference for million-token sequences, offering hardware-aware Mamba-1 and Mamba-2 variants that beat Transformers on speed without KV cache overhead.

Automates guidance on identifying and responding to model drift within ML deployment workflows, offering best practices, production-ready code, and validation support so you can keep inference systems stable.

Compress large language models quickly using Wanda, SparseGPT, and N:M pruning to enforce hardware-friendly sparsity, enabling faster inference and deployment without retraining.

Automates guidance for deploying machine learning models with quantization, covering tooling, best practices, and production readiness for inference, MLops, and monitoring tasks.

Outlines guarantees valid JSON/XML/code outputs by applying CFG/FSM-constrained token filtering, Pydantic schemas, and grammar-level sampling while supporting local Transformers, llama.cpp, and vLLM backends for fast inference.

Phoenix is an open-source observability platform for tracing, evaluating, and monitoring LLM applications; use it to debug prompts, run dataset evaluations, build experiment pipelines, and watch production inference in real time.

Prediction Monitor Assistant automates machine learning deployment monitoring tasks, offering step-by-step guidance, best-practice patterns, production-ready code and configuration generation, and validation to keep serving and inference workflows reliable.

RWKV is an RNN+Transformer hybrid architecture with linear-time inference, infinite context, and GPT-style training. It delivers constant-memory, streaming generation on CUDA and CPU, ideal for long-context, memory-constrained deployments.

Auto-activating assistant that guides AWS Sagemaker endpoint deployment, from best practices to production-ready code, configurations, and validation for inference serving and monitoring workflows.

SGLang is a high-performance serving framework for LLMs and VLMs that uses RadixAttention prefix caching to deliver structured outputs (JSON/regex/grammar) and agentic workflows with function calls, providing up to 5× faster inference than vLLM across multi-GPU production deployments.