Skill UI

Defines best-practice Groq deployment with tiered model routing, middleware, streaming pipelines, and fallback chains for ultra-fast LLM inference and production monitoring when launching new Groq integrations.

Deploy llama.cpp to run LLM inference across CPUs, Apple Silicon, and non-NVIDIA GPUs, making it ideal for edge devices or CUDA-free setups with GGUF quantization for faster, lower-memory results.

LlamaGuard is Meta’s 7–8B safety-specialized LLM that filters both prompts and responses by classifying six threat categories, enabling fast inference via vLLM/SageMaker and integration into NeMo Guardrails for end-to-end moderation.

Mamba provides selective state-space models with O(n) inference complexity, letting you handle million-token sequences faster than transformers while skipping KV caches and benefiting from a hardware-aware design. Use it for long-context language modeling, streaming applications, and scalable low-memory sequence learners.

Modal's serverless GPU cloud platform lets teams run ML training, inference, and batch jobs with pay-per-second pricing, automatic scaling, Python-native infra definitions, fast cold starts, and container caching.

Outlines guarantees valid JSON/XML/code generation via CFG-driven FSM filtering, Pydantic schemas, and fast local or API models (Transformers, vLLM, llama.cpp), making structured inference safe and high-performance.

Optimizes LLM inference on NVIDIA GPUs, delivering 10-100× faster throughput, sub-10ms latency, and multi-GPU scaling with FP8/INT4 quantization, in-flight batching, and production-ready serving for real-time deployments.