Skill UI

Speeds up LLM inference by combining speculative decoding with Medusa multiple heads and lookahead techniques so chatbots, code assistants, and other real-time workloads can run with 1.5-3.6× lower latency while preserving target-model quality.

Automates guidance for streaming inference setup, covering best practices in ML deployment, serving, monitoring, and production-ready configurations whenever you ask about streaming inference patterns or implementation.

Optimizes large language model inference on NVIDIA GPUs with TensorRT, delivering 10-100× faster throughput, quantized precision (FP8/INT4), multi-GPU scaling, and serving-ready tooling for production deployments.

Automates generating TorchServe deployment configurations with guided best practices for model serving, MLOps pipelines, inference, and production monitoring so ML deployment workflows stay consistent and reliable.

Automates Triton inference config guidance for ML deployment, covering serving, MLOps pipelines, monitoring, and production-ready configurations aligned with best practices.

This comprehensive workflow guides users through the end-to-end process of utilizing cloud-based GPU computing resources on Vast.ai. It covers searching for ideal GPU offers based on specs and price, provisioning an instance, running complex training or inference jobs via SSH, collecting model artifacts, and crucially, destroying the instance to prevent unexpected billing. Ideal for deep learning training and research.

This comprehensive script suite provides methods to optimize GPU cloud computing expenses on Vast.ai. It guides users through selecting the most cost-efficient GPU for their workload, leveraging interruptible (spot) instances, and implementing critical lifecycle management safeguards. Features include detecting idle compute resources, auto-destroy timers, and calculating total cost burn rates, ensuring AI model training and inference jobs stay within budget.

A comprehensive guide for deploying Machine Learning training jobs and inference services on the Vast.ai GPU cloud. This skill covers the entire deployment lifecycle, including optimizing Docker images for performance, creating automated provisioning scripts using the Vast.ai CLI, implementing robust data transfer strategies (SCP, rsync), and performing post-deployment health checks.

Combines ESM3 and ESM C protein language models to generate sequences, predict structures, condition functions, and extract embeddings via local or Forge API inference for protein design and engineering workflows.

An expert guide for deploying and optimizing Large Language Models (LLMs) entirely on local hardware. Mastering the entire local AI ecosystem, including inference engines like Ollama, vLLM, and llama.cpp. Provides deep knowledge on quantization formats (GGUF, EXL2, AWQ), VRAM calculation, and ensuring privacy-first, offline deployment for state-of-the-art models.

Instrument LLM applications with OpenInference semantic conventions in Phoenix. Covers setup, auto and manual instrumentation, 9 span types, session tracking, production batching and PII masking for Python and TypeScript.

Use the esm Python SDK with ESM3 and ESMC for protein sequence generation, embeddings, function conditioning, structure prediction, inverse folding, and Forge/Biohub inference workflows.