Skill UI

Orchestrates PyTorch-native TorchTitan 4D parallelism (FSDP2, TP, PP, CP) for large-scale LLM pretraining, scaling from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing to boot Llama 3.1, DeepSeek V3, or custom models.

Modal Serverless GPU provides on‑demand, pay‑per‑second access to T4 through H200 GPUs, letting you deploy ML models as auto‑scaling APIs, run batch jobs, and prototype without managing infrastructure.

Optimizes large language model inference on NVIDIA GPUs with TensorRT, delivering 10-100× faster throughput, quantized precision (FP8/INT4), multi-GPU scaling, and serving-ready tooling for production deployments.

This guide details the process of deploying production-grade inference services on CoreWeave using KServe. It covers setting up autoscaling, GPU scheduling, and implementing scale-to-zero functionality for efficient ML model serving endpoints. Ideal for MLOps engineers deploying large language models (LLMs) or other complex AI models in a production environment.

This reference architecture provides a comprehensive blueprint for deploying Machine Learning models on CoreWeave's GPU cloud infrastructure. It details the structure for multi-model serving (e.g., vLLM, TGI), scalable Kubernetes deployments, shared persistent storage (PVC), and autoscaling mechanisms using KServe/Knative. Use this when designing robust MLOps pipelines, planning high-performance multi-model inference services, or establishing standardized best practices for GPU cloud deployment.

This skill enables horizontal scaling of Large Language Model inference clusters on Kubernetes. It utilizes KEDA for event-driven autoscaling based on custom GPU metrics like request queues and cache usage. The guide covers deploying vLLM on GPU nodes, configuring Prometheus for monitoring, implementing spot instance strategies for cost efficiency, and setting up queue-based scaling for batch jobs. It ensures optimal resource utilization and cost management for high-traffic AI workloads.