Login
Download
Skill UI
Browse and discover
18671+
curated skills
All
Development
Artificial Intelligence
Design & Creative
Product & Business
Data Science
Marketing
Soft Skills
Productivity
Engineering
Languages
Search
KEDA
, found
1
results
Default
Newest
Most Downloaded
Kubernetes LLM Inference Autoscaling
llm-inference-scaling
sickn33/agentic-awesome-skills
470
This skill enables horizontal scaling of Large Language Model inference clusters on Kubernetes. It utilizes KEDA for event-driven autoscaling based on custom GPU metrics like request queues and cache usage. The guide covers deploying vLLM on GPU nodes, configuring Prometheus for monitoring, implementing spot instance strategies for cost efficiency, and setting up queue-based scaling for batch jobs. It ensures optimal resource utilization and cost management for high-traffic AI workloads.
View Details
1
Language
简体中文
English