Skill UI

An expert in designing, implementing, and optimizing scalable, reliable, and cost-effective data pipelines for both batch and streaming data. Covers advanced topics including ETL/ELT patterns, Kappa/Lambda architectures, workflow orchestration (Airflow/Prefect), transformation using dbt/Spark, and managing modern storage formats like Delta Lake and Iceberg. Focuses heavily on data quality, monitoring, and cost optimization strategies.

This guide provides a comprehensive framework for minimizing cloud expenditure on Databricks. It covers identifying cost drivers via system billing tables, implementing cluster policies, utilizing spot instances for batch processing, right-sizing SQL warehouses, and automating the shutdown of idle development clusters. Use it when managing cloud spending or optimizing data platform usage.

Codified expertise for sequencing work orders, balancing discrete and batch lines, reducing changeover time, and shielding bottlenecks using FCS, DBR, and SMED so manufacturing meets commitments despite disruptions.

This comprehensive guide provides advanced strategies for optimizing and scaling Qdrant vector database throughput (QPS). It covers performance tuning techniques, such as optimizing segment size, enabling quantization, and using batch search. Furthermore, it details horizontal scaling methods using read replicas and addressing disk I/O bottlenecks, ensuring high performance for large-scale, high-concurrency retrieval applications.

A powerful, kubectl/docker-style CLI tool designed to manage and submit distributed computing jobs on the Qizhi (启智) platform. Use it for tasks like logging in, checking resource availability, discovering workspaces, and submitting complex GPU, CPU, or batch jobs efficiently.