Skill UI

Automates the creation of Airflow DAGs for ETL and streaming data workflows, offering best-practice guidance, production-ready code samples, and validation against common pipeline standards.

A comprehensive guide to building production-ready Apache Airflow Directed Acyclic Graphs (DAGs). This skill covers best practices for operators, sensors, testing, deployment, and designing robust data pipelines. Use this when orchestrating complex workflows, managing ETL processes, or ensuring reliable batch job execution in a production environment.

Guides you through building Airflow operators for data pipelines with automated best practices, production-ready code/config generation, and validation for ETL, transformation, orchestration, and streaming scenarios.

This skill provides the core mechanism for integrating Hex into external data pipelines. It enables triggering, monitoring, and managing Hex projects (e.g., via Airflow or Dagster) programmatically. Users can implement parameterized runs, execute multi-step ETL workflows sequentially, and handle status polling and error management, making it ideal for production data orchestration.

This skill helps diagnose performance bottlenecks, slow jobs, and expensive queries across various data platforms (Airflow, dbt, Databricks). It uses a tiered investigation approach, starting with broad discovery to pinpoint problematic jobs or queries, bridging to affected tables, and finally drilling down to identify root causes, latency trends, and change-related regressions.