Skill UI

Guides you through building Airflow operators for data pipelines with automated best practices, production-ready code/config generation, and validation for ETL, transformation, orchestration, and streaming scenarios.

This guide details setting up robust CI/CD pipelines for applications utilizing Clay. It covers automated unit and integration testing for webhook handlers, validating data schema compliance, and ensuring the reliability of data enrichment processes within your build workflow. Ideal for teams deploying webhooks or complex data transformation logic.

Guides users through compression optimizer scenarios within data pipelines, offering best practices, code templates, and validation steps for ETL, transformation, and streaming workflows.

Guides you through building Dagster data pipelines with best-practice advice, ready-to-use configurations, and validation checks for ETL, transformation, orchestration, and streaming workflows whenever the Dagster pipeline creator skill is activated.

Automates guidance for data catalog updater tasks, covering ETL, transformation, orchestration, and streaming workflows to keep pipeline metadata accurate.

An expert in designing, implementing, and optimizing scalable, reliable, and cost-effective data pipelines for both batch and streaming data. Covers advanced topics including ETL/ELT patterns, Kappa/Lambda architectures, workflow orchestration (Airflow/Prefect), transformation using dbt/Spark, and managing modern storage formats like Delta Lake and Iceberg. Focuses heavily on data quality, monitoring, and cost optimization strategies.

Offers automated guidance for implementing data lineage trackers across ETL and streaming pipelines, including best practices on transformations, orchestration, and validation for production environments.

Automated Data Quality Checker offers guidance, best practices, and production-ready code for validating pipelines whenever you work with ETL, streaming, or data transformation scenarios.

Master the advanced process of database schema and data migrations across major ORMs (Sequelize, TypeORM, Prisma). This skill provides best practices for handling complex transformations, including implementing zero-downtime deployments, renaming columns, changing data types, and executing complex data restructuring. It emphasizes robust rollback strategies to ensure data integrity during critical database updates.

Automates dbt test creation within data pipelines, offering guided best practices, code generation, and validation to keep ETL, transformation, and streaming workflows reliable.

A guide to production-ready patterns for dbt (data build tool). It covers essential practices for robust data pipeline development, including model organization (staging, intermediate, marts), implementing advanced testing strategies, ensuring data quality, and optimizing incremental processing for large datasets.

Automates file format conversion guidance within data pipelines, covering ETL, workflow orchestration, data transformation, and streaming standards with production-ready examples.