Skill UI

Guides users through compression optimizer scenarios within data pipelines, offering best practices, code templates, and validation steps for ETL, transformation, and streaming workflows.

Guides you through building Dagster data pipelines with best-practice advice, ready-to-use configurations, and validation checks for ETL, transformation, orchestration, and streaming workflows whenever the Dagster pipeline creator skill is activated.

Guides ML training teams through data augmentation pipelines, offering best practices, code/config generation, validation, and experiment-tracking support so pipelines stay production-ready.

A comprehensive, end-to-end workflow for building product features guided by rigorous data insights. This skill guides users through the entire lifecycle, encompassing exploratory data analysis, forming measurable business hypotheses, designing statistical A/B tests, architecting data pipelines, and implementing robust backend/frontend instrumentation. It ensures that every feature iteration is validated by data and measured for impact.

Offers automated guidance for implementing data lineage trackers across ETL and streaming pipelines, including best practices on transformations, orchestration, and validation for production environments.

Guides engineers through data partitioner setups within ETL workflows, offering best practice advice, ready-to-run code, and validation for streaming and batch pipelines.

Automated Data Quality Checker offers guidance, best practices, and production-ready code for validating pipelines whenever you work with ETL, streaming, or data transformation scenarios.

This skill provides production patterns for implementing robust data quality validation using Great Expectations, dbt tests, and formal data contracts. It is essential for building reliable, automated, and verifiable data pipelines, ensuring data integrity throughout the entire ETL/ELT process, and establishing clear data governance rules.

This skill details how to set up automated CI/CD pipelines for Databricks data pipelines. It leverages Declarative Automation Bundles (DABs) and GitHub Actions to manage the entire deployment lifecycle. Key functionalities include running unit tests on pull requests, validating and deploying to staging environments, and promoting validated code to production upon merge, ensuring reliable and automated data infrastructure.

This comprehensive diagnostic bundle collects critical evidence from the Databricks environment, including environment setup, cluster state, job run history, and Delta table diagnostics. It generates a redacted, safe tar.gz file specifically designed for submitting detailed support tickets and efficiently troubleshooting complex data pipeline issues.

Use Declarative Automation Bundles (DABs) to manage and deploy Databricks assets, including jobs, DLT pipelines, and ML models, using an Infrastructure-as-Code approach. This skill allows defining resources and deployment targets (dev, staging, prod) in a single YAML file, enabling robust CI/CD practices and controlled promotion of changes across environments.

This comprehensive checklist guides the deployment of Databricks data pipelines and jobs to a production environment. It covers critical stages including pre-deployment security hardening (Service Principals, Unity Catalog), validating infrastructure (cluster types, autoscaling), ensuring code quality (unit/integration tests, Delta Lake best practices), executing controlled deployment, and performing post-deployment health checks and monitoring setup. Ideal for CI/CD pipelines and go-live procedures.