Skill UI

This skill details how to set up automated CI/CD pipelines for Databricks data pipelines. It leverages Declarative Automation Bundles (DABs) and GitHub Actions to manage the entire deployment lifecycle. Key functionalities include running unit tests on pull requests, validating and deploying to staging environments, and promoting validated code to production upon merge, ensuring reliable and automated data infrastructure.

Use Declarative Automation Bundles (DABs) to manage and deploy Databricks assets, including jobs, DLT pipelines, and ML models, using an Infrastructure-as-Code approach. This skill allows defining resources and deployment targets (dev, staging, prod) in a single YAML file, enabling robust CI/CD practices and controlled promotion of changes across environments.

Master the creation of robust ELT (Extract, Load, Transform) data pipelines within Snowflake. This guide demonstrates advanced data orchestration using Snowflake Streams for Change Data Capture (CDC), Tasks for scheduled transformations and DAG management, and Dynamic Tables for declarative, auto-refreshing data models. Ideal for building sophisticated, scalable data warehousing solutions.