Skill UI

A comprehensive guide to production-ready patterns for the Databricks Python SDK. This skill covers essential best practices including singleton client initialization, structured error handling with typed exceptions, context-managed cluster lifecycle control (ensuring cleanup), and implementing type-safe job definitions. It is essential for building robust, maintainable, and idiomatic integrations with the Databricks platform.

A comprehensive guide to implementing best practices for security within Databricks. Learn how to manage secrets using scoped storage, enforce the principle of least privilege via Unity Catalog grants, audit API tokens, and apply advanced data protection techniques like column-level masking and row-level security for enhanced data governance and compliance.

This comprehensive guide provides structured steps and code examples for upgrading Databricks Runtime versions and performing critical data migrations. It covers best practices for version compatibility checking, transitioning from Hive Metastore to Unity Catalog using methods like SYNC and CTAS, and updating deprecated API endpoints, ensuring a smooth platform evolution for data pipelines.

This skill provides a comprehensive guide to configuring event-driven workflows and robust notification systems for Databricks jobs. It covers setting up various notification destinations (Slack, Teams, Email, PagerDuty) and triggering alerts based on job lifecycle events (success, failure, start). Furthermore, it demonstrates advanced monitoring using system audit tables and creating automated SQL alerts for critical data pipeline monitoring.

Guides using Rube MCP’s Composio toolkit to discover current Datagma tools, verify connections, and execute workflows while adhering to live schemas, session reuse, and required memory parameters.

Automates dataset loader creation within ML Training workflows, offering best-practice guidance, validation checks, and production-ready code to accelerate data prep and model tuning.

Automates guidance for date range analytics, covering SQL, visualization, statistics, and BI, with code generation and validation to support data analytics workflows.

This guide provides essential best practices for building reliable, fault-tolerant applications in Go using DBOS durable workflows. It covers architectural patterns for integrating DBOS into existing Go code, defining complex workflows, managing steps, and utilizing queues for robust concurrency control. Essential reading for developers setting up, configuring, or testing mission-critical asynchronous processes.

A comprehensive guide detailing best practices for building reliable, fault-tolerant Python applications utilizing DBOS durable workflows. This resource is essential when integrating DBOS into existing Python codebases, defining complex workflows and steps, managing concurrency via queues, or implementing robust workflow communication patterns.

A comprehensive guide for building reliable, fault-tolerant applications using DBOS durable workflows within a TypeScript environment. It covers best practices for initialization, structuring complex workflows and steps, managing concurrency via queues, and integrating DBOS into existing TypeScript codebases.

Guides you through generating dbt models inside data pipelines, covering setup, code generation, configuration, and validation while following best practices for ETL, orchestration, and streaming workloads.

Automates dbt test creation within data pipelines, offering guided best practices, code generation, and validation to keep ETL, transformation, and streaming workflows reliable.