Skill UI

A comprehensive guide to setting up a minimal working example on Databricks. This skill covers creating a development cluster, uploading and executing notebooks (Python SDK), running tasks as jobs, and setting up a Serverless SQL Warehouse. It is ideal for new users or initializing a basic data engineering project.

This skill automatically generates an interactive SQL Notebook designed to validate changes in dbt models or snapshots. It processes changes identified from a GitHub PR or local repository, generating comparison queries that compare 'before' (production) and 'after' (development) data states. It is essential for embedding data observability and quality checks into CI/CD pipelines.

Matplotlib is a foundational Python library for creating static, animated, and interactive plots with full customization. It supports various plot types like line, scatter, bar, and heatmaps, suitable for scientific and statistical visualizations. Users can customize appearance, create multi-panel figures, and export to PNG, PDF, or SVG. Ideal for publication-quality graphics and integrating with Jupyter notebooks.

A comprehensive guide and toolkit for optimizing Hex platform costs. This resource details strategies for reducing compute expenditure by auditing seat licenses, consolidating data connections, and optimizing notebook run schedules. It includes code examples for implementing result caching and usage monitoring to prevent unexpected billing spikes.

Execute arbitrary PySpark or pure Python code directly on Fabric Spark compute using the Livy API. This method creates ephemeral sessions without generating persistent notebook artifacts. It provides full read/write access to lakehouse Delta tables, making it ideal for agent-driven ETL pipelines, quick data transformations, or analyses where session cleanup is critical. Users must manage the session lifecycle (create, run, and delete).

A comprehensive guide on querying data stored in Fabric's OneLake using DuckDB. This skill supports two operational modes: local command line interface (CLI) for ad-hoc analysis, and in-notebook mode for seamless integration within Fabric notebooks. It can read Delta Lake tables and various raw formats (Parquet, CSV, JSON), making it ideal for data exploration, validation, and schema discovery in a read-only fashion.

Generates SQL validation notebooks for dbt PR changes with before/after comparison queries. Supports GitHub PR URLs and local repositories. Creates interactive notebooks for database validation across Snowflake, BigQuery, Redshift, and Athena. Includes schema inference from dbt_project.yml and model configs.