Skill UI

This comprehensive diagnostic bundle collects critical evidence from the Databricks environment, including environment setup, cluster state, job run history, and Delta table diagnostics. It generates a redacted, safe tar.gz file specifically designed for submitting detailed support tickets and efficiently troubleshooting complex data pipeline issues.

This comprehensive checklist guides the deployment of Databricks data pipelines and jobs to a production environment. It covers critical stages including pre-deployment security hardening (Service Principals, Unity Catalog), validating infrastructure (cluster types, autoscaling), ensuring code quality (unit/integration tests, Delta Lake best practices), executing controlled deployment, and performing post-deployment health checks and monitoring setup. Ideal for CI/CD pipelines and go-live procedures.

A comprehensive guide to implementing best practices for security within Databricks. Learn how to manage secrets using scoped storage, enforce the principle of least privilege via Unity Catalog grants, audit API tokens, and apply advanced data protection techniques like column-level masking and row-level security for enhanced data governance and compliance.

This comprehensive guide provides structured steps and code examples for upgrading Databricks Runtime versions and performing critical data migrations. It covers best practices for version compatibility checking, transitioning from Hive Metastore to Unity Catalog using methods like SYNC and CTAS, and updating deprecated API endpoints, ensuring a smooth platform evolution for data pipelines.

This skill provides a comprehensive guide to configuring event-driven workflows and robust notification systems for Databricks jobs. It covers setting up various notification destinations (Slack, Teams, Email, PagerDuty) and triggering alerts based on job lifecycle events (success, failure, start). Furthermore, it demonstrates advanced monitoring using system audit tables and creating automated SQL alerts for critical data pipeline monitoring.

Guides you through generating dbt models inside data pipelines, covering setup, code generation, configuration, and validation while following best practices for ETL, orchestration, and streaming workloads.

Automates dbt test creation within data pipelines, offering guided best practices, code generation, and validation to keep ETL, transformation, and streaming workflows reliable.

Enables Claude to apply anomaly-detection-system workflows for spotting outliers, fraud, or unusual patterns across datasets, selecting tailored algorithms like Isolation Forest and summarizing deviations for any data monitoring or security use case.

Automated guidance for configuring and applying early stopping callbacks across ML training workflows, covering best practices, code generation, and validation for PyTorch, TensorFlow, and scikit-learn experiments.

Provides detailed evaluation of machine learning models using plugin metrics to assess accuracy, precision, recall, F1, and other key indicators, helping users compare models and validate performance before deployment.

Automates guidance for setting up and validating feature store connectors within ML deployment pipelines, covering best practices, code generation, and production monitoring for inference.

This tool detects profitable arbitrage opportunities across centralized exchanges (CEX), decentralized exchanges (DEX), and cross-chain markets in real-time. It aggregates price data, calculates net profit after fees, and identifies various arbitrage paths, including direct, triangular, and cross-chain opportunities. It is designed for quantitative traders needing to monitor market efficiency and exploit price discrepancies.