Skill UI

This guide provides comprehensive, code-based solutions for diagnosing and fixing common errors encountered in Databricks environments. It covers critical issues such as cluster state failures, Spark OutOfMemory errors, Delta Lake concurrency conflicts, permission problems, and schema mismatches, offering tailored fixes in Python, SQL, and Bash.

Use Declarative Automation Bundles (DABs) to manage and deploy Databricks assets, including jobs, DLT pipelines, and ML models, using an Infrastructure-as-Code approach. This skill allows defining resources and deployment targets (dev, staging, prod) in a single YAML file, enabling robust CI/CD practices and controlled promotion of changes across environments.

A comprehensive guide to setting up a fast, isolated local development loop for data engineering projects using Databricks Connect v2, Asset Bundles, and standard IDEs (like VS Code). This workflow allows developers to run PySpark code locally for debugging and unit testing, while managing resource deployment and ensuring consistency across development, staging, and production environments.

This skill guides the setup and management of Databricks resources across development, staging, and production environments. It covers best practices for using isolated workspaces, managing environment-specific secrets, implementing Asset Bundles, and integrating with CI/CD pipelines to ensure consistent and reliable data deployments.

A comprehensive guide detailing optimization techniques for Databricks environments. It covers optimal cluster sizing for various workloads (ETL, ML, Streaming), advanced Spark configurations (AQE, memory management), and deep data optimization strategies for Delta Lake, including Z-Ordering and Liquid Clustering. It also provides methods for query plan analysis and join optimization.

A comprehensive, expert-level workflow for debugging complex software issues in production environments. This skill guides users through initial triage, advanced observability data collection (APM, distributed traces, logs), intelligent hypothesis generation, and full root cause analysis. Use it when dealing with intermittent failures, performance bottlenecks, or needing a complete fix and prevention plan.

Provides comprehensive instructions for deploying Deepgram transcription services into various production environments. This guide covers building production-ready Docker images, configuring Docker Compose, setting up robust Kubernetes manifests (including HPA and secrets), and implementing serverless handlers for platforms like AWS Lambda. Essential for scaling Deepgram services reliably and securely in the cloud.

A comprehensive guide for installing and configuring the Deepgram SDK for both Node.js and Python environments. This skill covers best practices for secure API key management using environment variables, client initialization (including version differences), and connection verification. Use this when integrating Deepgram's advanced speech-to-text and text-to-speech capabilities into a new application.

A comprehensive guide to deployment patterns and strategies for modern cloud environments. Covers techniques like Blue/Green, Canary, and Rolling Deployments, ensuring zero-downtime upgrades when deploying to Kubernetes, Docker, or various cloud providers. Includes best practices for health checks, logging, and rollback planning.

A comprehensive expert for configuration management, specializing in validating, testing, and ensuring the correctness of application settings. It creates robust validation schemas (e.g., JSON Schema) and performs critical checks, including identifying potential security secrets (API keys, passwords), ensuring data consistency across different environments (development, staging, production), and enforcing type safety for reliable deployments.

Detects discrepancies between current infrastructure and desired configurations by running drift-detect, helping DevOps teams ensure Terraform-managed environments stay consistent and report configuration differences.

Blueprint for designing GitHub Actions-powered CI/CD pipelines that leverage Docker multi-stage builds, Helm chart environments, ArgoCD GitOps, Kubernetes manifests, and monitoring best practices to keep deployments reliable across teams.