Skill UI

This skill guides the setup and management of Databricks resources across development, staging, and production environments. It covers best practices for using isolated workspaces, managing environment-specific secrets, implementing Asset Bundles, and integrating with CI/CD pipelines to ensure consistent and reliable data deployments.

This comprehensive checklist guides the deployment of Databricks data pipelines and jobs to a production environment. It covers critical stages including pre-deployment security hardening (Service Principals, Unity Catalog), validating infrastructure (cluster types, autoscaling), ensuring code quality (unit/integration tests, Delta Lake best practices), executing controlled deployment, and performing post-deployment health checks and monitoring setup. Ideal for CI/CD pipelines and go-live procedures.

This comprehensive guide provides structured steps and code examples for upgrading Databricks Runtime versions and performing critical data migrations. It covers best practices for version compatibility checking, transitioning from Hive Metastore to Unity Catalog using methods like SYNC and CTAS, and updating deprecated API endpoints, ensuring a smooth platform evolution for data pipelines.

Guides you through generating dbt models inside data pipelines, covering setup, code generation, configuration, and validation while following best practices for ETL, orchestration, and streaming workloads.

Automates dbt test creation within data pipelines, offering guided best practices, code generation, and validation to keep ETL, transformation, and streaming workflows reliable.

This skill provides comprehensive methods to optimize Deepgram API usage and manage costs. It implements budget-aware transcription pipelines that preemptively check projected spending against set limits, utilizes audio preprocessing techniques (like silence removal via ffmpeg) to minimize billable minutes, and includes API calls to monitor current usage and spending against defined budgets. Ideal for cost-sensitive, production-grade deployments.

Provides reference architectures for building high-scale, reliable transcription systems using Deepgram. It covers four main patterns: synchronous REST for short files, async queues (BullMQ) for batch processing, WebSocket for real-time streaming, and a hybrid router for mixed workloads. Use this when designing production-grade audio processing pipelines.

Provides production-ready coding patterns for integrating Deepgram using TypeScript and Python. This guide covers essential best practices, including implementing singleton client management, performing text-to-speech synthesis with Aura, and setting up advanced audio intelligence pipelines (e.g., summarization, topic detection, sentiment analysis). Use this when establishing robust Deepgram integration standards, refactoring existing SDK usage, or migrating to newer SDK versions.

Expert in modern CI/CD pipelines, GitOps workflows, and platform engineering. Specializes in designing and implementing secure, zero-downtime deployment strategies for enterprise-scale applications. Proficient with Kubernetes, ArgoCD, Terraform, and advanced security compliance practices.

This guide outlines advanced architectural patterns for building multi-stage CI/CD pipelines. It covers best practices for implementing robust deployment gates, including manual approvals, time-based triggers, and multi-approver workflows. It details various deployment strategies like Rolling, Blue-Green, and Canary releases, ensuring controlled, zero-downtime, and highly reliable software releases.

Automates Detrack operations through Composio’s Rube MCP toolkit by discovering available tools, managing connections, and executing workflows with schema-compliant arguments; ideal for teams integrating Detrack tasks inside Rube-driven pipelines.

Blueprint for designing GitHub Actions-powered CI/CD pipelines that leverage Docker multi-stage builds, Helm chart environments, ArgoCD GitOps, Kubernetes manifests, and monitoring best practices to keep deployments reliable across teams.