Skill UI

Offers automated guidance for implementing data lineage trackers across ETL and streaming pipelines, including best practices on transformations, orchestration, and validation for production environments.

This skill details how to set up automated CI/CD pipelines for Databricks data pipelines. It leverages Declarative Automation Bundles (DABs) and GitHub Actions to manage the entire deployment lifecycle. Key functionalities include running unit tests on pull requests, validating and deploying to staging environments, and promoting validated code to production upon merge, ensuring reliable and automated data infrastructure.

This guide provides comprehensive, code-based solutions for diagnosing and fixing common errors encountered in Databricks environments. It covers critical issues such as cluster state failures, Spark OutOfMemory errors, Delta Lake concurrency conflicts, permission problems, and schema mismatches, offering tailored fixes in Python, SQL, and Bash.

Use Declarative Automation Bundles (DABs) to manage and deploy Databricks assets, including jobs, DLT pipelines, and ML models, using an Infrastructure-as-Code approach. This skill allows defining resources and deployment targets (dev, staging, prod) in a single YAML file, enabling robust CI/CD practices and controlled promotion of changes across environments.

A comprehensive guide detailing optimization techniques for Databricks environments. It covers optimal cluster sizing for various workloads (ETL, ML, Streaming), advanced Spark configurations (AQE, memory management), and deep data optimization strategies for Delta Lake, including Z-Ordering and Liquid Clustering. It also provides methods for query plan analysis and join optimization.

This skill provides comprehensive, structured audit logging for OpenRouter API interactions. It automatically captures essential metadata like total cost, token usage, model details, latency, and status, while also performing PII redaction. The resulting audit trail is stored in an append-only SQLite database, making it ideal for building compliance-ready systems, detailed cost monitoring, and deep debugging of production environments.

This comprehensive guide outlines advanced strategies for optimizing PostHog in production environments. Key techniques covered include implementing local feature flag evaluation to eliminate network latency, optimizing event batching for network efficiency, utilizing strategic event sampling, and writing efficient HogQL queries. Master these methods to dramatically improve data collection reliability and query speed.

Guides production ML engineering with patterns for deploying models, building MLOps pipelines, monitoring drift, integrating LLMs and RAG systems, and enforcing feature-store, retraining, and cost controls in live environments.

This guide details how to detect WMI event subscription persistence, a common attacker technique (T1546.003). It focuses on analyzing suspicious Sysmon Event IDs 19, 20, and 21 to identify malicious EventFilters, EventConsumers, and Bindings. Essential for incident response and threat hunting in Windows environments.

Provides a detailed, multi-platform workflow to monitor long-running computational experiments. It checks the status of jobs across various environments (SSH, vast.ai, Modal), collects screen logs, fetches structured JSON results, and pulls detailed training metrics (loss curves, accuracy) from Weights & Biases (W&B). Essential for MLOps and automated result review.

This comprehensive tool manages the complete lifecycle of ML experiments, providing robust deployment capabilities across diverse environments. It supports running training jobs on local GPUs, dedicated remote servers, Vast.ai, and serverless platforms like Modal. The workflow handles environment detection, code synchronization (rsync/git), dependency pinning, and integrates advanced logging tools (e.g., W&B), ensuring reliable, scalable, and reproducible ML training runs.

A comprehensive guide for advanced geospatial data processing using GeoPandas. This skill targets professional workflows involving planar vector data, emphasizing rigorous data provenance, coordinate reference system (CRS) management, and complex spatial operations. It details necessary safety contracts, correctness gates, and best practices for handling sensitive geographic information in reproducible Python environments.