Skill UI

A robust solution for implementing and handling real-time webhooks from Customer.io. This tool processes critical marketing events, including delivery status, opens, clicks, bounces, and unsubscribes. Key features include HMAC signature verification for security, reliable asynchronous event processing, and structured routing to manage various customer interactions, allowing data to be streamed efficiently to a data warehouse or CRM.

Expert in designing, building, and optimizing scalable data pipelines, modern data warehouses, and real-time streaming architectures. Mastery covers the complete modern data stack, including Spark, dbt, Airflow, and cloud services (AWS, Azure, GCP). Focuses on ELT/ETL, data lakehouse implementation (Delta Lake, Iceberg), data governance, and ensuring high data quality and operational reliability.

Guides data engineering patterns for ETL pipelines, Spark analytics, and quality checks, showcasing Python and SQL snippets to extract, transform, load, validate, and model sales data in modern warehouses.

This guide provides a comprehensive framework for minimizing cloud expenditure on Databricks. It covers identifying cost drivers via system billing tables, implementing cluster policies, utilizing spot instances for batch processing, right-sizing SQL warehouses, and automating the shutdown of idle development clusters. Use it when managing cloud spending or optimizing data platform usage.

A comprehensive guide to setting up a minimal working example on Databricks. This skill covers creating a development cluster, uploading and executing notebooks (Python SDK), running tasks as jobs, and setting up a Serverless SQL Warehouse. It is ideal for new users or initializing a basic data engineering project.

This skill provides comprehensive guidelines for monitoring Granola usage, tracking team meeting patterns, and building detailed analytics dashboards. It guides users in defining critical metrics (e.g., adoption rate, efficiency score) and establishing robust data pipelines. Users can stream meeting metadata from Granola to data warehouses like BigQuery, enabling deep, customizable reporting and automated weekly insights via services like Zapier and Slack.

Provides comprehensive patterns for data synchronization, ETL processes, and data migration specifically for MaintainX. Use this when needing to keep data consistent between MaintainX and external databases, data warehouses (like BigQuery), or when performing bulk data exports to formats like CSV. It covers incremental syncing, full exports, and data reconciliation checks.

This comprehensive guide details production-grade architectural patterns for integrating MaintainX with large enterprise systems, including ERPs, SCADA, and data warehouses. It covers best practices for event-driven synchronization using webhooks and Pub/Sub, implementing bi-directional sync gateways, and establishing scalable analytics data pipelines.

This comprehensive guide details the expertise needed for a Senior Data Engineer. It covers building, optimizing, and governing scalable data pipelines and robust ETL/ELT systems using modern tools like Spark, Kafka, Airflow, and dbt. Includes decision frameworks for architecting data solutions (Batch vs Streaming, Lakehouse vs Warehouse) and implementing advanced data modeling patterns.

Automates Snowflake data warehouse workflows via the Composio MCP, letting you list databases, schemas, and tables, run SQL (SELECT, DDL, DML), and integrate those operations into cross-app pipelines for faster data ops.

Automate Benchling lab data workflows via the Python SDK or REST API: manage registry entities (DNA, proteins), inventory, ELN entries, workflows, and Data Warehouse queries for syncing and analytics.

An expert agent designed to collect critical data assets—including metadata, data lineage, and query logs—from any data warehouse platform. It uses the push ingestion API to send this data directly to Monte Carlo. This method is essential for integrating data sources that cannot be reliably covered by standard pull collectors, such as custom lineage paths or historical query logs.