Skill UI

Guides data engineering patterns for ETL pipelines, Spark analytics, and quality checks, showcasing Python and SQL snippets to extract, transform, load, validate, and model sales data in modern warehouses.

Provides comprehensive patterns for data synchronization, ETL processes, and data migration specifically for MaintainX. Use this when needing to keep data consistent between MaintainX and external databases, data warehouses (like BigQuery), or when performing bulk data exports to formats like CSV. It covers incremental syncing, full exports, and data reconciliation checks.

This comprehensive guide details production-grade architectural patterns for integrating MaintainX with large enterprise systems, including ERPs, SCADA, and data warehouses. It covers best practices for event-driven synchronization using webhooks and Pub/Sub, implementing bi-directional sync gateways, and establishing scalable analytics data pipelines.

This comprehensive guide details the expertise needed for a Senior Data Engineer. It covers building, optimizing, and governing scalable data pipelines and robust ETL/ELT systems using modern tools like Spark, Kafka, Airflow, and dbt. Includes decision frameworks for architecting data solutions (Batch vs Streaming, Lakehouse vs Warehouse) and implementing advanced data modeling patterns.

Automate Benchling lab data workflows via the Python SDK or REST API: manage registry entities (DNA, proteins), inventory, ELN entries, workflows, and Data Warehouse queries for syncing and analytics.

This skill provides comprehensive integration with the Benchling cloud platform using the Python SDK and REST API. It allows users to programmatically manage critical life sciences data, including registry entities (DNA/RNA/proteins), inventory tracking, ELN entries, workflows, and querying the Data Warehouse. Ideal for automating complex lab data pipelines and syncing Benchling with external systems.

This guide provides advanced methods to significantly boost Clari API performance and data pipeline efficiency. It covers parallelizing multi-period data exports using Python's concurrency features, implementing a robust local caching mechanism with Time-To-Live (TTL), and optimizing data warehouse loading using SQL MERGE statements. Use this when facing slow exports, high API call volume, or requiring fast data synchronization in multi-forecast environments.

Provides production-grade strategies for synchronizing Navan's BOOKING and TRANSACTION data into a data warehouse. This skill covers three sync tiers: weekly full-refresh with idempotent merge-upsert logic for BOOKING, watermark-based incremental sync for TRANSACTION, and real-time webhook handling for event-driven processing. Essential for building robust, scalable data pipelines using ETL patterns.

This comprehensive guide demonstrates how to establish a robust Continuous Integration/Continuous Deployment (CI/CD) pipeline for Snowflake data warehousing. It integrates SchemaChange for managing version-controlled database migrations, GitHub Actions for automated testing and deployment triggering, and Terraform for provisioning and managing underlying Snowflake infrastructure (databases, warehouses, roles). Use this when setting up reliable, repeatable, and automated deployment workflows.

This guide provides minimal working examples for connecting to Snowflake and executing core SQL operations. It comprehensively covers connection setup, running simple SELECT queries, creating databases and tables, and performing parameterized inserts using both Python and Node.js. It is ideal for initial environment setup, testing the Snowflake SDK, or learning fundamental data warehouse interactions.

This guide provides comprehensive steps to install and configure Snowflake SDKs for Node.js and Python. It covers various authentication methods, including password, secure Key Pair authentication (ideal for CI/CD), and enterprise-grade SSO/OAuth integration. Use this when setting up data pipelines or applications that require reliable and secure connectivity to the Snowflake data warehouse.

This comprehensive guide details how to set up robust, isolated, and secure development, staging, and production environments within Snowflake. It utilizes best practices like zero-copy cloning for efficient data replication, implements granular Role-Based Access Control (RBAC), and defines environment-specific compute resources (warehouses) and data masking policies. This ensures data integrity, compliance, and cost optimization across all stages of the data lifecycle.