Skill UI

Automates schema validation within data pipelines, providing guidance on best practices, workflow orchestration, and streaming ETL scenarios while producing production-ready code and configurations.

Automates spark job creation within data pipelines by guiding etl, transformation, orchestration, and streaming workflows; includes industry best practices, code templates, and validation to deliver production-ready jobs.

Automates Spark SQL optimizer tasks in data pipelines by guiding best practices, producing production-ready code and configurations, and validating ETL, transformation, orchestration, and streaming workflows.

Automated guidance for SQL transform helper tasks within data pipelines, covering ETL, data transformation, workflow orchestration, and streaming, while following best practices, generating production-ready code, and validating results.

This skill provides comprehensive read-only access to explore, query, and analyze datasets hosted on Hugging Face. It utilizes the Dataset Viewer API to list splits, preview rows, perform text searches, filter records, and retrieve crucial metadata and Parquet links. Furthermore, it supports advanced data extraction by enabling complex SQL querying (via parquetlens) directly against dataset shards, allowing export into CSV, JSON, or Parquet formats.

Polars is a lightning-fast DataFrame library built on Apache Arrow for Python. It provides an expression-based API, lazy query optimization, and parallel computation capabilities for efficient data manipulation, analytics, and ETL pipelines. It is designed to handle large, memory-intensive datasets, making it an ideal alternative for migrating from pandas or optimizing complex data workflows.

This skill provides the core mechanism for integrating Hex into external data pipelines. It enables triggering, monitoring, and managing Hex projects (e.g., via Airflow or Dagster) programmatically. Users can implement parameterized runs, execute multi-step ETL workflows sequentially, and handle status polling and error management, making it ideal for production data orchestration.

This reference architecture provides a comprehensive blueprint for building a robust revenue intelligence data platform integrated with Clari. It covers the entire data lifecycle, including API data extraction (ETL pipeline), data warehousing design (Snowflake/BQ), change detection logic, and advanced analytics/alerting layers. Use this guide when designing a scalable forecast tracking system, planning complex data integrations, or establishing best practices for enterprise revenue analytics.

A comprehensive guide to designing high-performance schemas in ClickHouse. Learn how to select the optimal MergeTree engine (e.g., ReplacingMergeTree, AggregatingMergeTree), structure the ORDER BY key for maximum query efficiency, and implement effective partitioning strategies for large-scale analytical workloads and ETL processes.

A comprehensive guide for migrating data to, from, and between Notion workspaces. This skill covers complex data pipelines, including rate-limited bulk imports (CSV/JSON), full database exports with pagination, cross-database syncing, and data transformation patterns for integrating content from sources like Confluence and Google Docs. Essential for building robust Notion ETL workflows.

A structured guide for setting up a robust local development environment for Palantir Foundry data pipelines. This workflow enables users to run Python transforms and unit tests using PySpark against sample data, and mock the Foundry API client, ensuring fast, isolated iteration cycles before deploying code to the official Foundry environment. Ideal for data engineers and developers building complex ETL logic.

A comprehensive guide for executing core Snowflake data loading workflows. This covers setting up stages (S3, GCS, Azure), defining file formats (CSV, JSON, Parquet), and using COPY INTO for bulk loading. It also details advanced continuous ingestion using Snowpipe, enabling real-time data synchronization from cloud storage into Snowflake tables.