Skill UI

Automates dbt test creation within data pipelines, offering guided best practices, code generation, and validation to keep ETL, transformation, and streaming workflows reliable.

A guide to production-ready patterns for dbt (data build tool). It covers essential practices for robust data pipeline development, including model organization (staging, intermediate, marts), implementing advanced testing strategies, ensuring data quality, and optimizing incremental processing for large datasets.

Automates file format conversion guidance within data pipelines, covering ETL, workflow orchestration, data transformation, and streaming standards with production-ready examples.

This workflow provides the primary mechanism for converting complex, live websites into clean, LLM-ready markdown content. It supports single-page scraping using `scrapeUrl`, deep multi-page site crawling with `crawlUrl`, and scalable async crawling for large datasets. Ideal for building robust content ingestion pipelines and reliably extracting structured information from the web.

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

Firecrawl provides a comprehensive API for web data harvesting. It enables users to perform single-page scraping, multi-page crawling, site mapping (URL discovery), and advanced structured data extraction using Large Language Models (LLMs). Ideal for integrating data pipelines, building web monitoring tools, or conducting market research.

This comprehensive reference architecture details a multi-stage pipeline for web scraping and content ingestion using Firecrawl. It seamlessly integrates single-page scraping, site-wide crawling, URL mapping, and structured data extraction. The subsequent processing layer handles cleaning, deduplication, and chunking, making this blueprint ideal for designing robust knowledge bases, RAG systems, and complex AI data pipelines.

Master the implementation of Firecrawl webhooks for robust, real-time handling of asynchronous crawling and batch scraping jobs. Instead of inefficiently polling the crawl status, configure webhook endpoints to receive instant POST events (e.g., `crawl.page`, `crawl.completed`). This guide covers event types, signature verification using HMAC-SHA256, and processing streamed data for building advanced data pipelines.

Automated assistance for Flink job creation in data pipelines, triggered by Flink job creator requests. Guides ETL, streaming, and workflow orchestration, generates production-ready code/configurations, and validates outputs against industry standards.

This skill provides comprehensive guidelines for monitoring Granola usage, tracking team meeting patterns, and building detailed analytics dashboards. It guides users in defining critical metrics (e.g., adoption rate, efficiency score) and establishing robust data pipelines. Users can stream meeting metadata from Granola to data warehouses like BigQuery, enabling deep, customizable reporting and automated weekly insights via services like Zapier and Slack.

Guides data-engineering teams through configuring incremental loads in pipelines, covering best practices, orchestration tips, example code, and output validation for streaming or batch workflows.

Configure automated Continuous Integration and Continuous Deployment (CI/CD) pipelines for Juicebox AI data analysis integrations. This setup utilizes GitHub Actions to run comprehensive unit tests (with mocked datasets) and integration tests (connecting to live APIs) on every PR and merge to main. It ensures that data upload logic, analysis execution, and result parsing workflows are robust and reliable before deployment.