Skill UI

Master the implementation of Firecrawl webhooks for robust, real-time handling of asynchronous crawling and batch scraping jobs. Instead of inefficiently polling the crawl status, configure webhook endpoints to receive instant POST events (e.g., `crawl.page`, `crawl.completed`). This guide covers event types, signature verification using HMAC-SHA256, and processing streamed data for building advanced data pipelines.

A comprehensive guide and implementation template for building robust data ingestion pipelines into ClickHouse. Supports multiple sources including HTTP webhooks (with batching), Kafka streams, S3, and direct SQL inserts. Demonstrates best practices for data handling, such as deduplication using ReplacingMergeTree and monitoring query logs, ensuring reliable, scalable data analytics.

This comprehensive guide details the process of implementing a secure and robust system for handling Palantir Foundry webhook events. It covers key steps including API-based webhook registration, mandatory signature verification with replay protection, sophisticated event routing for various object lifecycles (create, update, delete), and ensuring idempotent processing using caching mechanisms like Redis. Use this when external microservices must react reliably and immediately to changes within the Foundry Ontology or data assets.