Skill UI

Provides expert guidance on secure and performant file upload mechanisms for cloud environments. Covers best practices for using services like AWS S3 and Cloudflare R2, including implementing presigned URLs, handling large files via streaming, and optimizing images. Crucially emphasizes critical security measures such as validating file types using magic bytes, enforcing size limits, and preventing path traversal attacks.

This tool detects profitable arbitrage opportunities across centralized exchanges (CEX), decentralized exchanges (DEX), and cross-chain markets in real-time. It aggregates price data, calculates net profit after fees, and identifies various arbitrage paths, including direct, triangular, and cross-chain opportunities. It is designed for quantitative traders needing to monitor market efficiency and exploit price discrepancies.

This comprehensive skill covers the full scope of Firebase services, including Authentication, Firestore, Cloud Functions, and Hosting. It goes beyond basic implementation, focusing on critical architectural principles: adopting a query-first, denormalized NoSQL mindset. Key insights include mandatory security rule design, optimal data modeling for performance, and correctly utilizing modular SDKs to build scalable and secure backends.

A comprehensive guide to selecting and implementing the optimal Firecrawl architecture pattern for various data ingestion needs. Learn to choose between On-Demand, Scheduled Crawl Pipelines, and Real-Time Ingestion Pipelines based on data volume, required latency, and complexity. This is essential for designing scalable, production-grade scraping infrastructure, including RAG and knowledge base applications.

This guide details setting up automated Continuous Integration/Continuous Deployment (CI/CD) pipelines for Firecrawl scraping services. It covers configuring GitHub Actions, managing API keys, and implementing both mock unit tests and live integration tests. Use this when ensuring scraping behavior is validated automatically across pull requests and merges.

A comprehensive quick-reference guide for diagnosing and fixing common Firecrawl API errors and response codes. Covers critical issues such as API key validation (401), rate limiting (429), credit exhaustion (402), and handling dynamic content rendering failures in Single Page Applications (SPA). Use this when debugging failed web scraping or crawl job runs.

This workflow provides the primary mechanism for converting complex, live websites into clean, LLM-ready markdown content. It supports single-page scraping using `scrapeUrl`, deep multi-page site crawling with `crawlUrl`, and scalable async crawling for large datasets. Ideal for building robust content ingestion pipelines and reliably extracting structured information from the web.

This advanced workflow executes sophisticated web data gathering tasks. It supports three primary functions: LLM-powered structured data extraction using custom JSON schemas or prompts; efficient batch scraping of multiple known URLs; and rapid site mapping to discover all reachable URLs on a domain. Use this when you require reliable, typed data structures rather than raw markdown content.

This skill provides advanced techniques to significantly reduce API costs when using Firecrawl for web scraping. It covers best practices such as setting crawl limits, utilizing batch scraping for known URLs, implementing local caching, and monitoring credit usage. Learn how to efficiently scrape large sites while managing budgets and minimizing unexpected billing spikes.

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

A comprehensive guide for deploying web scraping applications powered by Firecrawl to various production environments. Covers Vercel serverless functions, Google Cloud Run, and self-hosted Docker setups. Learn how to configure platform-specific secrets, implement API routes, and handle asynchronous crawl results via webhooks for scalable, reliable production service.

This guide outlines how to implement robust Role-Based Access Control (RBAC) for Firecrawl scraping resources. It allows enterprises to manage multiple API keys, enforce per-team credit consumption limits, and restrict scraping to specific allowed domains (allowlisting). Use this when scaling scraping operations across various departments or services that require granular resource governance and budget tracking.