Skill UI

A systematic guide for debugging complex, hard-to-diagnose issues with Firecrawl. Use this when standard troubleshooting fails, such as when scrapes return empty content, crawl jobs hang indefinitely, or webhooks fail to fire. It employs layer-by-layer isolation techniques, covering API connectivity, target URL accessibility, and content quality to pinpoint failure points.

This guide details setting up automated Continuous Integration/Continuous Deployment (CI/CD) pipelines for Firecrawl scraping services. It covers configuring GitHub Actions, managing API keys, and implementing both mock unit tests and live integration tests. Use this when ensuring scraping behavior is validated automatically across pull requests and merges.

A comprehensive quick-reference guide for diagnosing and fixing common Firecrawl API errors and response codes. Covers critical issues such as API key validation (401), rate limiting (429), credit exhaustion (402), and handling dynamic content rendering failures in Single Page Applications (SPA). Use this when debugging failed web scraping or crawl job runs.

This advanced workflow executes sophisticated web data gathering tasks. It supports three primary functions: LLM-powered structured data extraction using custom JSON schemas or prompts; efficient batch scraping of multiple known URLs; and rapid site mapping to discover all reachable URLs on a domain. Use this when you require reliable, typed data structures rather than raw markdown content.

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

This guide outlines how to implement robust Role-Based Access Control (RBAC) for Firecrawl scraping resources. It allows enterprises to manage multiple API keys, enforce per-team credit consumption limits, and restrict scraping to specific allowed domains (allowlisting). Use this when scaling scraping operations across various departments or services that require granular resource governance and budget tracking.

A comprehensive runbook providing systematic procedures for responding to Firecrawl API outages and integration failures. It guides technical teams through triage, diagnosis, and mitigation for common issues (e.g., 401, 429, 5xx errors). Includes steps for fallback implementation, evidence collection, and postmortem documentation to ensure rapid service recovery.

This guide provides comprehensive instructions for setting up the Firecrawl SDK, covering both Node.js and Python environments. It details how to install the necessary packages via npm or pip, and how to securely configure the API key using environment variables. Use this guide when initializing web scraping functionality or integrating Firecrawl into a new development project.

This comprehensive guide details how to set up a robust, local development environment for Firecrawl integrations. It utilizes self-hosted Docker containers and Redis to eliminate reliance on paid API credits during development. The setup supports both unit testing (with mocked SDK) and full integration testing against the local instance, ensuring a seamless and efficient workflow for scraping and web data extraction projects.

This guide details how to configure Firecrawl for reliable multi-environment operations (Development, Staging, Production). It addresses critical operational concerns, such as managing credit budgets and ensuring separation between environments. Users learn how to implement self-hosted Docker setups for development, manage environment-specific API keys, and integrate safe scraping wrappers into CI/CD pipelines.

Provides comprehensive observability patterns for Firecrawl web scraping operations. It covers key metrics such as scrape success rates, credit consumption tracking, content quality assessment, and latency monitoring. Use this skill for implementing robust monitoring dashboards, setting up proactive failure alerts, and optimizing the reliability of web crawling workflows.

This comprehensive guide details advanced techniques to significantly boost Firecrawl scraping speed and efficiency. Learn how to optimize performance by selecting the most efficient output formats (e.g., Markdown over HTML), implementing URL caching, utilizing batch scraping for multiple URLs, and tuning delay parameters for dynamic Single Page Applications (SPAs). Mastering these methods is crucial for building high-throughput, cost-effective web scraping pipelines.