Download

Skill UI

Browse and discover 9677+ curated skills

All Development Artificial Intelligence Design & Creative Product & Business Data Science Marketing Soft Skills Productivity Engineering Languages

Search Crawl , found 59 results

Default Newest Most Downloaded

Apify Automation Toolkit

apify-automation

ComposioHQ/awesome-claude-skills

Automate Apify web scraping Actors inside Claude Code to run crawlers synchronously or asynchronously, manage datasets, inspect logs, create reusable tasks, and pull structured results without leaving your workflow.

Advanced Firecrawl Debugging and Isolation

firecrawl-advanced-troubleshooting

jeremylongshore/claude-code-plugins-plus-skills

A systematic guide for debugging complex, hard-to-diagnose issues with Firecrawl. Use this when standard troubleshooting fails, such as when scrapes return empty content, crawl jobs hang indefinitely, or webhooks fail to fire. It employs layer-by-layer isolation techniques, covering API connectivity, target URL accessibility, and content quality to pinpoint failure points.

Firecrawl Architecture Variants Guide

firecrawl-architecture-variants

jeremylongshore/claude-code-plugins-plus-skills

A comprehensive guide to selecting and implementing the optimal Firecrawl architecture pattern for various data ingestion needs. Learn to choose between On-Demand, Scheduled Crawl Pipelines, and Real-Time Ingestion Pipelines based on data volume, required latency, and complexity. This is essential for designing scalable, production-grade scraping infrastructure, including RAG and knowledge base applications.

Firecrawl Web Automation

firecrawl-automation

ComposioHQ/awesome-claude-skills

Firecrawl Automation lets you use Claude Code to scrape pages, crawl sites, batch URLs, extract structured data, and monitor jobs via the Composio integration, ideal for terminal-driven web data pipelines.

Automated Testing for Firecrawl CI/CD

firecrawl-ci-integration

jeremylongshore/claude-code-plugins-plus-skills

This guide details setting up automated Continuous Integration/Continuous Deployment (CI/CD) pipelines for Firecrawl scraping services. It covers configuring GitHub Actions, managing API keys, and implementing both mock unit tests and live integration tests. Use this when ensuring scraping behavior is validated automatically across pull requests and merges.

Firecrawl Error Debugging and Troubleshooting Guide

firecrawl-common-errors

jeremylongshore/claude-code-plugins-plus-skills

A comprehensive quick-reference guide for diagnosing and fixing common Firecrawl API errors and response codes. Covers critical issues such as API key validation (401), rate limiting (429), credit exhaustion (402), and handling dynamic content rendering failures in Single Page Applications (SPA). Use this when debugging failed web scraping or crawl job runs.

Website Scraping and Crawling Pipeline

firecrawl-core-workflow-a

jeremylongshore/claude-code-plugins-plus-skills

This workflow provides the primary mechanism for converting complex, live websites into clean, LLM-ready markdown content. It supports single-page scraping using `scrapeUrl`, deep multi-page site crawling with `crawlUrl`, and scalable async crawling for large datasets. Ideal for building robust content ingestion pipelines and reliably extracting structured information from the web.

Structured Data Extraction, Batch Scraping, and Mapping

firecrawl-core-workflow-b

jeremylongshore/claude-code-plugins-plus-skills

This advanced workflow executes sophisticated web data gathering tasks. It supports three primary functions: LLM-powered structured data extraction using custom JSON schemas or prompts; efficient batch scraping of multiple known URLs; and rapid site mapping to discover all reachable URLs on a domain. Use this when you require reliable, typed data structures rather than raw markdown content.

Firecrawl Cost Optimization Guide

firecrawl-cost-tuning

jeremylongshore/claude-code-plugins-plus-skills

This skill provides advanced techniques to significantly reduce API costs when using Firecrawl for web scraping. It covers best practices such as setting crawl limits, utilizing batch scraping for known URLs, implementing local caching, and monitoring credit usage. Learn how to efficiently scrape large sites while managing budgets and minimizing unexpected billing spikes.

Firecrawl Data Processing and Management

firecrawl-data-handling

jeremylongshore/claude-code-plugins-plus-skills

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

Firecrawl Debug Bundle Generator

firecrawl-debug-bundle

jeremylongshore/claude-code-plugins-plus-skills

This utility collects comprehensive diagnostic data for troubleshooting Firecrawl issues. It automatically checks environment variables, verifies API connectivity, captures SDK versions, gathers recent error logs, and runs diagnostic scrapes. This is essential for generating complete, redacted support tickets when encountering persistent or hard-to-reproduce bugs.

Deploying Firecrawl Web Scraping Integrations

firecrawl-deploy-integration

jeremylongshore/claude-code-plugins-plus-skills

A comprehensive guide for deploying web scraping applications powered by Firecrawl to various production environments. Covers Vercel serverless functions, Google Cloud Run, and self-hosted Docker setups. Learn how to configure platform-specific secrets, implement API routes, and handle asynchronous crawl results via webhooks for scalable, reliable production service.

Language