Skill UI

This comprehensive guide identifies common anti-patterns and integration mistakes when using Firecrawl for web scraping and content extraction. It serves as a vital code review checklist, helping developers avoid pitfalls such as unbounded crawling, unspecified output formats, or aggressive API polling, ensuring robust and efficient data pipelines.

This skill provides advanced methods for load testing and scaling web scraping pipelines using Firecrawl. It manages rate limits and optimizes throughput by implementing concurrency control, batch processing, and queue-based request throttling. Essential for capacity planning, stress-testing scraping performance, or maximizing data ingestion efficiency across various service plans.

A comprehensive guide for migrating web scraping pipelines from tools like Puppeteer, Playwright, and Cheerio to Firecrawl. This skill demonstrates how to replace complex, browser-intensive code with simple API calls, handling single-page scraping, structured data extraction using LLMs, and large-scale site crawling, while eliminating the need for manual browser management and anti-bot workarounds.

Provides comprehensive observability patterns for Firecrawl web scraping operations. It covers key metrics such as scrape success rates, credit consumption tracking, content quality assessment, and latency monitoring. Use this skill for implementing robust monitoring dashboards, setting up proactive failure alerts, and optimizing the reliability of web crawling workflows.

This skill provides a comprehensive guide to systematic dark web monitoring, involving scanning underground forums, paste sites, and marketplaces for organizational threats. It covers setting up Tor-based crawling, automating credential leak detection (e.g., HIBP integration), and analyzing threat intelligence (CTI) using Python. Essential for proactive incident response and security auditing.

A comprehensive guide to using the Apify Client SDK. Learn the fundamental pattern of executing remote web scraping Actors (like content crawling or general scraping), managing asynchronous run states, and retrieving structured data from resulting datasets and key-value stores. Essential for integrating powerful web scraping capabilities into applications.

A comprehensive guide for safely upgrading and migrating between Apify core packages, including apify-client, apify, and crawlee. This skill is essential for handling major breaking changes, particularly the transition from Apify SDK v2 to v3, which separated crawling functionality into the dedicated 'crawlee' package. It provides detailed steps for version checking, dependency management, and code refactoring.

A lightweight utility designed for fetching content from a single URL via HTTP or HTTPS. It supports various methods (GET, POST, PUT, DELETE) and allows configuration of timeouts and maximum response body sizes. This tool is ideal for quick, simple data extraction steps within a workflow DAG, offering significant performance gains over complex LLM agent loops. Note that complex tasks like JS rendering or multi-page crawling require dedicated sub-agents.

This skill provides comprehensive, non-destructive testing for deployed Power Pages sites. It navigates the site, performs page crawling, verifies HTTP status codes, captures network API requests (Web API/OData), and detects console errors. It supports testing core site load, authentication flow, and data integrity.