Skill UI

This technical guide provides patterns for handling rate limits, throttling, and concurrency issues when scraping data using Bright Data services. It demonstrates implementing concurrency control using queuing, applying exponential backoff for transient errors (like 429 or 503), and managing specific API trigger rate limits (e.g., 20/min, 60/hr). Essential for building robust, large-scale web scrapers.

A comprehensive, production-ready reference architecture for building robust data scraping systems using Bright Data's full suite of tools (Web Unlocker, Scraping Browser, SERP API). This template provides best practices for project layout, managing proxies, designing data pipelines, and integrating scheduling/storage components in a Node.js/TypeScript environment.

This guide provides production-ready patterns for integrating with Bright Data proxy services. It covers critical aspects like setting up a proxy client, implementing robust retry logic for transient errors (e.g., 502, 503), managing sticky sessions for consistent IPs, and parsing complex HTML responses. Essential for building reliable, high-scale web scrapers using TypeScript and Python.

This comprehensive guide provides developers with step-by-step instructions for migrating between different Bright Data services and API versions. It covers complex transitions, such as upgrading from raw proxies to Web Unlocker, moving from Puppeteer to Scraping Browser, and updating datasets API endpoints (v2 to v3). Includes prerequisites, code examples, and rollback procedures for seamless integration.

This comprehensive guide details how to implement robust webhook endpoints to receive, validate, and process data deliveries from Bright Data's Web Scraper and Datasets APIs. Learn best practices for handling large JSON/NDJSON payloads, verifying authorization headers, processing status notifications, and ensuring data deduplication in automated scraping pipelines.

A browser-based skill for search, web scraping, login-required actions, and dynamic page interaction. Use it to access real websites, extract content, and operate pages in a live Chrome environment.

Full-site crawling, scraping, and site mapping via Firecrawl MCP. Use for SEO audits, broken link detection, SPA content extraction, and site structure discovery. Automates web crawling tasks with JavaScript rendering and credit-efficient mapping.

This skill guides the design of robust, production-grade data extraction pipelines, moving beyond brittle one-off scripts. It supports three modes: API-driven web scraping (Firecrawl), traditional local Python extraction (PDF, CSV), and hybrid workflows. Key features include intelligent routing, mandatory data validation, secure key management, and adherence to scraping best practices.

HasData is a comprehensive cloud platform designed for extracting structured public web data. It supports three core execution modes: Web Scraping API for arbitrary URLs (including JS rendering), pre-parsed Scraper APIs for known platforms (like Google or Amazon), and asynchronous Scraper Jobs for bulk or recursive crawling. Ideal for market research, e-commerce intelligence, job data collection, and local business lead generation.

A powerful CLI tool designed for deep web interaction and data scraping. bb-browser simulates genuine user behavior within a real browser environment, allowing access to both public and private enterprise systems, even requiring active user login. Key features include a Site System to turn complex websites into simple command-line APIs, advanced network request handling (with session state), form automation, and element interaction, making it ideal for enterprise data gathering and workflow automation.

A powerful command-line utility that transforms virtually any website into a structured CLI command. By utilizing OpenClaw's browser, it allows users to scrape and extract structured data (such as JSON) from over 36 platforms—including social media, finance, and developer communities—directly from the terminal. This tool eliminates the need for complex scraping code or browser extensions, making web data accessible via simple one-liners.

A powerful skill for generating scripts using Puppeteer, enabling robust browser automation, data scraping, and PDF creation. Use this when you need to simulate user interactions, perform complex web testing, or extract content from dynamic websites, including handling wait strategies, network interception, and generating media files.