Download

Skill UI

Browse and discover 19382+ curated skills

All Development Artificial Intelligence Design & Creative Product & Business Data Science Marketing Soft Skills Productivity Engineering Languages

Search Scraping , found 27 results

Default Newest Most Downloaded

Website Scraping and Crawling Pipeline

firecrawl-core-workflow-a

jeremylongshore/claude-code-plugins-plus-skills

This workflow provides the primary mechanism for converting complex, live websites into clean, LLM-ready markdown content. It supports single-page scraping using `scrapeUrl`, deep multi-page site crawling with `crawlUrl`, and scalable async crawling for large datasets. Ideal for building robust content ingestion pipelines and reliably extracting structured information from the web.

Web Scraping, Crawling, and Data Extraction API

firecrawl-hello-world

jeremylongshore/claude-code-plugins-plus-skills

Firecrawl provides a comprehensive API for web data harvesting. It enables users to perform single-page scraping, multi-page crawling, site mapping (URL discovery), and advanced structured data extraction using Large Language Models (LLMs). Ideal for integrating data pipelines, building web monitoring tools, or conducting market research.

Firecrawl Content Ingestion Pipeline Architecture

firecrawl-reference-architecture

jeremylongshore/claude-code-plugins-plus-skills

This comprehensive reference architecture details a multi-stage pipeline for web scraping and content ingestion using Firecrawl. It seamlessly integrates single-page scraping, site-wide crawling, URL mapping, and structured data extraction. The subsequent processing layer handles cleaning, deduplication, and chunking, making this blueprint ideal for designing robust knowledge bases, RAG systems, and complex AI data pipelines.

Firecrawl Webhook Event Handling Guide

firecrawl-webhooks-events

jeremylongshore/claude-code-plugins-plus-skills

Master the implementation of Firecrawl webhooks for robust, real-time handling of asynchronous crawling and batch scraping jobs. Instead of inefficiently polling the crawl status, configure webhook endpoints to receive instant POST events (e.g., `crawl.page`, `crawl.completed`). This guide covers event types, signature verification using HMAC-SHA256, and processing streamed data for building advanced data pipelines.

Web Search, Crawling, and Content Extraction

sickn33/antigravity-awesome-skills

This skill provides comprehensive capabilities for web searching, content extraction, and web crawling using the Tavily API. It is ideal for tasks requiring up-to-date information retrieval, deep content scraping from specific URLs, or comprehensive website crawling for research purposes, making it a powerful research augmentation tool.

X/Twitter Data Scraping and Automation Suite

x-twitter-scraper

sickn33/antigravity-awesome-skills

A comprehensive toolkit designed for advanced data scraping and automation workflows on X/Twitter. Capabilities include deep tweet search, user profile analysis, bulk follower/media export, automated posting (replies, DMs), real-time account monitoring via webhooks, and integration with multiple AI agent platforms (Hermes, OpenClaw). Ideal for market research, social listening, and content automation.

Brazilian Auctioneers Data

junta-leiloeiros

sickn33/antigravity-awesome-skills

Aggregates official auctioneer information from all 27 Brazilian Commercial Boards into a SQLite store, exposes a FastAPI REST interface, and enables CSV/JSON exports plus customizable scraping per state.

Automated AI Data Scraping Agent

data-scraper-agent

affaan-m/everything-claude-code

Build a production-ready, AI-powered data collection agent for any public source (job boards, prices, news, etc.). This agent automatically scrapes data, enriches it using free LLMs (like Gemini Flash), and stores the results in databases such as Notion, Sheets, or Supabase. It runs on a free schedule using GitHub Actions and learns from user feedback, making it ideal for continuous data monitoring and tracking.

Structured X/Twitter Post Reader

sickn33/antigravity-awesome-skills

ADHX provides a dedicated API to fetch X/Twitter posts as clean, structured JSON. It extracts comprehensive data, including author profiles, engagement metrics (likes, retweets), and full article content for long-form posts. This method is vastly superior to traditional scraping techniques, making it ideal for LLM analysis, summarization, and structured data workflows.

Apify Core Storage and Workflow Management

apify-core-workflow-b

jeremylongshore/claude-code-plugins-plus-skills

Provides comprehensive programmatic control over Apify's core data infrastructure, including Datasets, Key-Value Stores, and Request Queues. This skill allows users to perform CRUD operations on various storage types, manage data persistence, and orchestrate complex, multi-stage data pipelines (e.g., scrape -> transform -> export). Ideal for building robust, end-to-end web scraping and data processing automation workflows.

Scraping Dynamic Data With Bright Data

brightdata-core-workflow-a

jeremylongshore/claude-code-plugins-plus-skills

Utilize Bright Data's Scraping Browser with Playwright or Puppeteer to reliably scrape structured data from JavaScript-rendered pages, Single Page Applications (SPAs), and anti-bot-protected sites. This tool manages proxy routing, CAPTCHA solving, and fingerprint management, ensuring robust data collection from complex web environments.

Optimize Bright Data Scraping Costs

brightdata-cost-tuning

jeremylongshore/claude-code-plugins-plus-skills

This skill provides comprehensive methods and code examples for optimizing spending on Bright Data services. It covers product selection (Proxy vs. API), implementing caching strategies, bulk usage monitoring, and setting budget alerts, ensuring cost efficiency in large-scale web scraping projects.

Language