Skill UI

This guide details how to configure and implement Apify webhooks for robust, event-driven data pipelines. Learn to receive real-time notifications when Actor runs start, succeed, fail, or time out. By supporting both persistent (all runs) and ad-hoc (single run) webhooks, this pattern is ideal for building highly reliable and scalable production scraping integrations.

This workflow provides a complete, end-to-end solution for building and deploying robust web scrapers using Crawlee on the Apify platform. It guides users through every stage: defining input schemas, setting up the Actor with a sophisticated router pattern, extracting structured data (e.g., product details), saving results to datasets, and managing the full lifecycle from local testing to cloud deployment via apify-cli.

A comprehensive guide to using the Apify Client SDK. Learn the fundamental pattern of executing remote web scraping Actors (like content crawling or general scraping), managing asynchronous run states, and retrieving structured data from resulting datasets and key-value stores. Essential for integrating powerful web scraping capabilities into applications.

This guide details how to set up and test Apify Actors locally using the Apify CLI. It emulates the full platform environment (including datasets, key-value stores, and request queues), allowing developers to run scrapers, test data handling, and debug the entire development lifecycle before deploying to the cloud. Perfect for rapid iteration and rigorous local testing.

This comprehensive guide details how to optimize Apify Actors for maximum efficiency, cost-effectiveness, and reliability. Learn to select the appropriate crawler (e.g., Cheerio, Playwright), tune concurrency limits, manage memory consumption, and implement robust proxy rotation strategies for diverse web scraping workloads, ensuring your scraping tasks are fast and stable.

A comprehensive guide on managing rate limits when interacting with the Apify Platform API. It details best practices for building robust, high-throughput automation workflows, covering built-in retries, request queuing using p-queue, batching large payloads, and staggering concurrent operations to prevent 429 errors and maintain system stability.

A comprehensive guide to building robust, production-ready applications using the Apify SDK and apify-client in TypeScript. Covers advanced topics like standardized client singleton patterns, selecting the right Crawlee crawler (Cheerio, Playwright, Puppeteer), implementing robust Actor lifecycles, and managing dataset/key-value store operations.

A comprehensive guide for safely upgrading and migrating between Apify core packages, including apify-client, apify, and crawlee. This skill is essential for handling major breaking changes, particularly the transition from Apify SDK v2 to v3, which separated crawling functionality into the dedicated 'crawlee' package. It provides detailed steps for version checking, dependency management, and code refactoring.

This comprehensive skill facilitates deep integration with X (Twitter) data, supporting advanced scraping, user lookup, and data extraction. It provides multiple integration patterns for developers: dedicated SDKs (Python, TypeScript, Go, etc.), REST endpoints, hosted Apify Actors, and agent automation via MCP. Ideal for building robust data pipelines, monitoring systems, and AI-powered applications requiring real-time X data.