Download

Skill UI

Browse and discover 15857+ curated skills

All Development Artificial Intelligence Design & Creative Product & Business Data Science Marketing Soft Skills Productivity Engineering Languages

Search Data Processing , found 51 results

Default Newest Most Downloaded

Data Pipeline Policy Guardrails

clay-policy-guardrails

jeremylongshore/claude-code-plugins-plus-skills

A comprehensive system for implementing policy guardrails in data enrichment pipelines. It enforces critical controls including credit spending limits, blocks the enrichment of sensitive PII (e.g., SSN, bank details), and validates input data quality. Essential for maintaining compliance (GDPR/CCPA) and managing costs when processing large volumes of personal and business data.

Data Transformation Patterns Using dbt

dbt-transformation-patterns

sickn33/antigravity-awesome-skills

A guide to production-ready patterns for dbt (data build tool). It covers essential practices for robust data pipeline development, including model organization (staging, intermediate, marts), implementing advanced testing strategies, ensuring data quality, and optimizing incremental processing for large datasets.

Firecrawl Data Processing and Management

firecrawl-data-handling

jeremylongshore/claude-code-plugins-plus-skills

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

Firecrawl Content Ingestion Pipeline Architecture

firecrawl-reference-architecture

jeremylongshore/claude-code-plugins-plus-skills

This comprehensive reference architecture details a multi-stage pipeline for web scraping and content ingestion using Firecrawl. It seamlessly integrates single-page scraping, site-wide crawling, URL mapping, and structured data extraction. The subsequent processing layer handles cleaning, deduplication, and chunking, making this blueprint ideal for designing robust knowledge bases, RAG systems, and complex AI data pipelines.

Firecrawl Webhook Event Handling Guide

firecrawl-webhooks-events

jeremylongshore/claude-code-plugins-plus-skills

Master the implementation of Firecrawl webhooks for robust, real-time handling of asynchronous crawling and batch scraping jobs. Instead of inefficiently polling the crawl status, configure webhook endpoints to receive instant POST events (e.g., `crawl.page`, `crawl.completed`). This guide covers event types, signature verification using HMAC-SHA256, and processing streamed data for building advanced data pipelines.

Deploy Juicebox AI Integration via Docker

juicebox-deploy-integration

jeremylongshore/claude-code-plugins-plus-skills

This skill provides a comprehensive guide and implementation for deploying a production-ready, containerized integration service using Docker. It connects to the Juicebox API to manage datasets, execute advanced AI-powered analyses, and retrieve structured business insights. The service includes health check mechanisms and supports zero-downtime rolling updates, ensuring reliable real-time data processing and analysis.

Kafka Stream Processor

kafka-stream-processor

jeremylongshore/claude-code-plugins-plus-skills

Automates Kafka stream processor workflows within data pipelines, offering step-by-step guidance, best practices, production-ready code and configurations, and validation to support ETL, data transformation, workflow orchestration, and streaming data processing.

Automated Data Preprocessing

preprocessing-data-with-automated-pipelines

jeremylongshore/claude-code-plugins-plus-skills

This skill builds and executes automated pipelines to clean, validate, and transform data for machine learning, covering ETL workflows, error handling, and quality reporting so Claude can deliver analysis-ready datasets.

Ray Data Processing

Orchestra-Research/AI-Research-SKILLs

Ray Data offers scalable distributed data processing for ML workloads, streaming across CPU/GPU clusters and integrating with PyTorch, TensorFlow, and Ray Train. Use it for batch inference, ETL, and multi-modal preprocessing from a laptop up to hundreds of nodes.

Clustering Analysis Assistant

running-clustering-algorithms

jeremylongshore/claude-code-plugins-plus-skills

Automates clustering workflows on supplied datasets with K-means, DBSCAN, or hierarchical algorithms, handling preprocessing, validation, metrics, and visualizations to expose groupings and anomalies.

Spark Job Optimization and Tuning

spark-optimization

sickn33/antigravity-awesome-skills

This guide provides advanced patterns and best practices for optimizing Apache Spark jobs. It covers critical areas like effective partitioning, memory tuning, minimizing data skew, and optimizing different types of joins (broadcast, bucket). Use this when troubleshooting slow, large-scale data processing workflows or needing to scale Spark pipelines efficiently and reliably.

FlowIO FCS Handler

K-Dense-AI/claude-scientific-skills

FlowIO reads and writes Flow Cytometry Standard (FCS) files, exposing metadata, channels, and NumPy event arrays while allowing CSV/DataFrame conversion, making it ideal for cytometry pipelines and backend preprocessing services.

Language