Skill UI

Automated assistance for pyspark transformer work within data pipelines, activating when you request transformer patterns or best practices and covering ETL, data transformation, orchestration, and streaming scenarios.

Expert in designing and implementing advanced Retrieval-Augmented Generation (RAG) pipelines. This role focuses on optimizing the entire information retrieval process—from document chunking and embedding model selection to vector database architectures and hybrid search strategies. The goal is to bridge the gap between raw knowledge documents and accurate LLM understanding, drastically reducing hallucination and enhancing context relevance for enterprise applications.

Automates schema validation within data pipelines, providing guidance on best practices, workflow orchestration, and streaming ETL scenarios while producing production-ready code and configurations.

Provides Meta AI’s Segment Anything Model for zero-shot image segmentation via interactive prompts or automatic mask generation, enabling rapid dataset creation and integration in vision pipelines.

This skill provides expertise in building production-grade, scalable data systems and pipelines. It covers the full spectrum of data engineering, including designing robust batch ETL/ELT workflows, implementing real-time streaming architectures (Kafka/Flink), and utilizing modern tools like Airflow, Spark, and dbt. Essential for data architecture design, data lakehouse implementation, and data governance.

Automates sklearn pipeline builder tasks within ML training, activating on relevant triggers to guide data prep, model training, hyperparameter tuning, and experiment tracking with production-ready code and standards validation.

Automates Snowflake data warehouse workflows via the Composio MCP, letting you list databases, schemas, and tables, run SQL (SELECT, DDL, DML), and integrate those operations into cross-app pipelines for faster data ops.

Guides building, tuning, and validating production Apache Spark jobs, covering DataFrame/RDD choices, partitioning, broadcast joins, skew handling, caching, and cluster configuration for big data pipelines.

Automates spark job creation within data pipelines by guiding etl, transformation, orchestration, and streaming workflows; includes industry best practices, code templates, and validation to deliver production-ready jobs.

Optimize Apache Spark jobs with partitioning, caching, shuffle optimization, and memory tuning. Use when improving Spark performance, debugging slow jobs, or scaling data processing pipelines.

Automates Spark SQL optimizer tasks in data pipelines by guiding best practices, producing production-ready code and configurations, and validating ETL, transformation, orchestration, and streaming workflows.

Automated guidance for SQL transform helper tasks within data pipelines, covering ETL, data transformation, workflow orchestration, and streaming, while following best practices, generating production-ready code, and validating results.