Skill UI

A comprehensive guide to building production-ready Apache Airflow Directed Acyclic Graphs (DAGs). This skill covers best practices for operators, sensors, testing, deployment, and designing robust data pipelines. Use this when orchestrating complex workflows, managing ETL processes, or ensuring reliable batch job execution in a production environment.

Expert in designing, building, and optimizing scalable data pipelines, modern data warehouses, and real-time streaming architectures. Mastery covers the complete modern data stack, including Spark, dbt, Airflow, and cloud services (AWS, Azure, GCP). Focuses on ELT/ETL, data lakehouse implementation (Delta Lake, Iceberg), data governance, and ensuring high data quality and operational reliability.

Expert guidance on designing highly scalable, distributed NoSQL databases, specifically covering Apache Cassandra and Amazon DynamoDB. This skill emphasizes the 'Query-First' modeling approach, teaching best practices like partitioning, denormalization, single-table design, and avoiding common anti-patterns (e.g., hot partitions, joins) to ensure optimal read/write performance at massive scale.

Guides building, tuning, and validating production Apache Spark jobs, covering DataFrame/RDD choices, partitioning, broadcast joins, skew handling, caching, and cluster configuration for big data pipelines.

This guide provides advanced patterns and best practices for optimizing Apache Spark jobs. It covers critical areas like effective partitioning, memory tuning, minimizing data skew, and optimizing different types of joins (broadcast, bucket). Use this when troubleshooting slow, large-scale data processing workflows or needing to scale Spark pipelines efficiently and reliably.

Polars is a blazing fast in-memory DataFrame library based on Apache Arrow that replaces pandas for 1-100GB workloads, offering expression-based lazy execution, parallel operators, and rich I/O for ETL pipelines.

This tool provides a comprehensive framework for analyzing web server logs (Apache/Nginx) to detect various security threats. It uses advanced regex matching against OWASP attack signatures to identify SQL injection, Local File Inclusion, and Directory Traversal attempts, alongside behavioral analysis for brute-force attacks and web scanner fingerprints. The system also enriches data using GeoIP for source attribution, generating a prioritized report for SOC analysts.

Polars is a lightning-fast DataFrame library built for Python and Rust, leveraging Apache Arrow. It provides an expression-based API and lazy evaluation framework for highly efficient data manipulation. It is ideal for complex data pipelines, large-scale ETL tasks, and migrating from Pandas due to its superior performance, parallel execution capabilities, and optimized query planning.

A comprehensive guide to Test-Driven Development (TDD) for Quarkus 3.x services, focusing on achieving high coverage (80%+). It covers unit testing with Mockito, integration testing of REST endpoints, and advanced testing of event-driven architectures using Apache Camel and RabbitMQ.

This powerful tool compares identical tables across two SQL Server instances (e.g., Production vs. Staging). It efficiently detects missing rows, extra columns, schema drifts, and data mismatches. Leveraging Python and Apache Arrow for fast columnar data transfer, it provides a comprehensive reconciliation report, making it essential for data migration validation and ETL verification.

This guide details setting up advanced Command and Control (C2) redirectors, acting as protective intermediaries between implant beacons and the hidden team server. It covers using Nginx and Apache for request filtering based on Malleable C2 profiles, layering Let's Encrypt TLS, and implementing advanced OPSEC controls like domain fronting, geo/UA filtering, and decoy diversion. Essential for red-teaming operations requiring high resilience against blue-team detection and takedown.

A comprehensive guide to architectural patterns for building cloud-native, event-driven microservices using Quarkus 3.x. Covers RESTful API design with JAX-RS, implementing event messaging with Apache Camel and RabbitMQ, structured data access using Hibernate Panache, managing transactions, and advanced logging context handling (MDC). Ideal for developers building robust, scalable, and maintainable enterprise systems.