Skill UI

This SDK provides comprehensive management plane capabilities for provisioning and controlling Microsoft Fabric compute capacities via Azure Resource Manager (ARM). Use it to perform core lifecycle operations such as creating, updating (including scaling), suspending, resuming, and deleting Fabric capacities. Note: This SDK manages the underlying compute resource, not the data plane elements like workspaces or lakehouses.

This SDK provides comprehensive management plane capabilities for provisioning and controlling Microsoft Fabric compute capacities via Azure Resource Manager (ARM). Use it to perform core lifecycle operations such as creating, updating (including scaling), suspending, resuming, and deleting Fabric capacities. Note: This SDK manages the underlying compute resource, not the data plane elements like workspaces or lakehouses.

Expert in designing, building, and optimizing scalable data pipelines, modern data warehouses, and real-time streaming architectures. Mastery covers the complete modern data stack, including Spark, dbt, Airflow, and cloud services (AWS, Azure, GCP). Focuses on ELT/ETL, data lakehouse implementation (Delta Lake, Iceberg), data governance, and ensuring high data quality and operational reliability.

This comprehensive guide outlines the end-to-end process of building production-grade ETL data pipelines using Databricks and Delta Lake. It implements the robust Medallion Architecture (Bronze, Silver, Gold) for scalable data management. Techniques covered include Auto Loader for incremental streaming ingestion, MERGE INTO for upsert semantics, data cleansing, and advanced aggregation into analytics-ready layers. Ideal for establishing reliable data pipelines in a data lakehouse environment.

This guide outlines a production-ready lakehouse architecture using Unity Catalog, Delta Lake, and the Medallion pattern. It provides best practices for organizing Databricks workspaces, implementing three-level governance, defining compute strategies, and adopting CI/CD best practices with Asset Bundles for robust team collaboration.

This comprehensive guide details the expertise needed for a Senior Data Engineer. It covers building, optimizing, and governing scalable data pipelines and robust ETL/ELT systems using modern tools like Spark, Kafka, Airflow, and dbt. Includes decision frameworks for architecting data solutions (Batch vs Streaming, Lakehouse vs Warehouse) and implementing advanced data modeling patterns.

LaminDB is a Python-driven open-source framework for managing biological datasets, ensuring traceable, reproducible, FAIR data through lineage tracking, schema validation, ontology annotation, workflow integration, and lakehouse-style querying.

Explains Fabric Lakehouse storage, data organization, security, shortcuts, and optimization practices so you can design, build, and monitor Lakehouse solutions leveraging Microsoft Fabric best practices.

LaminDB is an open-source lineage-native lakehouse designed for biological datasets and models. It enables queryable, traceable, and validated data management across various storage formats like S3 and Postgres. Users can track computational workflows, curate data with ontologies via Bionty, and ensure FAIR compliance. Ideal for managing scRNA-seq, spatial transcriptomics, and integrating ML pipelines while maintaining reproducibility and collaboration.

This comprehensive guide provides blueprints for implementing optimal Snowflake data architectures. It covers diverse patterns, including traditional data warehouses, Lakehouse designs using Iceberg tables, decentralized Data Mesh structures, and advanced data sharing and Snowpark native patterns. Use this when designing enterprise data platforms across varying scales.

Decision-driven Chief Data Officer interrogation of any plan touching training data, data architecture, data productization, or data team hiring. Use when validating training-data rights before model work, choosing warehouse vs lakehouse vs mesh, valuing data assets for productization or M&A, or vetting a data team hire.

Provides high-level strategic guidance for founders and Chief Data Officers (CDOs) in startups. This skill covers critical decisions such as AI training data rights and consent provenance, selecting optimal data architectures (warehouse, lakehouse, data mesh), valuing customer data as a strategic asset for M&A or fundraising, and planning data team organizational evolution. It focuses exclusively on strategic decision-making, not tactical engineering.