Skill UI

A comprehensive guide to implementing best practices for security within Databricks. Learn how to manage secrets using scoped storage, enforce the principle of least privilege via Unity Catalog grants, audit API tokens, and apply advanced data protection techniques like column-level masking and row-level security for enhanced data governance and compliance.

TileDB-VCF lets you ingest, store, query, and export large genomic VCF/BCF datasets using TileDB’s sparse arrays, supporting incremental sample growth, parallel regions queries, and cloud-local interoperability for population genomics workflows.

Zarr is a Python toolkit for chunked N-D array storage with compression, sharding, and native integrations with NumPy/Dask/Xarray, ideal for parallel I/O and scalable cloud pipelines.

Detects abnormal access patterns across major cloud storage platforms, including AWS S3, GCS, and Azure Blob Storage. By analyzing cloud audit logs (CloudTrail, etc.), this tool identifies critical security threats such as after-hours bulk data downloads, sudden spikes in API calls (GetObject spikes), access from new geographic IPs, and potential data exfiltration. It utilizes statistical baselines and time-series anomaly detection for proactive threat hunting and security incident investigation.

This framework manages the ingestion, organization, synchronization, and search of knowledge across multiple storage layers, including local files, persistent memory, vector stores, and Git repositories. It provides a structured approach for ensuring knowledge durability, deduplication, and maintaining a 'single source of truth' across complex development and operational contexts.

A comprehensive expert prompt for evaluating Power BI data models. It guides the assessment of model architecture (Star Schema, relationships), storage optimization (Import, DirectQuery, Dual), data quality, performance scalability, and governance best practices. Use this when reviewing an existing BI model to ensure it is robust, maintainable, and highly performant.

Zarr is a robust Python library designed for storing massive N-dimensional arrays. It supports efficient chunking, compression, and parallel I/O, making it ideal for large-scale scientific computing pipelines. Seamlessly integrates with NumPy, Dask, and Xarray, and supports cloud storage backends like S3 and GCS via fsspec, ensuring high performance in cloud-native environments.

Provides comprehensive programmatic control over Apify's core data infrastructure, including Datasets, Key-Value Stores, and Request Queues. This skill allows users to perform CRUD operations on various storage types, manage data persistence, and orchestrate complex, multi-stage data pipelines (e.g., scrape -> transform -> export). Ideal for building robust, end-to-end web scraping and data processing automation workflows.

A comprehensive guide to reducing expenses on ClickHouse Cloud. Learn advanced techniques for storage optimization (compression, TTL), compute resource tuning (scaling, thread limits), and query efficiency improvements. Use this when managing data budgets or analyzing high ClickHouse billing charges.

This guide details how to set up and execute large-scale, distributed GPU training jobs on the CoreWeave platform. It covers both single-node multi-GPU setups and multi-node training utilizing PyTorch DDP and Kubernetes resources (A100/H100). This workflow is ideal for fine-tuning large language models (LLMs) or running complex deep learning models that require high-performance, shared-storage compute clusters.

A comprehensive guide for executing core Snowflake data loading workflows. This covers setting up stages (S3, GCS, Azure), defining file formats (CSV, JSON, Parquet), and using COPY INTO for bulk loading. It also details advanced continuous ingestion using Snowpipe, enabling real-time data synchronization from cloud storage into Snowflake tables.

A comprehensive guide detailing the process of migrating data warehouses to Snowflake from major sources like Amazon Redshift, Google BigQuery, or on-premise databases. This resource covers critical steps including schema conversion (handling dialect differences like STRUCT/ARRAY to VARIANT), robust data transfer methods (S3/Cloud Storage, Python/Pandas), data validation techniques (checksums, row counts), and essential SQL syntax translation patterns.