Skill UI

A comprehensive guide detailing optimization techniques for Databricks environments. It covers optimal cluster sizing for various workloads (ETL, ML, Streaming), advanced Spark configurations (AQE, memory management), and deep data optimization strategies for Delta Lake, including Z-Ordering and Liquid Clustering. It also provides methods for query plan analysis and join optimization.

FAISS (Facebook AI Similarity Search) is a library designed for efficient similarity search and clustering of dense vectors. It supports indexing and searching on billions of vectors at scale, featuring GPU acceleration and multiple index types (Flat, IVF, HNSW, PQ) to balance speed and accuracy. It is essential for building high-performance, large-scale applications requiring fast k-NN retrieval or pure vector similarity matching, especially in fields like RAG and recommendation systems.

Automates clustering workflows on supplied datasets with K-means, DBSCAN, or hierarchical algorithms, handling preprocessing, validation, metrics, and visualizations to expose groupings and anomalies.

Sentence Transformers provides 5000+ pre-trained models that generate sentence, text, and image embeddings for RAG, semantic search, clustering, and multilingual production workloads without relying on external APIs.

Aeon is a scikit-learn compatible toolkit for time series machine learning, covering classification, regression, clustering, forecasting, anomaly detection, segmentation, similarity search, and feature extraction on temporal data.

Datamol wraps RDKit with Pythonic APIs tailored for drug discovery, offering SMILES/SELFIES/InChI parsing, standardization, descriptor and fingerprint calculation, clustering, 3D conformer workflows, and parallel/cloud-aware batch I/O while returning native rdkit.Chem.Mol objects.

Geniml builds machine learning workflows around BED and scATAC datasets to learn embeddings for regions, metadata, and cells, enabling similarity searches, clustering, consensus peaks, and downstream analysis of genomic intervals.

Scanpy is a scalable Python toolkit for single-cell RNA-seq analyses, covering QC, normalization, dimensionality reduction, clustering, marker discovery, annotation, and publication-ready visualization across h5ad/10X/CSV datasets.

Use UMAP for fast nonlinear manifold reduction to produce embeddings that keep local and global structure, making it ideal for visualization, clustering preprocessing, supervised/parametric workflows, or high-dimensional feature engineering.

Batch-categorize uncategorized transactions by clustering similar descriptions, researching unknown merchants, and creating rules or annotations so future payments auto-classify with minimal manual review.

This tool applies advanced role mining techniques (including clustering and formal concept analysis) to analyze existing user-permission assignments. It discovers optimal, minimal role sets for Role-Based Access Control (RBAC) systems, helping organizations mitigate 'role explosion' and enforce the principle of least privilege. Ideal for identity governance and access control architecture redesign.

Scikit-learn compatible Python toolkit for time series machine learning: classification, regression, clustering, forecasting, anomaly detection, segmentation, similarity search and distance metrics for univariate and multivariate temporal data.