Skill UI

This skill details how to set up automated CI/CD pipelines for Databricks data pipelines. It leverages Declarative Automation Bundles (DABs) and GitHub Actions to manage the entire deployment lifecycle. Key functionalities include running unit tests on pull requests, validating and deploying to staging environments, and promoting validated code to production upon merge, ensuring reliable and automated data infrastructure.

This comprehensive checklist guides the deployment of Databricks data pipelines and jobs to a production environment. It covers critical stages including pre-deployment security hardening (Service Principals, Unity Catalog), validating infrastructure (cluster types, autoscaling), ensuring code quality (unit/integration tests, Delta Lake best practices), executing controlled deployment, and performing post-deployment health checks and monitoring setup. Ideal for CI/CD pipelines and go-live procedures.

Configure automated Continuous Integration and Continuous Deployment (CI/CD) pipelines for Juicebox AI data analysis integrations. This setup utilizes GitHub Actions to run comprehensive unit tests (with mocked datasets) and integration tests (connecting to live APIs) on every PR and merge to main. It ensures that data upload logic, analysis execution, and result parsing workflows are robust and reliable before deployment.

NetworkX is a powerful Python library used for creating, manipulating, and analyzing complex networks and graphs. It supports graph structures (directed/undirected, multi-edges) and provides extensive algorithms for tasks such as finding shortest paths, calculating centrality measures (e.g., PageRank), detecting communities, and simulating various network types (e.g., scale-free, small-world). Ideal for social science, biology, and transportation network modeling.

This skill guides the setup of CI/CD pipelines for Hex data analytics projects using GitHub Actions. It enables running automated unit tests on pull requests and triggering live project runs on the main branch merge. Use this when establishing automated testing workflows, validating data transformation logic, or integrating Hex outputs into a continuous delivery process.

A structured guide for setting up a robust local development environment for Palantir Foundry data pipelines. This workflow enables users to run Python transforms and unit tests using PySpark against sample data, and mock the Foundry API client, ensuring fast, isolated iteration cycles before deploying code to the official Foundry environment. Ideal for data engineers and developers building complex ETL logic.