Skill UI

A comprehensive guide to building production-ready Apache Airflow Directed Acyclic Graphs (DAGs). This skill covers best practices for operators, sensors, testing, deployment, and designing robust data pipelines. Use this when orchestrating complex workflows, managing ETL processes, or ensuring reliable batch job execution in a production environment.

This skill guides the construction of robust, production-grade backtesting systems for financial trading strategies. It helps users systematically validate performance, avoid common biases (like look-ahead bias), and implement advanced techniques such as walk-forward analysis, ensuring reliable performance estimates before live deployment.

This skill details how to set up automated CI/CD pipelines for Databricks data pipelines. It leverages Declarative Automation Bundles (DABs) and GitHub Actions to manage the entire deployment lifecycle. Key functionalities include running unit tests on pull requests, validating and deploying to staging environments, and promoting validated code to production upon merge, ensuring reliable and automated data infrastructure.

Use Declarative Automation Bundles (DABs) to manage and deploy Databricks assets, including jobs, DLT pipelines, and ML models, using an Infrastructure-as-Code approach. This skill allows defining resources and deployment targets (dev, staging, prod) in a single YAML file, enabling robust CI/CD practices and controlled promotion of changes across environments.

This comprehensive checklist guides the deployment of Databricks data pipelines and jobs to a production environment. It covers critical stages including pre-deployment security hardening (Service Principals, Unity Catalog), validating infrastructure (cluster types, autoscaling), ensuring code quality (unit/integration tests, Delta Lake best practices), executing controlled deployment, and performing post-deployment health checks and monitoring setup. Ideal for CI/CD pipelines and go-live procedures.

Provides detailed evaluation of machine learning models using plugin metrics to assess accuracy, precision, recall, F1, and other key indicators, helping users compare models and validate performance before deployment.

Automates guidance for setting up and validating feature store connectors within ML deployment pipelines, covering best practices, code generation, and production monitoring for inference.

Configure automated Continuous Integration and Continuous Deployment (CI/CD) pipelines for Juicebox AI data analysis integrations. This setup utilizes GitHub Actions to run comprehensive unit tests (with mocked datasets) and integration tests (connecting to live APIs) on every PR and merge to main. It ensures that data upload logic, analysis execution, and result parsing workflows are robust and reliable before deployment.

Provides automated guidance for managing model versions in ML deployment, including serving, monitoring, and production optimization whenever someone requests the model versioning manager.

Guides TensorFlow Serving setup with step-by-step ML deployment orchestration, enforcing best practices, ready-to-run configurations, and output validation for production inference services.

This comprehensive skill guides the deployment and utilization of MISP, a powerful open-source threat intelligence platform. It covers configuring various threat feeds (TAXII, CSV, etc.) and using the PyMISP Python API to programmatically automate the collection, storage, and correlation of Indicators of Compromise (IOCs) and threat events from diverse community and commercial sources.

This comprehensive tool manages the complete lifecycle of ML experiments, providing robust deployment capabilities across diverse environments. It supports running training jobs on local GPUs, dedicated remote servers, Vast.ai, and serverless platforms like Modal. The workflow handles environment detection, code synchronization (rsync/git), dependency pinning, and integrates advanced logging tools (e.g., W&B), ensuring reliable, scalable, and reproducible ML training runs.