Skill UI

Guides building, tuning, and validating production Apache Spark jobs, covering DataFrame/RDD choices, partitioning, broadcast joins, skew handling, caching, and cluster configuration for big data pipelines.

Automates spark job creation within data pipelines by guiding etl, transformation, orchestration, and streaming workflows; includes industry best practices, code templates, and validation to deliver production-ready jobs.

This guide provides advanced patterns and best practices for optimizing Apache Spark jobs. It covers critical areas like effective partitioning, memory tuning, minimizing data skew, and optimizing different types of joins (broadcast, bucket). Use this when troubleshooting slow, large-scale data processing workflows or needing to scale Spark pipelines efficiently and reliably.

Automates Spark SQL optimizer tasks in data pipelines by guiding best practices, producing production-ready code and configurations, and validating ETL, transformation, orchestration, and streaming workflows.

Generates a single formula-driven Excel dashboard from CSV/tabular data, complete with KPI cards, charts, sparklines, and conditional formatting so analysts can instantly review metrics and share an executive-ready .xlsx output.

Comprehensive guide for setting up Continuous Integration/Continuous Deployment (CI/CD) pipelines for Palantir Foundry integrations using GitHub Actions. This workflow covers running PySpark unit tests, performing SDK integration tests with mocked APIs, and implementing custom linting rules to enforce Foundry-specific best practices. Use this when ensuring code quality and validating complex business logic changes in an automated build process.

This guide provides a comprehensive workflow for building robust ETL data pipelines within Palantir Foundry. It teaches how to utilize Python transforms, including the @transform_df decorator, for dataset-to-dataset processing. Key topics covered include multi-input joins, implementing incremental updates for efficiency, low-level file I/O, and configuring Spark resources for heavy computations. Essential for data engineers working with complex data modeling in Foundry.

This guide provides comprehensive technical instructions for optimizing Palantir Foundry costs. Learn to use incremental transforms to drastically reduce compute rebuilds, right-size Spark profiles based on data volume, and migrate from expensive API polling to efficient, event-driven webhooks, ensuring both cost efficiency and system stability.

A structured guide for setting up a robust local development environment for Palantir Foundry data pipelines. This workflow enables users to run Python transforms and unit tests using PySpark against sample data, and mock the Foundry API client, ensuring fast, isolated iteration cycles before deploying code to the official Foundry environment. Ideal for data engineers and developers building complex ETL logic.

This guide provides advanced techniques for optimizing performance within Palantir Foundry. It covers efficient pagination, implementing client-side caching, using batch object retrieval (solving the N+1 problem), and tuning Spark transforms with memory configuration and incremental updates. Use this when experiencing slow API responses or inefficient data pipeline builds in Foundry.

A comprehensive tool for the full lifecycle management of applications within the Lark ecosystem (Feishu/Miaoda). It supports three development paths: local full-stack development, HTML static hosting, and cloud-based AI session generation. Use this when creating, updating, deploying, or publishing any complex system, tool, or platform application.

This skill enables the generation of rich, inline SVG graphics directly within Power BI reports. By writing DAX measures that return SVG markup strings and setting the data category to ImageUrl, users can visualize complex charts (like sparklines, gauges, and progress bars) within standard visuals such as Tables, Cards, and Matrices. It guides users through the entire process, including recommended best practices like using existing UDF libraries and adhering to strict VAR-based DAX structures.