Skill UI

This workflow provides the primary mechanism for converting complex, live websites into clean, LLM-ready markdown content. It supports single-page scraping using `scrapeUrl`, deep multi-page site crawling with `crawlUrl`, and scalable async crawling for large datasets. Ideal for building robust content ingestion pipelines and reliably extracting structured information from the web.

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

Firecrawl provides a comprehensive API for web data harvesting. It enables users to perform single-page scraping, multi-page crawling, site mapping (URL discovery), and advanced structured data extraction using Large Language Models (LLMs). Ideal for integrating data pipelines, building web monitoring tools, or conducting market research.

NeMo Curator streams GPU-accelerated curation for LLM training, with super-fast fuzzy deduplication, multimodal quality filters, semantic dedupl, PII redaction, NSFW checks, and RAPIDS-scaled pipelines for prepping massive corpora.

A practical decision framework for determining whether to use regular expressions or Large Language Models (LLMs) when parsing structured text (e.g., forms, quizzes, invoices). It recommends starting with deterministic, low-cost regex for 95-98% of structured data, and only invoking expensive LLM calls for identifying and validating ambiguous edge cases, optimizing both cost and accuracy.

Guides production ML engineering with patterns for deploying models, building MLOps pipelines, monitoring drift, integrating LLMs and RAG systems, and enforcing feature-store, retraining, and cost controls in live environments.

Build a production-ready, AI-powered data collection agent for any public source (job boards, prices, news, etc.). This agent automatically scrapes data, enriches it using free LLMs (like Gemini Flash), and stores the results in databases such as Notion, Sheets, or Supabase. It runs on a free schedule using GitHub Actions and learns from user feedback, making it ideal for continuous data monitoring and tracking.

ADHX provides a dedicated API to fetch X/Twitter posts as clean, structured JSON. It extracts comprehensive data, including author profiles, engagement metrics (likes, retweets), and full article content for long-form posts. This method is vastly superior to traditional scraping techniques, making it ideal for LLM analysis, summarization, and structured data workflows.

Collection of battle-tested PyTorch training recipes covering architecture selection, optimizers, LR scheduling, precision tips, and debugging to train or fine-tune LLMs, vision, diffusion, and biomedical models with high GPU throughput.

This skill provides comprehensive commands for creating, managing, versioning, and querying structured datasets within the Arize platform. It covers dataset CRUD operations, appending examples, and exporting data in various formats (JSON, CSV, Parquet) using the `ax` CLI. Ideal for preparing test data, generating evaluation examples, or retrieving golden datasets for LLM development and model testing.

This tool functions as a persistent, structured knowledge base for the entire research lifecycle. It accumulates and maps all core research entities—including published papers, generated ideas, experiments, and testable claims—along with their complex relationships (e.g., contradiction, support, inspiration). Inspired by LLM Wiki patterns, it ensures that knowledge is compounded and perpetually queryable, moving beyond single-use literature reviews.

This guide details how to set up and execute large-scale, distributed GPU training jobs on the CoreWeave platform. It covers both single-node multi-GPU setups and multi-node training utilizing PyTorch DDP and Kubernetes resources (A100/H100). This workflow is ideal for fine-tuning large language models (LLMs) or running complex deep learning models that require high-performance, shared-storage compute clusters.