Skill UI

A comprehensive pipeline for processing, validating, and optimizing scraped web content from Firecrawl. This tool handles markdown cleaning, structured data extraction using Zod validation, content deduplication, and chunking tailored for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems. It ensures crawled data is standardized, clean, and ready for downstream consumption or knowledge base indexing.

Firecrawl provides a comprehensive API for web data harvesting. It enables users to perform single-page scraping, multi-page crawling, site mapping (URL discovery), and advanced structured data extraction using Large Language Models (LLMs). Ideal for integrating data pipelines, building web monitoring tools, or conducting market research.

A practical decision framework for determining whether to use regular expressions or Large Language Models (LLMs) when parsing structured text (e.g., forms, quizzes, invoices). It recommends starting with deterministic, low-cost regex for 95-98% of structured data, and only invoking expensive LLM calls for identifying and validating ambiguous edge cases, optimizing both cost and accuracy.

MarkItDown is a lightweight utility designed to convert heterogeneous documents (PDF, Office files, HTML, etc.) into structured, plain Markdown. It is specifically optimized for text analysis, search indexing, and preparing data for Large Language Models (LLM) and Retrieval-Augmented Generation (RAG) systems, prioritizing content structure over visual fidelity. Supports local, stream, and advanced multimodal extraction scenarios.

This guide details how to set up and execute large-scale, distributed GPU training jobs on the CoreWeave platform. It covers both single-node multi-GPU setups and multi-node training utilizing PyTorch DDP and Kubernetes resources (A100/H100). This workflow is ideal for fine-tuning large language models (LLMs) or running complex deep learning models that require high-performance, shared-storage compute clusters.

A practical decision framework for determining whether to use regular expressions or Large Language Models (LLMs) when parsing structured text (e.g., forms, quizzes, invoices). It recommends starting with deterministic, low-cost regex for 95-98% of structured data, and only invoking expensive LLM calls for identifying and validating ambiguous edge cases, optimizing both cost and accuracy.

A meticulous style guide for authors submitting papers to the International Conference on Database Theory (ICDT). It mandates a rigorous theorem-proof structure, emphasizing precise statement of data models, query languages, and complexity measures. The guide teaches how to structure the introduction to lead with the core result and its data-management implications, adhering to top-tier academic standards.

This guide provides comprehensive best practices for packaging and documenting NLP research artifacts (datasets, models, prompts, etc.) for academic submission (e.g., NAACL/ARR). It emphasizes rigorous provenance tracing, ensuring reproducibility, and ethically handling sensitive data, particularly community-owned or indigenous language resources.