Production reliability patterns for Clay data enrichment pipelines. Clay's async enrichment model, credit-based billing, and multi-provider waterfall architecture require specific resilience strategies.
Track submitted jobs in Redis with table ID, row count, submission time, and status. Detect stuck enrichments by checking elapsed time against a 30-minute timeout.
Track daily credit usage in Redis. Before each batch, check if estimated credits would exceed the daily limit. Stop processing when budget is exhausted to prevent runaway costs.
Track success/failure rates per enrichment provider over a sliding window of 100 requests. Filter out providers below 50% success rate from waterfall configurations.
Process rows in batches of 50 with 2-second delays. On failure, push the batch to a DLQ for manual review instead of blocking the pipeline.
For complete Python implementations with Redis-backed tracking, load the reference guide:
Read(${CLAUDE_SKILL_DIR}/references/implementation-guide.md)
| Issue | Cause | Solution |
|---|---|---|
| Runaway credit spend | No budget limit | Implement credit circuit breaker |
| Stuck enrichments | Async job not completing | Track jobs with timeout detection |
| Provider degradation | Upstream data source down | Monitor provider success rates |
| Batch failures | API errors mid-batch | DLQ for failed rows, retry later |
See Clay Reliability Patterns implementation details for output format specifications.
Basic usage: Apply clay reliability patterns to a standard project setup with default configuration options.
Advanced scenario: Customize clay reliability patterns for production environments with multiple constraints and team-specific requirements.
