OpenEvidence CI Integration

Table of Contents

• Overview
• Prerequisites
• Instructions
• Output
• Error Handling
• Examples
• Resources

Overview

Integrate OpenEvidence testing and validation into CI/CD pipelines. Covers GitHub Actions workflows, unit tests with SDK mocks, integration tests against sandbox, and clinical validation test suites.

Prerequisites

• GitHub, GitLab, or other CI platform
• OpenEvidence sandbox API credentials
• Test suite configured (vitest recommended)
• Secret management in CI platform

Instructions

Step 1: Create GitHub Actions Workflow

Set up parallel jobs: lint-and-typecheck, unit-tests (with coverage), integration-tests (sandbox only, gated by branch/label), and clinical-validation (on main).

Step 2: Configure Test Environments

Create separate vitest configs: base (unit, 10s timeout), integration (60s timeout, retry 2, maxConcurrency 1 to avoid rate limits).

Step 3: Write Unit Tests with Mocks

Mock @openevidence/sdk to test clinical query service logic, error handling, and response formatting without API calls.

Step 4: Write Integration Tests

Test against sandbox: verify clinical queries return answers with citations, validate empty queries fail gracefully, check citation fields.

Step 5: Add Clinical Validation

Create known-answer test cases that verify responses contain expected medical keywords (e.g., "metformin" for diabetes, "statin" for cardiovascular) with confidence > 0.7.

Step 6: Configure Secrets

Store OPENEVIDENCE_SANDBOX_API_KEY and OPENEVIDENCE_SANDBOX_ORG_ID as GitHub secrets. Use .env.test for local development.

Output

• CI workflow with lint, unit tests, integration tests, clinical validation
• Vitest configurations (unit + integration)
• Unit tests with mocked SDK
• Integration tests against sandbox
• Clinical validation with known-answer testing
• Coverage reports

Error Handling

Examples

Package.json Scripts

{
  "test:unit": "vitest run -c vitest.config.ts",
  "test:integration": "vitest run -c vitest.config.integration.ts",
  "test:clinical-validation": "vitest run tests/clinical-validation"
}

See detailed implementation for advanced patterns.

Resources

• GitHub Actions Documentation
• Vitest Documentation