Exa Reference Architecture

Overview

Production architecture for Exa neural search integration. Covers search pipeline design, content extraction patterns, embedding-based retrieval, and caching strategies for semantic search applications.

Prerequisites

• Exa API key
• Understanding of neural vs keyword search
• TypeScript project with async support
• Cache layer for search result deduplication

Architecture Diagram

┌─────────────────────────────────────────────────────┐
│                Application Layer                     │
│   RAG Pipeline │ Research Agent │ Content Discovery  │
└───────────┬─────────────┬────────────┬──────────────┘
            │             │            │
            ▼             ▼            ▼
┌─────────────────────────────────────────────────────┐
│              Exa Search Service                      │
│  ┌───────────┐  ┌───────────┐  ┌────────────────┐   │
│  │ Neural    │  │ Keyword   │  │ Auto (hybrid)  │   │
│  │ Search    │  │ Search    │  │ Search         │   │
│  └─────┬─────┘  └─────┬─────┘  └──────┬─────────┘   │
│        │              │               │              │
│        ▼              ▼               ▼              │
│  ┌──────────────────────────────────────────────┐    │
│  │           Content Extraction                  │    │
│  │   Text │ Highlights │ Full HTML │ Summary    │    │
│  └──────────────────────┬───────────────────────┘    │
│                         │                            │
│  ┌──────────────────────┴───────────────────────┐    │
│  │           Result Cache (LRU + Redis)          │    │
│  └──────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────┘

Instructions

Step 1: Exa Client Service Layer

import Exa from 'exa-js';

const exa = new Exa(process.env.EXA_API_KEY!);

interface SearchOptions {
  query: string;
  type?: 'neural' | 'keyword' | 'auto';
  numResults?: number;
  startDate?: string;
  endDate?: string;
  includeDomains?: string[];
  excludeDomains?: string[];
  category?: string;
}

async function searchWithContents(options: SearchOptions) {
  return exa.searchAndContents(options.query, {
    type: options.type || 'auto',
    numResults: options.numResults || 10,
    text: { maxCharacters: 3000 },  # 3000: 3 seconds in ms
    highlights: { numSentences: 3 },
    startPublishedDate: options.startDate,
    endPublishedDate: options.endDate,
    includeDomains: options.includeDomains,
    excludeDomains: options.excludeDomains,
    category: options.category,
  });
}

Step 2: Search Pipeline with Content Extraction

async function researchTopic(topic: string) {
  // Step 1: Broad neural search for relevant sources
  const sources = await exa.searchAndContents(topic, {
    type: 'neural',
    numResults: 20,
    text: true,
    highlights: { numSentences: 5 },
    startPublishedDate: '2024-01-01',  # 2024 year
    category: 'research paper',
  });

  // Step 2: Find similar content to top results
  const topUrl = sources.results[0]?.url;
  const similar = topUrl
    ? await exa.findSimilarAndContents(topUrl, {
        numResults: 5,
        text: { maxCharacters: 2000 },  # 2000: 2 seconds in ms
      })
    : { results: [] };

  return {
    primary: sources.results,
    similar: similar.results,
    totalSources: sources.results.length + similar.results.length,
  };
}

Step 3: RAG Integration Pattern

async function ragSearch(
  userQuery: string,
  contextWindow = 5
) {
  const results = await exa.searchAndContents(userQuery, {
    type: 'neural',
    numResults: contextWindow,
    text: { maxCharacters: 2000 },  # 2000: 2 seconds in ms
    highlights: { numSentences: 3 },
  });

  // Format for LLM context injection
  const context = results.results.map((r, i) => (
    `[Source ${i + 1}] ${r.title}\n` +
    `URL: ${r.url}\n` +
    `Content: ${r.text}\n` +
    `Key highlights: ${r.highlights?.join(' | ')}\n`
  )).join('\n---\n');

  return {
    context,
    sources: results.results.map(r => ({
      title: r.title,
      url: r.url,
      score: r.score,
    })),
  };
}

Step 4: Domain-Specific Search Configuration

const SEARCH_PROFILES = {
  technical: {
    includeDomains: ['github.com', 'stackoverflow.com', 'arxiv.org', 'docs.python.org'],
    category: 'github repo' as const,
  },
  news: {
    includeDomains: ['techcrunch.com', 'theverge.com', 'arstechnica.com'],
    category: 'news' as const,
  },
  research: {
    includeDomains: ['arxiv.org', 'scholar.google.com', 'nature.com'],
    category: 'research paper' as const,
  },
};

async function profiledSearch(query: string, profile: keyof typeof SEARCH_PROFILES) {
  const config = SEARCH_PROFILES[profile];
  return searchWithContents({ query, ...config, numResults: 10 });
}

Error Handling

Examples

Content Discovery Pipeline

async function discoverCompetitors(companyUrl: string) {
  const similar = await exa.findSimilar(companyUrl, {
    numResults: 10,
    excludeDomains: [new URL(companyUrl).hostname],
  });
  return similar.results.map(r => ({ title: r.title, url: r.url }));
}

Resources

• Exa API Documentation
• Exa Search Types
• Exa Content Extraction

Output

• Configuration files or code changes applied to the project
• Validation report confirming correct implementation
• Summary of changes made and their rationale