Groq Events & Async Patterns

Overview

Build event-driven architectures around Groq's ultra-fast LLM inference API. Groq does not provide native webhooks, but its sub-second response times at api.groq.com enable unique patterns: real-time streaming, batch processing with callbacks, and event-driven pipelines where Groq acts as the processing engine within your webhook infrastructure.

Prerequisites

• Groq API key stored in GROQ_API_KEY environment variable
• Groq SDK installed (npm install groq-sdk or pip install groq)
• Queue system for batch processing (BullMQ, Celery)
• Understanding of Groq model options (llama, mixtral, gemma)

Event Patterns

Instructions

Step 1: Real-Time Streaming with SSE

import Groq from "groq-sdk";

const groq = new Groq({ apiKey: process.env.GROQ_API_KEY! });

// Express SSE endpoint
app.post("/api/chat/stream", async (req, res) => {
  const { messages, model } = req.body;

  res.writeHead(200, {  # HTTP 200 OK
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    "Connection": "keep-alive",
  });

  const stream = await groq.chat.completions.create({
    model: model || "llama-3.3-70b-versatile",
    messages,
    stream: true,
    max_tokens: 2048,  # 2048: 2 KB
  });

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content;
    if (content) {
      res.write(`data: ${JSON.stringify({ content })}\n\n`);
    }
  }

  res.write("data: [DONE]\n\n");
  res.end();
});

Step 2: Batch Processing with Callbacks

import { Queue, Worker } from "bullmq";

const groqQueue = new Queue("groq-batch");

// Queue a batch of prompts with callback webhook
async function queueBatch(prompts: string[], callbackUrl: string) {
  const batchId = crypto.randomUUID();

  for (const [index, prompt] of prompts.entries()) {
    await groqQueue.add("inference", {
      batchId,
      index,
      prompt,
      callbackUrl,
      totalItems: prompts.length,
    });
  }

  return batchId;
}

const worker = new Worker("groq-batch", async (job) => {
  const { prompt, callbackUrl, batchId, index, totalItems } = job.data;

  const completion = await groq.chat.completions.create({
    model: "llama-3.3-70b-versatile",
    messages: [{ role: "user", content: prompt }],
  });

  const result = {
    batchId,
    index,
    content: completion.choices[0].message.content,
    usage: completion.usage,
    model: completion.model,
    processingTime: completion.usage?.total_time,
  };

  // Fire callback webhook on completion
  await fetch(callbackUrl, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      event: "groq.batch_item.completed",
      data: result,
    }),
  });

  return result;
});

Step 3: Webhook Event Processor

// Use Groq to process incoming webhook events with LLM
async function processWebhookWithGroq(event: any) {
  const completion = await groq.chat.completions.create({
    model: "llama-3.1-8b-instant",
    messages: [
      {
        role: "system",
        content: "Classify this event and extract key information. Respond with JSON.",
      },
      {
        role: "user",
        content: JSON.stringify(event),
      },
    ],
    response_format: { type: "json_object" },
    temperature: 0,
  });

  return JSON.parse(completion.choices[0].message.content!);
}

Step 4: Monitor API Health

async function checkGroqHealth(): Promise<boolean> {
  try {
    const start = Date.now();
    await groq.chat.completions.create({
      model: "llama-3.1-8b-instant",
      messages: [{ role: "user", content: "ping" }],
      max_tokens: 1,
    });
    const latency = Date.now() - start;
    console.log(`Groq health: OK (${latency}ms)`);
    return true;
  } catch (error) {
    console.error("Groq health check failed:", error);
    return false;
  }
}

Error Handling

Examples

Python Async Batch

import asyncio
from groq import AsyncGroq

client = AsyncGroq(api_key=os.environ["GROQ_API_KEY"])

async def process_batch(prompts: list[str]):
    tasks = [
        client.chat.completions.create(
            model="llama-3.3-70b-versatile",
            messages=[{"role": "user", "content": p}],
        )
        for p in prompts
    ]
    return await asyncio.gather(*tasks)

Resources

• Groq API Documentation
• Groq Models
• Groq Rate Limits

Next Steps

For deployment setup, see groq-deploy-integration.

Output

• Configuration files or code changes applied to the project
• Validation report confirming correct implementation
• Summary of changes made and their rationale