Skills Development Retell AI Observability

Retell AI Observability

v20260311
retellai-observability
Provides observability coverage for Retell AI voice agents, streaming webhook metrics, latency stats, per-agent KPIs, and alert rules so teams can monitor performance, cost, and call quality.
Get Skill
226 downloads
Overview

Retell AI Observability

Overview

Monitor Retell AI voice agent performance, call quality, and costs. Key signals include call completion rate (successful conversations vs dropped/failed calls), average call duration, latency between user speech and agent response (conversational latency), per-minute cost tracking, and agent-level success metrics (did the voice agent accomplish its goal).

Prerequisites

  • Retell AI account with active voice agents
  • API access for call data queries
  • Webhook endpoint for real-time call events

Instructions

Step 1: Monitor Call Quality via Webhooks

// retell-webhook-handler.ts
app.post('/webhooks/retell', (req, res) => {
  const { call_id, agent_id, status, duration_seconds, cost_usd, disconnect_reason } = req.body;

  emitCounter('retell_calls_total', 1, { agent: agent_id, status });
  emitHistogram('retell_call_duration_sec', duration_seconds, { agent: agent_id });
  emitCounter('retell_cost_usd', cost_usd, { agent: agent_id });

  if (disconnect_reason === 'agent_error' || disconnect_reason === 'system_error') {
    emitCounter('retell_call_errors_total', 1, { agent: agent_id, reason: disconnect_reason });
  }

  res.sendStatus(200);  # HTTP 200 OK
});

Step 2: Track Conversational Latency

set -euo pipefail
# Query recent calls for response latency metrics
curl "https://api.retellai.com/v1/calls?limit=20&sort=-created_at" \
  -H "Authorization: Bearer $RETELL_API_KEY" | \
  jq '.[] | {
    call_id, agent_name, duration_sec: .duration,
    avg_response_latency_ms: .avg_agent_response_latency_ms,
    cost_usd: .cost,
    disconnect_reason
  }'

Step 3: Monitor Per-Agent Performance

// Track which agents are performing well vs poorly
async function agentPerformanceReport() {
  const agents = await retellApi.listAgents();
  for (const agent of agents) {
    const calls = await retellApi.listCalls({ agent_id: agent.agent_id, limit: 100 });
    const completed = calls.filter(c => c.status === 'completed').length;
    const avgDuration = calls.reduce((s, c) => s + c.duration, 0) / calls.length;
    const totalCost = calls.reduce((s, c) => s + c.cost, 0);

    emitGauge('retell_agent_completion_rate', completed / calls.length * 100, { agent: agent.agent_name });
    emitGauge('retell_agent_avg_duration_sec', avgDuration, { agent: agent.agent_name });
    emitGauge('retell_agent_total_cost_usd', totalCost, { agent: agent.agent_name });
  }
}

Step 4: Alert on Voice Quality Issues

groups:
  - name: retell
    rules:
      - alert: RetellHighDropRate
        expr: rate(retell_calls_total{status="failed"}[1h]) / rate(retell_calls_total[1h]) > 0.1
        annotations: { summary: "Retell call failure rate exceeds 10%" }
      - alert: RetellHighLatency
        expr: histogram_quantile(0.95, rate(retell_response_latency_ms_bucket[1h])) > 2000  # 2000: 2 seconds in ms
        annotations: { summary: "Retell agent response latency P95 exceeds 2 seconds" }
      - alert: RetellCostSpike
        expr: increase(retell_cost_usd[1h]) > 50
        annotations: { summary: "Retell voice costs exceed $50/hour" }
      - alert: RetellShortCalls
        expr: histogram_quantile(0.25, rate(retell_call_duration_sec_bucket[1h])) < 10
        annotations: { summary: "25% of calls ending in <10 seconds (agent issue?)" }

Step 5: Dashboard Panels

Track: call volume by agent, call completion rate (pie chart), duration distribution, per-minute cost trend, conversational latency p50/p95, disconnect reasons breakdown, and daily cost by agent. Short calls (<10s) often indicate agent prompt issues where the bot fails to engage.

Error Handling

Issue Cause Solution
High call drop rate Agent prompt causing hang-ups Review and simplify agent greeting prompt
Latency >2 seconds LLM response slow Use faster model or reduce prompt complexity
Unexpected high costs Long average call duration Add conversation time limits in agent config
No webhook events Endpoint unreachable Verify webhook URL and SSL certificate

Examples

Basic usage: Apply retellai observability to a standard project setup with default configuration options.

Advanced scenario: Customize retellai observability for production environments with multiple constraints and team-specific requirements.

Output

  • Configuration files or code changes applied to the project
  • Validation report confirming correct implementation
  • Summary of changes made and their rationale

Resources

  • Official monitoring documentation
  • Community best practices and patterns
  • Related skills in this plugin pack
Info
Category Development
Name retellai-observability
Version v20260311
Size 5.19KB
Updated At 2026-03-12
Language