Back to Projects
AI Voice Agent
AI Edcademy
2026

Chappo — AI Voice Agent

Chappo is a production-style multilingual AI voice agent that lets website visitors talk to AI Edcademy in real time over WebRTC. It answers questions grounded in company knowledge via RAG, recommends trainers, captures qualified leads, and gives admins full visibility into conversations, costs, uploads, and visitor activity. Built as a dual-repo NestJS + Next.js system for embeddable voice chat on the company website.

Key Features

  • Real-time bidirectional voice conversations using WebRTC and the OpenAI Realtime API, with server-side VAD, Whisper transcription, and an animated avatar UI
  • RAG-grounded answers through two Qdrant collections (website content and trainers), powered by OpenAI text-embedding-3-small and tool-calling (searchWebsiteContent, searchTrainers)
  • AI tool orchestration for lead capture (captureLead) and graceful conversation end (endConversation), executed on NestJS and returned over the WebRTC data channel
  • Multilingual support for English, Traditional Chinese, Japanese, and auto-detect, including language-aware system prompts, Whisper bias for Traditional Chinese, OpenCC Simplified→Traditional normalization, and hallucination filtering
  • Embeddable iframe voice widget (/chat/embed) for hosting Chappo on external pages
  • JWT-protected admin panel: dashboard KPIs, conversation transcripts with filters, lead management with CSV export, profile (email/password) updates
  • Knowledge-base administration: upload and process PDFs/DOCX/TXT/MD and trainer Excel/CSV files, with chunking, embedding, Qdrant upsert, status tracking, search, and pagination
  • Scheduled ingestion cron jobs (every 10 minutes) with manual process triggers and retry/backoff for embedding APIs
  • Session lifecycle tracking with per-message token usage, audio duration, and OpenAI Realtime cost aggregation
  • Visitor logging with IP geolocation (ip-api.com), search/sort, and IP block/unblock enforcement on token and session creation
  • Rate limiting (global + per-route), CORS, DTO validation, Winston structured logging, and global exception handling

Technical Architecture

  • Modular NestJS monolith with feature modules (auth, realtime, tools, sessions, leads, qdrant, admin, visitors, trainers, website-content, logger)
  • Service-layer design: controllers → services → TypeORM repositories / Qdrant client
  • Frontend component-based App Router architecture with Zustand for client state
  • Hybrid realtime pattern: REST for tokens, sessions, tools, and admin APIs; WebRTC + data channel for low-latency audio and tool-call events
  • RAG pipeline: upload → parse → chunk (overlap) → embed (batched, retried) → upsert to Qdrant → semantic search at query time
  • Tool definitions as a single source of truth shared by Realtime session config and tool execution

Highlights

Chappo demonstrates end-to-end ownership of a complex realtime AI product: WebRTC voice UX, ephemeral Realtime sessions, function-calling tool loops, dual-collection RAG with document/Excel ingestion, multilingual speech handling, lead capture, usage/cost analytics, and an operational admin console with visitor controls.
The architecture cleanly separates browser-side media from NestJS business logic while keeping tool schemas, session state, and vector knowledge synchronized for production-ready website engagement.