System Overview

The WaveGo Whisper Bot is an offline, autonomous robotic system that combines local Speech-to-Text (STT), Semantic Intent Classification, Retrieval-Augmented Generation (RAG) using a local Small Language Model (SLM), Computer Vision (Object, Face, and Handwriting Detection), and Bluetooth Audio connectivity.

Main Program / Initiative

"Creating Awareness in the Field of Artificial Intelligence through Hands-on Activities for Secondary School Children in Selected Districts of Karnataka, Kerala and Tamil Nadu."

Developed as part of the NIT-K Internship Project 2026 to design and deploy educational, hands-on bionic robotics and embedded AI models to secondary school students.

Core Capabilities

Offline SLM Assistant: CPU-optimized local generation pipeline using Gemma3 model with past Key-Value (KV) cache optimizations.
Semantic Command Execution: Sentence-level semantic parsing via MiniLM-v2 (384-dimensional cosine similarity matching) instead of strict regex keywords.
Computer Vision Dashboard: Facial registration, targeted color tracking, and LeNet-5 CNN visualizer telemetry.
Smart Bluetooth Routing: Auto-scanning and audio sink configuration targeting A2DP speakers via BlueZ interface commands.

System Processing Flow

        [Speech / Text Input]
                  │
                  ▼
         [Vosk Transcription]
                  │
                  ▼
       [Normalize word-digits]  (e.g., "five steps" -> "5 steps")
                  │
                  ▼
         [Intent Classifier]
               /       \
       (Conf >= 0.6)   (Conf < 0.6)
             /           \
            ▼             ▼
     [Gait Control]      [RAG Context Retrieval]
                         [Prompt Assembly]
                         [Gemma3 Generation]
                         [Text Response]

Hardware & Architecture Diagrams

Raspberry Pi Brain Board Architecture

ESP32 Locomotion Driver Board