Overview
Bell Tutor (belltutor.com) is an AI tutoring service that sits with a child while they do schoolwork. The family picks a teacher from eight animated characters, six women and two men, each with their own name and voice. The teacher watches the lesson the child shares, listens, and talks it through like a patient teacher. When a problem needs working out, the teacher writes it on a whiteboard: math, code, graphs, and simple sketches. The first version joined a Google Meet call as its own participant. Today it is a web app families open in Chrome, and the Meet mode is kept for demos. I started it on October 12, 2026 under the working name Lucy. It reached a small pilot four days later and moved to its own name and domain, Bell Tutor at belltutor.com.
The Challenge
A useful tutor has to read the screen, follow speech that comes in fragments, and answer within a second or two, while keeping a whole session in context. It must never hand over the answer to a graded question, even when the student pushes. Lesson videos talk too, so it has to tell the narrator apart from the student. Kids bring weak Chromebooks and flaky Wi-Fi. And because the users are children, privacy, safety, and parental consent had to be built in from the start.
What I Built
- Google Meet participant — The first build used virtual audio devices and a VTube Studio Live2D character fed to Meet through an OBS virtual camera. I then rebuilt it as a headless Chrome bot: an injected script gives Meet a camera drawn from the avatar canvas and a mic fed by the tutor's voice, and the bot sees the call through Chrome DevTools screenshots.
- Real-time voice loop — Voice activity detection, hosted Whisper speech-to-text, and Chatterbox text-to-speech streamed back sentence by sentence, with push-to-talk, barge-in, and a 120-second reconnect grace period.
- Screen vision with an answer policy — Shared-screen frames go to a vision LLM under a hints-only policy. A 16-case eval suite brought answer leaks down to 1 slip in 60 runs per model. Motion detection spots lesson videos so the narrator is treated as background.
- Model routing — A fast open-weight model handles most subjects (about 1.2 s to first words). History and civics switch to a second model, chosen after a side-by-side test.
- Eight teachers — Seven 3D (VRM) characters and one Live2D character, each its own teacher with a name and a synthetic voice, defined in one cast file. They lip-sync to the voice and step down from full 3D to a still portrait on slow computers. If the child switches teachers mid-lesson, the new one picks up the same lesson where it left off.
- Whiteboard — Renders KaTeX, code, plots from a safe expression parser, and sanitized SVG sketches.
- Memory — Postgres with pgvector holds each student's session summaries and recall, walled off by student key.
- Child safety — Real-time distress checks on what the student says, a word filter on every sentence before the teacher speaks it, and a background Llama Guard review of each exchange.
- Parent portal and billing — Magic-link sign-in, clickwrap consent records, Stripe per-child subscriptions, weekly booking, transcripts, a daily summary email, support tickets, and full data deletion.
- GPU pool — A schedule-driven autoscaler rents Vast.ai GPU workers ahead of booked sessions, keeps each student on one worker, destroys orphaned instances, and falls back to Together AI when no worker is ready.
Results
The full pipeline worked in a live Google Meet on day one. By day four the web app was running behind a Cloudflare Tunnel with parent sign-up, Stripe checkout, and nightly encrypted backups with a tested restore. On day five it was rebranded as Bell Tutor across the site, emails, legal pages, and billing, and moved to belltutor.com. Speech-to-text answers in a median 0.70 s, and children's voices never reach a rented GPU. The codebase is about 9,000 lines of Python and JavaScript, with tests for the portal, push-to-talk, and in-session safety. Capacity per GPU is still being measured in the pilot.
Tech Stack
Python, asyncio, WebSockets, JavaScript, Web Audio, WebGL, three.js VRM, Live2D Cubism, PixiJS, KaTeX, Chrome DevTools Protocol, Google Meet, VTube Studio API, OBS, Whisper, Chatterbox TTS, OpenRouter, DeepSeek, Claude Haiku, Llama Guard, PostgreSQL, pgvector, SQLite, Stripe, Vast.ai, LXD, Cloudflare Tunnel, restic, Backblaze B2