Kub — Local AI Assistant
Status: OngoingIt is a personal project in active development, and for now not a public release.
What Kub is
Kub is a desktop app that gives me a ChatGPT-style assistant running on my own computer. For now the language model is Gemma 4 26B-A4B (a mixture-of-experts model), running on my RTX 4080 SUPER through a local CUDA build of llama.cpp; I want the model to be swappable later. The window is WebView2 over a local Python backend. Its servers listen only on this PC, and it needs no account and no online AI service.
What works so far
I want Kub to share the PC with me. When free graphics memory runs low, it moves part of the model to system memory, with the aim that I can keep doing other things on the PC while it runs. How fast it answers varies with the model, how it is split between graphics and system memory, and whether speech is running at the same time. Conversations, spaces, personalisation and opt-in memory stay on the PC, and Kub can use my own text, PDF, Word and PowerPoint files as reference material, for now through local keyword search.
Voice is local too. Push-to-talk dictation in Norwegian and English runs on whisper.cpp: the text lands in the message box so I can correct it, and nothing is sent until I press Send. The audio goes straight to the recogniser without being saved. Kub can read answers aloud, either with the voices built into Windows or with more natural-sounding voices that run locally. For now there are four natural voices, two in Norwegian and two in English. Switching voice keeps the same language model, although the natural voices need so much graphics memory that Kub may reload the model to make room. A separate conversation mode sends each spoken turn automatically and starts reading the answer while it is still being written.
One measurement that changed the design
In a September test with Whisper's small model on 12 Norwegian clips, the word error rate was 21% when Whisper was told the language, and 109% (more errors than there were words) when it had to detect the language itself; one clip collapsed into a repetition loop. So for now, Kub asks which language I'm speaking instead of guessing. The app has since moved to a larger Whisper model.
Not done yet
- Natural read-aloud still has a long cold start, close to a minute before the first reading in one test, and the delivered build has known Norwegian quality problems that are being fixed.
- For now, conversation mode takes turns: interrupting it by voice is planned.
Where I want it to go
My goal is a natural conversation partner that works locally with my own data and tools: it remembers what I approve, keeps my conversations and chosen voice when the model is swapped, asks before going online, and adapts to whatever else is using the PC. That is the direction of my five-year plan, not something Kub does today.
How I started, and how I work on it
I started Kub with questions, not code. Before any implementation, I spent about three hours mapping out the project: I asked what was possible on my own hardware, explained what I wanted Kub to become, and described how it should work in everyday use. That gave me the first direction and requirements. Since then I have kept the same role: I decide what comes next, test hands-on and approve every change.
I work on Kub in small stages, against my five-year plan (2026–2031). I only release a build once it has been checked against its manifest and fingerprint, and the last known good build is kept for rollback.
What I've done so far
- Led the initial discovery and requirements work: asked the questions and set out what Kub should do and how it should work.
- Set the direction: a local assistant that shares the PC with me, keeps my data independent of any one model, and never goes online silently.
- Organised the work in stages: I set the requirements for each stage, test the result, and nothing is pushed or released without my go-ahead.
- Tested the features myself in the packaged app: dictated and sent messages in Norwegian and English, and checked read-aloud and voice switching.
- Compared Norwegian voice settings in blind listening tests on audio samples.
- Required evidence before claims: each result names the build it was measured on, and each release is verified against its fingerprint, with a rollback copy kept.
Tech Breakdown
Python 3.12 backend with a pywebview (WebView2) window and a plain HTML/CSS/JS interface · llama.cpp (CUDA) running Gemma 4 26B-A4B (GGUF), the current model, in its own process · automatic mixture-of-experts offload by free VRAM · BM25 keyword search with stemming over local files · whisper.cpp for speech recognition · Windows SAPI, VoxCPM2 and Chatterbox for speech · pause detection for conversation mode · PyInstaller packaging with a build manifest, SHA-256 fingerprint and rollback · more than 1,100 automated tests