Skip to content
OpenCohost Kira
Source-of-truth developer guide

Verified architecture, current implementation, and active-development boundaries.

This section was refreshed against Engram project memory and the current website copy. It separates shipped behavior from old marketing copy and future work, because developer documentation is only useful when it refuses to overpromise.

Audit snapshot

Green claims were already aligned with project memory. Amber claims were corrected in this pass. Blue claims are active-development boundaries that developers should not treat as release guarantees.

Verified

OpenCohost is the orchestration chassis

Correct: the product value is coordinating Ollama, TTS, LiveAudio context, Agenda Mode, chat reaction, profiles, avatar/music presence, and resilience rather than being only a wrapper.

Corrected

The voice story is not just Qwen-TTS

Updated: the verified implementation includes Piper local synthesis, optional Edge-TTS light synthesis when privacy allows it, and active custom/Qwen voice work. The docs no longer promise a guaranteed Qwen-only stack.

Verified

Local-first does not mean compute-free

Correct: local mode avoids default cloud/API billing, but developers must still budget GPU/VRAM, RAM, electricity, model choice, game load, and setup time.

Verified

LiveAudio stays a separate bridge

Correct: voice listening, Silero VAD, Whisper transcription, subtitles, transcripts, and clean voice context belong to the LiveAudio bridge rather than an always-on OpenCohost microphone path.

Corrected

Installer path now real and documented

The site now links the CI-generated Windows alpha installer on GitHub Releases and labels it as alpha; the old no-installer-promise copy was removed once the release flow became real.

Composition root

app_shell.py wires the motor thread, health monitor, OBS client, SmartAggregator, Stream Admin, topic inbox bridge, TTS controls, and UI panels through typed protocols. Keep it orchestration-only; push logic into testable modules.

UIState observer

A framework-agnostic, thread-safe state container with typed properties. Observers dispatch on a daemon thread; UI callbacks must hop back to the Tk main loop before touching widgets.

Priority queue + accumulation

The motor thread prioritizes real-time inputs such as PTT, chat, and agenda work while compacting overflow into bounded consultations so local models are not buried by raw stream noise.

LLM tier switching

Manual Quality / Balanced / Fast slots map to Ollama model tags. Switching preserves conversation/profile state and rolls back to the last known good model on failure.

Privacy-aware voice path

The TTS path supports Piper local synthesis, persisted Piper speed presets, and a tts_local_only switch that blocks Edge-TTS before text can leave the machine.

Human-gated topic intake

The topic inbox lets agents propose ideas, but approve remains human-only. Read-time namespace validation, short SQLite timeouts, and rollback keep UI suggestions safe under load.

Avatar state bridge

A pub/sub bridge lets core modules signal avatar states without coupling to UI or OBS. OBSClient can subscribe and update image sources when state changes.

Degradation ladder

Agenda recovery and model switching prefer graceful degradation: retries, stale-prefetch discard, explicit pause states, and rollback instead of silent deadlocks during a live show.

Recent verified implementation notes

TTS local-only switch

tts_local_only.json is persisted under config. When ON, the motor routes light synthesis to Piper and server_qwen.py returns HTTP 400 before any Edge-TTS call can run.

Piper speed presets

The UI exposes Rápida, Media, Calma, and Lenta presets backed by length_scale values. Engine changes persist to tts_speed.json and rebuild Piper synthesis config under lock.

Agenda double-close fix

The controller no longer prefetches a second kira-agenda-stop while a topic is already CLOSING, and stale prefetched actions are discarded instead of played later.

Topic inbox

Agents can propose topic candidates with the ti_ namespace. UI polling is fail-open, approval is human-only, and failed queue operations roll back approved rows.

Gotchas for contributors

Never proxy internal state from external metadata
A real agenda bug came from checking current_speech_source.startsWith("kira-agenda") to infer controller state. The controller can emit actions with source="chat"; trust the controller state, not labels attached to an event.
Tk widgets are single-threaded
Any widget mutation must happen on the main loop. The UIState observer dispatch thread is separate, so callbacks must use schedule_ui_update() / after_idle before touching Tk widgets.
Reasoning token budget
Models such as qwen3 and gemma can spend part of the token budget on internal reasoning. A hard num_predict cap can yield empty or truncated answers; the engine removes the cap for those families.
Storage paths resolve at import time
STORAGE_PATHS is resolved when config/storage.py is imported. apply_storage_environment() runs before library initialization; changing storage.yaml mid-runtime will not move already-resolved paths.
Privacy gates must run before convenience fallbacks
The local-only switch must be checked before the Edge-TTS offline/light fallback. Reordering those branches can silently send text to Microsoft even when the user asked for local-only synthesis.
Persisted settings can pollute tests
Any motor command test that writes preferences such as tts_local_only or tts_speed must patch the save/load helper to a temporary path; otherwise it can mutate the user's real config.
Validate topic namespaces when reading, not only when routing
Hostile or legacy rows can already exist in SQLite. The topic inbox quarantines foreign IDs at read time so render and dismiss paths agree on what the UI owns.
app_shell.py has a hard line budget
The integration guard keeps app_shell.py below 3100 lines. New UI behavior should usually live in a small injected module, with app_shell only wiring it.

Extension points

OpenCohost does not have a formal plugin system yet, but developers can extend these surfaces deliberately:

  • Profiles (perfiles.json): system prompt text and use_system flag. Defaults live in config/default_profiles.json.
  • LLM tier slots (llm_tiers.json): quality, balanced, and fast model slots validated against runtime-installed Ollama models at startup.
  • TTS privacy and speed files: tts_local_only.json and tts_speed.json persist user voice policy and Piper length_scale choices.
  • Config YAMLs: storage.yaml, smart_aggregator.yaml, avatar.yaml, and stream_admin.yaml configure paths, chat shaping, OBS/avatar state, OAuth, and moderation.
  • Model catalog (config/settings.py): MODELS_CATALOG entries need display, desc, size_gb, and family metadata before appearing safely in the UI.
  • Protocol system (ui/protocols.py): MotorEventCallback, SmartAggregatorCallbacks, and StreamAdminCallbacks define typed callback contracts.
  • Topic inbox (opencohost/core/topic_inbox.py + ui/topic_inbox_bridge.py): add agent proposals without bypassing human approval.
  • Crash reporting (ui/crash_reporting.py): Python excepthook, threading excepthook, Tk callback hook, and faulthandler cover different failure classes.
Solved challenges

Hard live-stream problems we already had to solve

Kira was shaped by real streaming problems: noisy chats, delayed speech, local hardware limits, long sessions, and the need to stay useful without taking control away from the host.

Multiple AI workloads on one streamer PC

Challenge

Running a local co-host is not one button and magic. The stream may need a local LLM, custom voice output, and a listening/transcription path while OBS, the game, and overlays are also active.

Live example

Game + OBS + Kira voice + LiveAudio listening on the same machine.

Result

OpenCohost coordinates the pieces and makes the tradeoff visible: quality, speed, VRAM, and stream load stay under the host's control.

Agenda Mode with live context

Challenge

A co-host needs rhythm. If Kira only reads a fixed script, she feels dead. If she reacts to everything, she becomes noise. The challenge was keeping her focused while still letting the stream influence the conversation.

Live example

Kira can continue a planned topic, notice the room changing, and adapt the angle without stealing control from the streamer.

Result

We shaped Agenda Mode around a reactive event loop: the host sets direction, Kira keeps flow, and chat context can bend the conversation without taking over.

Co-host silence recovery

Challenge

In earlier versions, some combinations of co-host mode and live chat could leave Kira quiet. That is one of the worst failures for a stream assistant: the audience sees nothing, the host gets no help, and the moment dies.

Live example

A stream with active chat and an agenda running should not make Kira pause forever.

Result

We corrected the product flow so co-host behavior, chat reaction, and streamer control cooperate instead of blocking one another.

Chat context without overwhelming the model

Challenge

Injecting raw live chat into a local LLM is the fastest way to kill the experience. Streams are messy: spam, jokes, repeated lines, reaction waves, short messages, and sudden chat storms all arrive mixed together.

Live example

Instead of Kira repeating chat word-for-word, she understands the room's pressure and reacts like a co-host.

Result

OpenCohost shapes chat into usable context so Kira can respond to the stream atmosphere without mirroring the room or getting overwhelmed when chat gets chaotic.

Push-to-talk for real voice control

Challenge

When LiveAudio sends every transcription through WebSocket, Kira can react to too much. Without headphones, she may even hear herself through the microphone and answer in a loop. Also, speech transcription can arrive a little late.

Live example

Hold a key, speak naturally — even a long thought — then release and Kira receives the full spoken context.

Result

PTT makes listening intentional: it collects voice while active, sends the full context on release, and accepts a small late-transcription margin for more natural interaction.

Robustness under live pressure

Challenge

Live shows are not clean demos. Connections can hiccup, components can be busy, windows can stay open for hours, and inputs can arrive in unexpected ways. A streamer tool has to bend before it breaks.

Live example

If one part is delayed or unavailable, the whole show should not collapse around it.

Result

OpenCohost was hardened so problems degrade gracefully: Kira should remain useful even when the stream environment gets messy.

Long-session memory discipline

Challenge

A stream can run for hours. If every message, joke, transcript, and response is treated as equally important, the local model gets buried and Kira loses focus.

Live example

Kira should remember the useful direction of the show, not drag every old chat line forever.

Result

OpenCohost keeps continuity through compact context instead of endless memory, helping Kira stay coherent across long sessions.

Avatar and OBS presence

Challenge

Kira needed to feel present on stream, not hidden inside a textbox. The audience should understand when she is listening, thinking, or speaking without the host explaining it every time.

Live example

A viewer can glance at the stream and immediately read Kira's current presence.

Result

We connected visual presence with OBS-friendly behavior so Kira feels like part of the show, not just a background tool.

A UI for streamers, not technicians

Challenge

The first UI was too dense. It exposed too many controls at once and felt closer to a technical cockpit than a stream product. That makes people hesitate, even when the features are useful.

Live example

A host should quickly find voice, stream, co-host, music, and avatar controls without reading a manual first.

Result

We remodeled the interface around clearer tabs, calmer sections, explanations, and a stronger Kira-focused layout for non-technical users.

Music that behaves around Kira

Challenge

Background music can make the stream feel alive, but it can also fight the co-host. If music keeps stepping over Kira, the audience has to work harder to understand the moment.

Live example

Kira speaks, the stream stays readable, and music supports the atmosphere instead of competing with her.

Result

OpenCohost now handles music flow more gracefully so the show feels smoother when Kira enters the conversation.

LiveAudio as a separate listening bridge

Challenge

Voice listening is important enough to live as its own connected piece. LiveAudio detects when someone is actually speaking, transcribes voice locally, and passes clean context into OpenCohost.

Live example

Speech can become subtitles, transcripts, and usable context without making OpenCohost itself permanently listen by default.

Result

LiveAudio uses Silero VAD and OpenAI Whisper open-source transcription as a separate bridge for voice context, subtitles, and transcripts.