Early development. Measured on an RTX 3090: 472 ms from the caller finishing to audio coming back. Concurrency and long-call behaviour are not measured yet.
open-weight · self-hosted · in one process

STT, LLM and TTS
in one process, on your GPU.

Speech-to-text, the language model and text-to-speech run together on hardware you own and stream into each other, so a reply starts playing while it's still being generated. No per-minute API bills. No audio leaves your machine.

Read the docs Get notified about hosted cloud
# clone and install (Python 3.11+)
$ git clone https://github.com/<org>/fusion-runtime.git && cd fusion-runtime
$ pip install -e ".[talk]"
 
# pull the default models (~0.9 GB) and check your machine
$ frun models pull && frun doctor
 
# start the server
$ frun up
 
# open http://localhost:8000 and click Talk
# (or talk from a second terminal: frun talk)
How it works

One process, one GPU, the whole conversation

Everything below runs in-process today — no network hop between stages.

mic audio
→
Silero VADspeech only
→
faster-whisperrolling window
→
turn detectionsilence + resume
→
llama.cppstreams tokens
→
Kokoro TTSspeaks each sentence
→
audio out

A barge-in watcher runs alongside the pipeline: talk over the reply and generation and playback stop. The server decides that, because it hears clean audio — echo cancellation happens on the device, in the browser or in the terminal client, so it works on laptop speakers without headphones.

Measured, not estimated

472 ms from "…and that's my order" to the reply starting

RTX 3090 at $0.50/hour, Qwen 7B, Whisper and Kokoro all on the one card. Nine-turn conversation through the browser client, 19 September 2026.

 MedianRange
Response — caller stops, audio comes back472 ms169–706 ms
Speech-to-text109 ms37–477 ms
LLM first token28 ms20–199 ms
LLM tokens per second~10592–122
Text-to-speech, first chunk442 ms147–619 ms

Interruptions fired on six of six attempts, and correctly ignored 192 ms of speech over the agent. The point isn't that a GPU beats a laptop — it's that a 7 B model answers as fast as a 0.5 B did on an 8 GB MacBook Air, because the pause you configure and the first sentence of speech dominate, not the model. Every figure comes from the runtime's own per-turn telemetry, so you can reproduce them on your own hardware.

On your site

Two script tags, no build step

The runtime serves the browser client it uses itself, so the page you demo with is the one your site embeds.

<script src="https://your-server/fusion-runtime.js"></script>
<button id="talk"></button>
<script>FusionRuntime.attach({ button: "#talk" });</script>

The browser's own echo canceller handles the speaker, the microphone is resampled off the main thread, and replies are scheduled slightly ahead of real time so network jitter doesn't leave gaps. The page never holds an API key — your backend mints it a token that works once. Microphones need https://, so a deployment needs TLS and wss://. Browser client docs →

Why run it this way

What owning the whole stack buys you

You own the latency

Every stage runs in your process. Profile and fix any millisecond yourself instead of waiting on a vendor's queue or incident.

Nothing leaves your machine

Audio and transcripts stay local by default. No third-party inference call in the default path — an architecture, not a config flag.

Zero per-minute cost

No API metering per stage. It runs on hardware you already pay for, so the marginal cost of another conversation is compute, not a bill.

The honest trade

Open weights are behind the best closed APIs — today

Whisper tiny isn't Nova-3. Qwen 0.5B/7B isn't GPT-4o. Kokoro isn't ElevenLabs. That trade is right for some teams and not others.

Good fit

  • Narrow-domain agents — reservations, order status, logistics updates, IVR replacement
  • Privacy / data-residency — teams that can't send audio to a third-party cloud
  • Cost-sensitive, high-volume — where per-minute API pricing is the line item that hurts
  • Local-first developers — kiosks, robots, desktop apps, single-board devices

Not (yet) a fit

  • Open-ended assistants that need frontier-model reasoning
  • Many callers on one GPU, out of the box — conversations are fully isolated, but the in-process model decodes one reply at a time. Point the LLM at vLLM or llama-server and that goes away; shared-model scaling for speech-to-text is the next piece of work
  • Phone deployments — telephony isn't built yet
  • Guaranteed numbers under load — one caller on a 3090 is measured; several at once, and calls of any length, are not

Hosted cloud, later

Not everyone wants to run their own GPU. We're planning a hosted version of fusion-runtime — no date yet, and it isn't built. If you'd rather not run infrastructure yourself, leave your email and we'll tell you when it's real. This isn't a waitlist for the open-source runtime — that's just pip install away.

No spam, one email when it ships. Unsubscribe anytime.