We've just made a breakthrough: the first true omni model running 100% locally in a browser... fully end-to-end, with no separate ASR or TTS models needed.
With custom WebGPU kernels, we're able to run LFM3-Omni-2B at ~280 tok/s on my M4 Max, accepting visual, audio, or text and producing interleaved text and audio tokens. Just 90 ms to the first token and 140 ms to the first audio.
The future of on-device multimodal AI agents is going to be wild.