AEC/AGC audit fix + two hardening items from the engine review.
- Add an `autoGainControl` capture param (UrlParams -> CallViewModel ->
ConnectionFactory audioCaptureDefaults), mirroring echoCancellation/
noiseSuppression. Defaults true (unchanged); the host sets it false only for
the ML tier so the browser's auto gain control doesn't fight the in-source ML
denoiser (pumping). Echo cancellation stays on. Tests cover the URL parse and
the audioCaptureDefaults wiring.
- L1: init() now closes the owned AudioContext on a build failure (was orphaned;
browsers cap live contexts, so repeated failures could exhaust them).
- L2: buildGraph() disposes its partially-built nodes on failure (disposeGraph
previously only cleaned the prior graph).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track-B audio-quality changes to reduce the "robotic/underwater" artifact.
- Dry/wet attenuation floor (default 0.15 ≈ -16 dB) blends a little of the raw
mic under the denoised signal so suppression can't fully collapse the noise
floor between words (the main cause of the RNNoise "underwater"/pumping
sound). Applied ONLY to the low-latency flat models (RNNoise/Speex); DTLN/DFN
add algorithmic latency that would comb-filter an undelayed dry mix, so they
rely on their own level instead. Tunable via `lotusDenoiseFloor`.
- Noise gate now runs AFTER the ML model, not before — gating the raw signal
fed hard-zeroed frames into the model and tuned the threshold on pre-denoise
levels.
- DeepFilterNet 3 noiseReductionLevel 80 -> 60: full strength was the main
"over-processed" contributor; 60 keeps voice natural.
Defaults are conservative and tunable; final values are meant to be dialed in
with real-call A/B listening.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track-A robustness fixes from the engine review; no quality/model changes.
- H1: auto-resume the AudioContext on `statechange` if it suspends mid-call
(mobile backgrounding / audio interruption). Previously the dest node emitted
digital silence with no recovery — a silent mute of the sender.
- H2: `resumeCtx()` races `resume()` against a timeout. A suspended context can
only resume on a user gesture; the action can arrive via postMessage, so a
bare `await resume()` inside LiveKit's track-change lock could hang and
deadlock all later mute/unmute/device-switch. Now it proceeds and the H1
watcher heals it.
- M1: don't cache a REJECTED wasm fetch — a transient blip during a reconnect
used to permanently disable denoise for the session. Evict on failure.
- M2: activate denoise off `allConnections$` (local participant's connections)
instead of `livekitRoomItems$`, which excludes the local participant and only
surfaces rooms with a remote member — so denoise now also runs when you're
alone and no longer couples to a remote-render concern.
- Context lifecycle: `closeContext()` removes the state watcher before closing;
`ensureContext()` closes a half-initialised context on any failure (no leak).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Faithful port of cinny's proven pipeline into the TrackProcessor, closing
protocol gap F3 (host offers rnnoise/speex/dtln/deepfilternet; only the
first two existed in-source).
- Fix real bug: gate worklet registers as "noise-gate" (hyphenated), not
"noiseGate" — the gated path would have failed to construct the node.
- Per-model sample rate: DTLN runs at 16kHz, others 48kHz (worklets don't
resample); verify the context actually got the rate, else fall back.
- resume() a suspended context (host postMessage isn't a gesture).
- DTLN via dynamic-imported @workadventure helper (bypassUntilReady);
DeepFilterNet via dynamic-imported ESM + DeepFilterNet3Core pointed at
the self-hosted base. Same-origin base (kept from the C1 fix) makes these
dynamic imports safe.
- Prefer SIMD rnnoise.wasm with non-SIMD fallback; cache wasm per URL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Denoise deep-review (CRITICAL): restart() read opts.audioContext, which
LiveKit does NOT pass on restart — so reconnect (the A7 scenario) and mic
device-switch threw after stopping the old track, leaving the mic SILENT
(A7 reintroduced). Fix:
- Processor owns a dedicated 48kHz AudioContext (sapphi worklets require
48kHz; H1), reused across restart, closed on destroy.
- restart() never throws and never leaves a stopped track on the sender:
builds the new graph first, then disposes the old; on failure degrades
to RAW mic audio rather than silence.
- Cache wasm per URL (no re-fetch each reconnect); gate threshold default
-45 and accept an explicit 0 (M2); document the cross-repo asset contract.
Protocol audit:
- Non-silent warning when an unsupported denoise model (dtln/deepfilternet)
is requested instead of silent rnnoise fallback (F3).
- Correct the call_state enum comment (immediate error-reply, not 10s) (F2).
Build/CI audit:
- Stamp VITE_APP_VERSION in CI; document the vX.Y.Z-lotus.N version scheme.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements RNNoise/Speex noise suppression as a LiveKit audio
TrackProcessor attached to the local mic track, replacing the host's
build-time getUserMedia monkeypatch. Because EC re-attaches the processor
on every (re)publish (LocalTrackPublished), denoise now survives EC's
mid-call reconnect — the root cause of A7 "mic dead after reconnect".
Reuses the worklet/wasm assets already shipped under ./denoise/ (no new EC
dependency); model/gate configured via lotusDenoise/lotusModel/lotusGate
URL params. Additive: no-op unless lotusDenoise=ml.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lotus CIandClaude Opus 4.8