feat(calls): 3-tier mic noise suppression with on-device ML (P5-30)
Replace the boolean call noise-suppression setting with a 3-way control (Off / Browser-native / ML beta) in Settings -> General -> Calls. - Off: noiseSuppression=false to Element Call - Browser-native: EC's built-in WebRTC suppressor (prior default) - ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor) Element Call captures the mic inside its iframe and publishes to LiveKit, so the host can't reach that track; LiveKit's Krisp filter is Cloud-only (we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML tier instead injects a same-origin pre-init shim into the vendored EC index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin) that patches getUserMedia and routes the captured mic through an RNNoise AudioWorklet before LiveKit sees it -- the same post-capture pipeline as #3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if setup fails; keeps echoCancellation/AGC on the raw capture. - settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy boolean migration (true->browser, false->off) - CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends lotusDenoise=ml (native suppressor off in ML mode) - vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and inject the shim <script> before EC's module entry - docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
+25
-2
@@ -405,9 +405,32 @@ A local sound plays when another participant joins or leaves a call you're in.
|
||||
|
||||
Files: `src/app/utils/callSounds.ts`, `src/app/hooks/useCallJoinLeaveSounds.ts`
|
||||
|
||||
### Noise Suppression Toggle
|
||||
### Noise Suppression (3-Tier, incl. on-device ML) (P5-30)
|
||||
|
||||
A `noiseSuppression` URL parameter is passed to the Element Call widget URL, allowing the noise suppression feature to be toggled from within Lotus settings.
|
||||
A three-way mic noise-suppression control in **Settings → General → Calls**:
|
||||
|
||||
| Tier | What it does |
|
||||
|---|---|
|
||||
| **Off** | No suppression (`noiseSuppression=false` to Element Call). |
|
||||
| **Browser-native** | Element Call's built-in WebRTC suppressor (`noiseSuppression=true`). Default. |
|
||||
| **ML (beta)** | On-device RNNoise — Krisp-style removal of fans, keyboards, dogs, etc. |
|
||||
|
||||
**Why a shim, not a fork:** Element Call captures the mic *inside* its iframe and publishes to LiveKit; the host can't reach that track. LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU), and EC's own RNNoise work (PR #3892) is unmerged. So the **ML tier** is delivered by injecting a same-origin pre-init script into the vendored EC `index.html` that monkeypatches `getUserMedia` and routes the captured mic through an RNNoise `AudioWorklet` (`@sapphi-red/web-noise-suppressor`) before LiveKit ever sees it — the same post-capture pipeline #3892 uses, executed from the realm we already control. Works on the self-hosted LiveKit SFU, survives EC version bumps, no EC fork/AGPL/rebase burden.
|
||||
|
||||
**How it's wired:**
|
||||
|
||||
- `callNoiseSuppression` setting is `'off' | 'browser' | 'ml'` (legacy boolean migrates: `true`→`browser`, `false`→`off`)
|
||||
- `CallEmbed.getWidget()` maps the tier to the `noiseSuppression` URL param and appends `lotusDenoise=ml` for the ML tier (browser-native suppressor is disabled in ML mode so RNNoise owns suppression)
|
||||
- The `lotusDenoise` vite plugin copies the RNNoise worklet + wasm into `public/element-call/denoise/`, copies the shim, and injects `<script src="./lotus-denoise.js">` before EC's module entry
|
||||
- The shim keeps `echoCancellation`/`autoGainControl` on the raw capture and falls back to the raw mic if RNNoise setup fails, so calls never break
|
||||
|
||||
**Known beta caveat:** routing capture through WebAudio can weaken the browser's acoustic echo cancellation (AEC runs on the native capture track) — the same tradeoff EC's upstream feature makes; hence the "beta" label.
|
||||
|
||||
### Files
|
||||
|
||||
- `build/lotus-denoise.js` — injected RNNoise getUserMedia shim (classic script)
|
||||
- `vite.config.js` — `lotusDenoise()` plugin (asset copy + index.html injection)
|
||||
- `src/app/plugins/call/CallEmbed.ts` — tier → widget URL params
|
||||
|
||||
### Call Button Scoping
|
||||
|
||||
|
||||
Reference in New Issue
Block a user