feat(calls): 3-tier mic noise suppression with on-device ML (P5-30)
CI / Build & Quality Checks (push) Successful in 10m33s
Trigger Desktop Build / trigger (push) Successful in 6s

Replace the boolean call noise-suppression setting with a 3-way control
(Off / Browser-native / ML beta) in Settings -> General -> Calls.

- Off: noiseSuppression=false to Element Call
- Browser-native: EC's built-in WebRTC suppressor (prior default)
- ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor)

Element Call captures the mic inside its iframe and publishes to LiveKit,
so the host can't reach that track; LiveKit's Krisp filter is Cloud-only
(we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML
tier instead injects a same-origin pre-init shim into the vendored EC
index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin)
that patches getUserMedia and routes the captured mic through an RNNoise
AudioWorklet before LiveKit sees it -- the same post-capture pipeline as
#3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if
setup fails; keeps echoCancellation/AGC on the raw capture.

- settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy
  boolean migration (true->browser, false->off)
- CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends
  lotusDenoise=ml (native suppressor off in ML mode)
- vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and
  inject the shim <script> before EC's module entry
- docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 20:29:59 -04:00
parent f9edd2023d
commit 5deed79b42
10 changed files with 299 additions and 38 deletions
+12 -2
View File
@@ -18,6 +18,7 @@ import {
} from 'matrix-widget-api';
import { CallWidgetDriver } from './CallWidgetDriver';
import { trimTrailingSlash } from '../../utils/common';
import { NoiseSuppressionMode } from '../../state/settings';
import {
ElementCallIntent,
ElementCallThemeKind,
@@ -100,7 +101,7 @@ export class CallEmbed {
room: Room,
intent: ElementCallIntent,
themeKind: ElementCallThemeKind,
noiseSuppression = true,
denoiseMode: NoiseSuppressionMode = 'browser',
initialAudio = true,
initialVideo = false,
): Widget {
@@ -124,12 +125,21 @@ export class CallEmbed {
perParticipantE2EE: room.hasEncryptionStateEvent().toString(),
lang: 'en-EN',
theme: themeKind,
noiseSuppression: noiseSuppression.toString(),
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml' we
// disable it here so RNNoise (the Lotus denoise shim) owns suppression and
// the two don't fight each other.
noiseSuppression: (denoiseMode === 'browser').toString(),
audio: initialAudio.toString(),
video: initialVideo.toString(),
header: 'none',
});
if (denoiseMode === 'ml') {
// Signal the Lotus denoise shim (injected into the EC index.html) to route
// the mic through the RNNoise worklet before LiveKit publishes the track.
params.append('lotusDenoise', 'ml');
}
if (CallEmbed.startingCall(intent)) {
params.append('sendNotificationType', CallEmbed.dmCall(intent) ? 'ring' : 'notification');
}