feat(calls): 3-tier mic noise suppression with on-device ML (P5-30)
Replace the boolean call noise-suppression setting with a 3-way control (Off / Browser-native / ML beta) in Settings -> General -> Calls. - Off: noiseSuppression=false to Element Call - Browser-native: EC's built-in WebRTC suppressor (prior default) - ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor) Element Call captures the mic inside its iframe and publishes to LiveKit, so the host can't reach that track; LiveKit's Krisp filter is Cloud-only (we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML tier instead injects a same-origin pre-init shim into the vendored EC index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin) that patches getUserMedia and routes the captured mic through an RNNoise AudioWorklet before LiveKit sees it -- the same post-capture pipeline as #3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if setup fails; keeps echoCancellation/AGC on the raw capture. - settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy boolean migration (true->browser, false->off) - CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends lotusDenoise=ml (native suppressor off in ML mode) - vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and inject the shim <script> before EC's module entry - docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -18,6 +18,7 @@ import {
|
||||
} from 'matrix-widget-api';
|
||||
import { CallWidgetDriver } from './CallWidgetDriver';
|
||||
import { trimTrailingSlash } from '../../utils/common';
|
||||
import { NoiseSuppressionMode } from '../../state/settings';
|
||||
import {
|
||||
ElementCallIntent,
|
||||
ElementCallThemeKind,
|
||||
@@ -100,7 +101,7 @@ export class CallEmbed {
|
||||
room: Room,
|
||||
intent: ElementCallIntent,
|
||||
themeKind: ElementCallThemeKind,
|
||||
noiseSuppression = true,
|
||||
denoiseMode: NoiseSuppressionMode = 'browser',
|
||||
initialAudio = true,
|
||||
initialVideo = false,
|
||||
): Widget {
|
||||
@@ -124,12 +125,21 @@ export class CallEmbed {
|
||||
perParticipantE2EE: room.hasEncryptionStateEvent().toString(),
|
||||
lang: 'en-EN',
|
||||
theme: themeKind,
|
||||
noiseSuppression: noiseSuppression.toString(),
|
||||
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml' we
|
||||
// disable it here so RNNoise (the Lotus denoise shim) owns suppression and
|
||||
// the two don't fight each other.
|
||||
noiseSuppression: (denoiseMode === 'browser').toString(),
|
||||
audio: initialAudio.toString(),
|
||||
video: initialVideo.toString(),
|
||||
header: 'none',
|
||||
});
|
||||
|
||||
if (denoiseMode === 'ml') {
|
||||
// Signal the Lotus denoise shim (injected into the EC index.html) to route
|
||||
// the mic through the RNNoise worklet before LiveKit publishes the track.
|
||||
params.append('lotusDenoise', 'ml');
|
||||
}
|
||||
|
||||
if (CallEmbed.startingCall(intent)) {
|
||||
params.append('sendNotificationType', CallEmbed.dmCall(intent) ? 'ring' : 'notification');
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user