feat(calls): 3-tier mic noise suppression with on-device ML (P5-30)
CI / Build & Quality Checks (push) Successful in 10m33s
Trigger Desktop Build / trigger (push) Successful in 6s

Replace the boolean call noise-suppression setting with a 3-way control
(Off / Browser-native / ML beta) in Settings -> General -> Calls.

- Off: noiseSuppression=false to Element Call
- Browser-native: EC's built-in WebRTC suppressor (prior default)
- ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor)

Element Call captures the mic inside its iframe and publishes to LiveKit,
so the host can't reach that track; LiveKit's Krisp filter is Cloud-only
(we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML
tier instead injects a same-origin pre-init shim into the vendored EC
index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin)
that patches getUserMedia and routes the captured mic through an RNNoise
AudioWorklet before LiveKit sees it -- the same post-capture pipeline as
#3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if
setup fails; keeps echoCancellation/AGC on the raw capture.

- settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy
  boolean migration (true->browser, false->off)
- CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends
  lotusDenoise=ml (native suppressor off in ML mode)
- vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and
  inject the shim <script> before EC's module entry
- docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-15 20:29:59 -04:00
parent f9edd2023d
commit 5deed79b42
10 changed files with 299 additions and 38 deletions
+47
View File
@@ -65,6 +65,52 @@ function copyPdfWorker() {
};
}
// Lotus ML noise suppression: ship the RNNoise worklet/wasm + our getUserMedia
// shim alongside the vendored Element Call bundle, and inject the shim <script>
// into EC's index.html so it runs before EC captures the mic. Runs after
// viteStaticCopy has populated dist/public/element-call (closeBundle order).
function lotusDenoise() {
return {
name: 'lotus-denoise',
closeBundle() {
const ecDir = path.resolve('dist/public/element-call');
if (!fs.existsSync(ecDir)) return;
const denoiseDir = path.join(ecDir, 'denoise');
fs.mkdirSync(denoiseDir, { recursive: true });
const sapphi = path.resolve('node_modules/@sapphi-red/web-noise-suppressor/dist');
const assets = [
[
path.join(sapphi, 'rnnoise/workletProcessor.js'),
path.join(denoiseDir, 'rnnoiseWorklet.js'),
],
[path.join(sapphi, 'rnnoise.wasm'), path.join(denoiseDir, 'rnnoise.wasm')],
[path.join(sapphi, 'rnnoise_simd.wasm'), path.join(denoiseDir, 'rnnoise_simd.wasm')],
];
assets.forEach(([s, d]) => {
if (fs.existsSync(s)) fs.copyFileSync(s, d);
});
const shimSrc = path.resolve('build/lotus-denoise.js');
if (fs.existsSync(shimSrc)) fs.copyFileSync(shimSrc, path.join(ecDir, 'lotus-denoise.js'));
const indexPath = path.join(ecDir, 'index.html');
if (fs.existsSync(indexPath)) {
let html = fs.readFileSync(indexPath, 'utf8');
if (!html.includes('lotus-denoise.js')) {
// Classic (non-deferred) script runs before EC's deferred module entry.
html = html.replace(
/<script type="module"/,
'<script src="./lotus-denoise.js"></script><script type="module"',
);
fs.writeFileSync(indexPath, html);
}
}
},
};
}
function serverMatrixSdkCryptoWasm(wasmFilePath) {
return {
name: 'vite-plugin-serve-matrix-sdk-crypto-wasm',
@@ -133,6 +179,7 @@ export default defineConfig({
wasm(),
react(),
copyPdfWorker(),
lotusDenoise(),
...(process.env.SENTRY_AUTH_TOKEN
? [
sentryVitePlugin({