Compare commits
1 Commits
f9edd2023d
...
5deed79b42
| Author | SHA1 | Date | |
|---|---|---|---|
| 5deed79b42 |
+25
-2
@@ -405,9 +405,32 @@ A local sound plays when another participant joins or leaves a call you're in.
|
||||
|
||||
Files: `src/app/utils/callSounds.ts`, `src/app/hooks/useCallJoinLeaveSounds.ts`
|
||||
|
||||
### Noise Suppression Toggle
|
||||
### Noise Suppression (3-Tier, incl. on-device ML) (P5-30)
|
||||
|
||||
A `noiseSuppression` URL parameter is passed to the Element Call widget URL, allowing the noise suppression feature to be toggled from within Lotus settings.
|
||||
A three-way mic noise-suppression control in **Settings → General → Calls**:
|
||||
|
||||
| Tier | What it does |
|
||||
|---|---|
|
||||
| **Off** | No suppression (`noiseSuppression=false` to Element Call). |
|
||||
| **Browser-native** | Element Call's built-in WebRTC suppressor (`noiseSuppression=true`). Default. |
|
||||
| **ML (beta)** | On-device RNNoise — Krisp-style removal of fans, keyboards, dogs, etc. |
|
||||
|
||||
**Why a shim, not a fork:** Element Call captures the mic *inside* its iframe and publishes to LiveKit; the host can't reach that track. LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU), and EC's own RNNoise work (PR #3892) is unmerged. So the **ML tier** is delivered by injecting a same-origin pre-init script into the vendored EC `index.html` that monkeypatches `getUserMedia` and routes the captured mic through an RNNoise `AudioWorklet` (`@sapphi-red/web-noise-suppressor`) before LiveKit ever sees it — the same post-capture pipeline #3892 uses, executed from the realm we already control. Works on the self-hosted LiveKit SFU, survives EC version bumps, no EC fork/AGPL/rebase burden.
|
||||
|
||||
**How it's wired:**
|
||||
|
||||
- `callNoiseSuppression` setting is `'off' | 'browser' | 'ml'` (legacy boolean migrates: `true`→`browser`, `false`→`off`)
|
||||
- `CallEmbed.getWidget()` maps the tier to the `noiseSuppression` URL param and appends `lotusDenoise=ml` for the ML tier (browser-native suppressor is disabled in ML mode so RNNoise owns suppression)
|
||||
- The `lotusDenoise` vite plugin copies the RNNoise worklet + wasm into `public/element-call/denoise/`, copies the shim, and injects `<script src="./lotus-denoise.js">` before EC's module entry
|
||||
- The shim keeps `echoCancellation`/`autoGainControl` on the raw capture and falls back to the raw mic if RNNoise setup fails, so calls never break
|
||||
|
||||
**Known beta caveat:** routing capture through WebAudio can weaken the browser's acoustic echo cancellation (AEC runs on the native capture track) — the same tradeoff EC's upstream feature makes; hence the "beta" label.
|
||||
|
||||
### Files
|
||||
|
||||
- `build/lotus-denoise.js` — injected RNNoise getUserMedia shim (classic script)
|
||||
- `vite.config.js` — `lotusDenoise()` plugin (asset copy + index.html injection)
|
||||
- `src/app/plugins/call/CallEmbed.ts` — tier → widget URL params
|
||||
|
||||
### Call Button Scoping
|
||||
|
||||
|
||||
+5
-5
@@ -243,12 +243,12 @@ Themes:
|
||||
|
||||
---
|
||||
|
||||
### [ ] P5-30 · Advanced ML Noise Suppression (Krisp-style)
|
||||
### [x] P5-30 · Advanced ML Noise Suppression (Krisp-style)
|
||||
|
||||
**What:** High-end background noise cancellation using a pre-trained ML model (e.g. RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
|
||||
**Note:** This is a top-tier feature request and an EXTREME COMPLEXITY project.
|
||||
**[AUDIT REQUIRED]** Must verify if mixing a processed stream into Element Call's WebRTC implementation causes latency or AEC (Echo Cancellation) issues.
|
||||
**Complexity:** Extreme.
|
||||
**What:** High-end background noise cancellation using a pre-trained ML model (RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
|
||||
**Shipped:** 3-tier setting (Off / Browser-native / ML beta) in Settings → General → Calls. ML tier injects a same-origin pre-init shim into the vendored Element Call `index.html` that monkeypatches `getUserMedia` and routes the captured mic through an RNNoise `AudioWorklet` (`@sapphi-red/web-noise-suppressor`) before LiveKit publishes — no EC fork required. See LOTUS_FEATURES.md → "Noise Suppression (3-Tier)".
|
||||
**Key decision:** LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU); EC's own RNNoise PR #3892 is unmerged. The shim is the same post-capture pipeline #3892 uses, executed from the realm we control, so it survives EC version bumps.
|
||||
**AEC note (resolved-as-accepted):** WebAudio capture routing can weaken browser AEC (AEC runs on the native track) — same tradeoff as EC's upstream feature; mitigated by keeping `echoCancellation`/`autoGainControl` on the raw capture and labeling the tier "beta". Validate echo quality on real multi-party calls after deploy.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
/*
|
||||
* Lotus Chat — client-side ML noise suppression shim for Element Call.
|
||||
*
|
||||
* Element Call runs as a same-origin iframe widget that captures the mic
|
||||
* internally (via livekit-client -> getUserMedia) and publishes it to LiveKit.
|
||||
* We can't reach that track from the host. Instead this classic <script> is
|
||||
* injected (by the vite `lotus-denoise` plugin) into EC's index.html BEFORE its
|
||||
* deferred module entry, so it runs first and monkeypatches getUserMedia. When
|
||||
* the "ml" tier is selected (lotusDenoise=ml in the widget URL) we route the
|
||||
* captured mic through an RNNoise AudioWorklet (@sapphi-red/web-noise-suppressor)
|
||||
* and hand the processed track back to EC/LiveKit.
|
||||
*
|
||||
* This mirrors Element Call's own (still-unmerged) PR #3892 pipeline, executed
|
||||
* from the realm we already control instead of forking and rebuilding EC.
|
||||
*
|
||||
* Known beta caveat: routing capture through WebAudio can weaken the browser's
|
||||
* acoustic echo cancellation (AEC operates on the native capture track). We keep
|
||||
* echoCancellation/autoGainControl enabled on the raw capture to mitigate.
|
||||
*/
|
||||
(function () {
|
||||
'use strict';
|
||||
|
||||
try {
|
||||
var params = new URLSearchParams(window.location.search);
|
||||
if (params.get('lotusDenoise') !== 'ml') return;
|
||||
} catch (e) {
|
||||
return;
|
||||
}
|
||||
|
||||
var md = navigator.mediaDevices;
|
||||
if (!md || typeof md.getUserMedia !== 'function') return;
|
||||
if (typeof AudioWorkletNode === 'undefined' || typeof AudioContext === 'undefined') return;
|
||||
|
||||
var PROCESSOR_NAME = '@sapphi-red/web-noise-suppressor/rnnoise';
|
||||
var ASSET_BASE = './denoise/';
|
||||
var SAMPLE_RATE = 48000; // RNNoise worklet assumes 48kHz
|
||||
|
||||
var origGetUserMedia = md.getUserMedia.bind(md);
|
||||
var wasmPromise = null;
|
||||
|
||||
// SIMD feature detection (bytes from @sapphi-red/web-noise-suppressor / wasm-feature-detect)
|
||||
function hasSimd() {
|
||||
try {
|
||||
return WebAssembly.validate(
|
||||
new Uint8Array([
|
||||
0, 97, 115, 109, 1, 0, 0, 0, 1, 5, 1, 96, 0, 1, 123, 3, 2, 1, 0, 10, 10, 1, 8, 0, 65, 0,
|
||||
253, 15, 253, 98, 11,
|
||||
])
|
||||
);
|
||||
} catch (e) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function loadWasm() {
|
||||
if (!wasmPromise) {
|
||||
var url = ASSET_BASE + (hasSimd() ? 'rnnoise_simd.wasm' : 'rnnoise.wasm');
|
||||
wasmPromise = fetch(url).then(function (r) {
|
||||
if (!r.ok) throw new Error('rnnoise wasm fetch failed: ' + r.status);
|
||||
return r.arrayBuffer();
|
||||
});
|
||||
}
|
||||
return wasmPromise;
|
||||
}
|
||||
|
||||
function processStream(stream) {
|
||||
var audioTracks = stream.getAudioTracks();
|
||||
if (audioTracks.length === 0) return Promise.resolve(stream);
|
||||
|
||||
return loadWasm()
|
||||
.then(function (wasmBinary) {
|
||||
var ctx = new AudioContext({ sampleRate: SAMPLE_RATE });
|
||||
return ctx.audioWorklet
|
||||
.addModule(ASSET_BASE + 'rnnoiseWorklet.js')
|
||||
.then(function () {
|
||||
if (ctx.state === 'suspended') return ctx.resume().then(function () { return ctx; });
|
||||
return ctx;
|
||||
})
|
||||
.then(function () {
|
||||
var node = new AudioWorkletNode(ctx, PROCESSOR_NAME, {
|
||||
processorOptions: { maxChannels: 1, wasmBinary: wasmBinary },
|
||||
});
|
||||
var source = ctx.createMediaStreamSource(stream);
|
||||
var dest = ctx.createMediaStreamDestination();
|
||||
source.connect(node).connect(dest);
|
||||
|
||||
var origTrack = audioTracks[0];
|
||||
var processedTrack = dest.stream.getAudioTracks()[0];
|
||||
|
||||
var torndown = false;
|
||||
function cleanup() {
|
||||
if (torndown) return;
|
||||
torndown = true;
|
||||
try {
|
||||
node.port.postMessage('destroy');
|
||||
} catch (e) {}
|
||||
try {
|
||||
source.disconnect();
|
||||
node.disconnect();
|
||||
} catch (e) {}
|
||||
try {
|
||||
origTrack.stop();
|
||||
} catch (e) {}
|
||||
try {
|
||||
ctx.close();
|
||||
} catch (e) {}
|
||||
}
|
||||
|
||||
// When EC stops the track we handed it, release the raw capture + graph.
|
||||
var rawStop = processedTrack.stop.bind(processedTrack);
|
||||
processedTrack.stop = function () {
|
||||
cleanup();
|
||||
rawStop();
|
||||
};
|
||||
// Device unplugged / capture ended involuntarily.
|
||||
origTrack.addEventListener('ended', function () {
|
||||
try {
|
||||
rawStop();
|
||||
} catch (e) {}
|
||||
cleanup();
|
||||
});
|
||||
|
||||
// Return a stream with the processed audio plus any original video.
|
||||
var out = new MediaStream();
|
||||
out.addTrack(processedTrack);
|
||||
stream.getVideoTracks().forEach(function (t) {
|
||||
out.addTrack(t);
|
||||
});
|
||||
return out;
|
||||
});
|
||||
})
|
||||
.catch(function (e) {
|
||||
// Any failure -> fall back to the raw mic so calls never break.
|
||||
// eslint-disable-next-line no-console
|
||||
console.error('[lotus-denoise] RNNoise setup failed, using raw mic', e);
|
||||
return stream;
|
||||
});
|
||||
}
|
||||
|
||||
navigator.mediaDevices.getUserMedia = function (constraints) {
|
||||
var wantsAudio = !!(constraints && constraints.audio);
|
||||
var effective = constraints;
|
||||
if (wantsAudio) {
|
||||
// RNNoise owns noise suppression; keep AEC + AGC on the raw capture.
|
||||
var audioC = typeof constraints.audio === 'object' ? Object.assign({}, constraints.audio) : {};
|
||||
audioC.noiseSuppression = false;
|
||||
if (audioC.echoCancellation === undefined) audioC.echoCancellation = true;
|
||||
if (audioC.autoGainControl === undefined) audioC.autoGainControl = true;
|
||||
effective = Object.assign({}, constraints, { audio: audioC });
|
||||
}
|
||||
return origGetUserMedia(effective).then(function (stream) {
|
||||
return wantsAudio ? processStream(stream) : stream;
|
||||
});
|
||||
};
|
||||
})();
|
||||
Generated
+7
@@ -20,6 +20,7 @@
|
||||
"@giphy/js-types": "5.1.0",
|
||||
"@giphy/js-util": "5.2.0",
|
||||
"@giphy/react-components": "10.1.2",
|
||||
"@sapphi-red/web-noise-suppressor": "0.3.5",
|
||||
"@sentry/react": "10.53.1",
|
||||
"@tanstack/react-query": "5.100.13",
|
||||
"@tanstack/react-query-devtools": "5.100.13",
|
||||
@@ -3774,6 +3775,12 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@sapphi-red/web-noise-suppressor": {
|
||||
"version": "0.3.5",
|
||||
"resolved": "https://registry.npmjs.org/@sapphi-red/web-noise-suppressor/-/web-noise-suppressor-0.3.5.tgz",
|
||||
"integrity": "sha512-jh3+V9yM+zxLriQexoGm0GatoPaJWjs6ypFIbFYwQp+AoUb55eUXrjKtKQyuC5zShzzeAQUl0M5JzqB7SSrsRA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@sentry-internal/browser-utils": {
|
||||
"version": "10.53.1",
|
||||
"resolved": "https://registry.npmjs.org/@sentry-internal/browser-utils/-/browser-utils-10.53.1.tgz",
|
||||
|
||||
@@ -44,6 +44,7 @@
|
||||
"@giphy/js-types": "5.1.0",
|
||||
"@giphy/js-util": "5.2.0",
|
||||
"@giphy/react-components": "10.1.2",
|
||||
"@sapphi-red/web-noise-suppressor": "0.3.5",
|
||||
"@sentry/react": "10.53.1",
|
||||
"@tanstack/react-query": "5.100.13",
|
||||
"@tanstack/react-query-devtools": "5.100.13",
|
||||
|
||||
@@ -42,13 +42,11 @@ import {
|
||||
DateFormat,
|
||||
MessageLayout,
|
||||
MessageSpacing,
|
||||
NoiseSuppressionMode,
|
||||
Settings,
|
||||
settingsAtom,
|
||||
} from '../../../state/settings';
|
||||
import {
|
||||
SeasonalPreview,
|
||||
SeasonTheme,
|
||||
} from '../../../components/seasonal/SeasonalEffect';
|
||||
import { SeasonalPreview, SeasonTheme } from '../../../components/seasonal/SeasonalEffect';
|
||||
import { SettingTile } from '../../../components/setting-tile';
|
||||
import { KeySymbol } from '../../../utils/key-symbol';
|
||||
import { isMacOS } from '../../../utils/user-agent';
|
||||
@@ -1235,12 +1233,16 @@ function Calls() {
|
||||
<SequenceCard className={SequenceCardStyle} variant="SurfaceVariant" direction="Column">
|
||||
<SettingTile
|
||||
title="Noise Suppression"
|
||||
description="Apply AI noise suppression to filter background noise during calls (powered by Element Call)."
|
||||
description="Filter background noise from your mic during calls. Browser-native uses the built-in WebRTC suppressor; ML runs on-device RNNoise for stronger, Krisp-style removal (higher CPU)."
|
||||
after={
|
||||
<Switch
|
||||
variant="Primary"
|
||||
<SettingsSelect<NoiseSuppressionMode>
|
||||
value={callNoiseSuppression}
|
||||
onChange={setCallNoiseSuppression}
|
||||
options={[
|
||||
{ value: 'off', label: 'Off' },
|
||||
{ value: 'browser', label: 'Browser-native' },
|
||||
{ value: 'ml', label: 'ML (beta)' },
|
||||
]}
|
||||
/>
|
||||
}
|
||||
/>
|
||||
@@ -1346,22 +1348,25 @@ function Calls() {
|
||||
);
|
||||
}
|
||||
|
||||
const SEASONAL_OPTIONS: { value: Settings['seasonalThemeOverride']; label: string; emoji: string }[] =
|
||||
[
|
||||
{ value: 'auto', label: 'Auto', emoji: '🗓' },
|
||||
{ value: 'off', label: 'Off', emoji: '×' },
|
||||
{ value: 'newyear', label: 'New Year', emoji: '🎆' },
|
||||
{ value: 'lunar', label: 'Lunar New Year', emoji: '🏮' },
|
||||
{ value: 'valentines', label: "Valentine's", emoji: '💖' },
|
||||
{ value: 'stpatricks', label: "St. Patrick's", emoji: '🍀' },
|
||||
{ value: 'aprilfools', label: 'April Fools', emoji: '?' },
|
||||
{ value: 'earthday', label: 'Earth Day', emoji: '🌱' },
|
||||
{ value: 'autumn', label: 'Autumn', emoji: '🍂' },
|
||||
{ value: 'halloween', label: 'Halloween', emoji: '🎃' },
|
||||
{ value: 'christmas', label: 'Christmas', emoji: '❄️' },
|
||||
{ value: 'arcade', label: 'Arcade Day', emoji: '👾' },
|
||||
{ value: 'deepspace', label: 'Deep Space', emoji: '🚀' },
|
||||
];
|
||||
const SEASONAL_OPTIONS: {
|
||||
value: Settings['seasonalThemeOverride'];
|
||||
label: string;
|
||||
emoji: string;
|
||||
}[] = [
|
||||
{ value: 'auto', label: 'Auto', emoji: '🗓' },
|
||||
{ value: 'off', label: 'Off', emoji: '×' },
|
||||
{ value: 'newyear', label: 'New Year', emoji: '🎆' },
|
||||
{ value: 'lunar', label: 'Lunar New Year', emoji: '🏮' },
|
||||
{ value: 'valentines', label: "Valentine's", emoji: '💖' },
|
||||
{ value: 'stpatricks', label: "St. Patrick's", emoji: '🍀' },
|
||||
{ value: 'aprilfools', label: 'April Fools', emoji: '?' },
|
||||
{ value: 'earthday', label: 'Earth Day', emoji: '🌱' },
|
||||
{ value: 'autumn', label: 'Autumn', emoji: '🍂' },
|
||||
{ value: 'halloween', label: 'Halloween', emoji: '🎃' },
|
||||
{ value: 'christmas', label: 'Christmas', emoji: '❄️' },
|
||||
{ value: 'arcade', label: 'Arcade Day', emoji: '👾' },
|
||||
{ value: 'deepspace', label: 'Deep Space', emoji: '🚀' },
|
||||
];
|
||||
|
||||
function SeasonalBgGrid({
|
||||
value,
|
||||
|
||||
@@ -15,7 +15,7 @@ import { CallControlState } from '../plugins/call/CallControlState';
|
||||
import { useCallMembersChange, useCallSession } from './useCall';
|
||||
import { CallPreferences } from '../state/callPreferences';
|
||||
import { useSetting } from '../state/hooks/settings';
|
||||
import { settingsAtom } from '../state/settings';
|
||||
import { NoiseSuppressionMode, settingsAtom } from '../state/settings';
|
||||
import { unlockCallSounds } from '../utils/callSounds';
|
||||
|
||||
const CallEmbedContext = createContext<CallEmbed | undefined>(undefined);
|
||||
@@ -45,7 +45,7 @@ export const createCallEmbed = (
|
||||
themeKind: ElementCallThemeKind,
|
||||
container: HTMLElement,
|
||||
pref?: CallPreferences,
|
||||
noiseSuppression = true,
|
||||
denoiseMode: NoiseSuppressionMode = 'browser',
|
||||
forceAudioOff = false,
|
||||
): CallEmbed => {
|
||||
const rtcSession = mx.matrixRTC.getRoomSession(room);
|
||||
@@ -59,7 +59,7 @@ export const createCallEmbed = (
|
||||
room,
|
||||
intent,
|
||||
themeKind,
|
||||
noiseSuppression,
|
||||
denoiseMode,
|
||||
initialAudio,
|
||||
initialVideo,
|
||||
);
|
||||
@@ -96,7 +96,7 @@ export const useCallStart = (dm = false) => {
|
||||
theme.kind,
|
||||
container,
|
||||
pref,
|
||||
callNoiseSuppression ?? true,
|
||||
callNoiseSuppression ?? 'browser',
|
||||
!!pttMode,
|
||||
);
|
||||
|
||||
|
||||
@@ -18,6 +18,7 @@ import {
|
||||
} from 'matrix-widget-api';
|
||||
import { CallWidgetDriver } from './CallWidgetDriver';
|
||||
import { trimTrailingSlash } from '../../utils/common';
|
||||
import { NoiseSuppressionMode } from '../../state/settings';
|
||||
import {
|
||||
ElementCallIntent,
|
||||
ElementCallThemeKind,
|
||||
@@ -100,7 +101,7 @@ export class CallEmbed {
|
||||
room: Room,
|
||||
intent: ElementCallIntent,
|
||||
themeKind: ElementCallThemeKind,
|
||||
noiseSuppression = true,
|
||||
denoiseMode: NoiseSuppressionMode = 'browser',
|
||||
initialAudio = true,
|
||||
initialVideo = false,
|
||||
): Widget {
|
||||
@@ -124,12 +125,21 @@ export class CallEmbed {
|
||||
perParticipantE2EE: room.hasEncryptionStateEvent().toString(),
|
||||
lang: 'en-EN',
|
||||
theme: themeKind,
|
||||
noiseSuppression: noiseSuppression.toString(),
|
||||
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml' we
|
||||
// disable it here so RNNoise (the Lotus denoise shim) owns suppression and
|
||||
// the two don't fight each other.
|
||||
noiseSuppression: (denoiseMode === 'browser').toString(),
|
||||
audio: initialAudio.toString(),
|
||||
video: initialVideo.toString(),
|
||||
header: 'none',
|
||||
});
|
||||
|
||||
if (denoiseMode === 'ml') {
|
||||
// Signal the Lotus denoise shim (injected into the EC index.html) to route
|
||||
// the mic through the RNNoise worklet before LiveKit publishes the track.
|
||||
params.append('lotusDenoise', 'ml');
|
||||
}
|
||||
|
||||
if (CallEmbed.startingCall(intent)) {
|
||||
params.append('sendNotificationType', CallEmbed.dmCall(intent) ? 'ring' : 'notification');
|
||||
}
|
||||
|
||||
@@ -9,6 +9,11 @@ export type DateFormat =
|
||||
| 'YYYY-MM-DD'
|
||||
| '';
|
||||
export type MessageSpacing = '0' | '100' | '200' | '300' | '400' | '500';
|
||||
// Call mic noise suppression tier:
|
||||
// - 'off' : no suppression
|
||||
// - 'browser' : WebRTC built-in suppression (Element Call noiseSuppression param)
|
||||
// - 'ml' : client-side RNNoise ML suppression (Lotus denoise shim)
|
||||
export type NoiseSuppressionMode = 'off' | 'browser' | 'ml';
|
||||
export type ChatBackground =
|
||||
| 'none'
|
||||
| 'blueprint'
|
||||
@@ -109,7 +114,7 @@ export interface Settings {
|
||||
perMessageProfiles: boolean;
|
||||
|
||||
cameraOnJoin: boolean;
|
||||
callNoiseSuppression: boolean;
|
||||
callNoiseSuppression: NoiseSuppressionMode;
|
||||
pttMode: boolean;
|
||||
pttKey: string;
|
||||
|
||||
@@ -199,7 +204,7 @@ const defaultSettings: Settings = {
|
||||
perMessageProfiles: false,
|
||||
|
||||
cameraOnJoin: false,
|
||||
callNoiseSuppression: true,
|
||||
callNoiseSuppression: 'browser',
|
||||
pttMode: false,
|
||||
pttKey: 'Space',
|
||||
|
||||
@@ -235,6 +240,14 @@ export const getSettings = (): Settings => {
|
||||
return {
|
||||
...defaultSettings,
|
||||
...saved,
|
||||
// Migrate legacy boolean callNoiseSuppression -> 3-way mode:
|
||||
// true => browser-native, false => off. New string values pass through.
|
||||
callNoiseSuppression:
|
||||
typeof saved.callNoiseSuppression === 'boolean'
|
||||
? saved.callNoiseSuppression
|
||||
? 'browser'
|
||||
: 'off'
|
||||
: (saved.callNoiseSuppression ?? defaultSettings.callNoiseSuppression),
|
||||
composerToolbarButtons: {
|
||||
...DEFAULT_COMPOSER_TOOLBAR,
|
||||
...(saved.composerToolbarButtons ?? {}),
|
||||
|
||||
@@ -65,6 +65,52 @@ function copyPdfWorker() {
|
||||
};
|
||||
}
|
||||
|
||||
// Lotus ML noise suppression: ship the RNNoise worklet/wasm + our getUserMedia
|
||||
// shim alongside the vendored Element Call bundle, and inject the shim <script>
|
||||
// into EC's index.html so it runs before EC captures the mic. Runs after
|
||||
// viteStaticCopy has populated dist/public/element-call (closeBundle order).
|
||||
function lotusDenoise() {
|
||||
return {
|
||||
name: 'lotus-denoise',
|
||||
closeBundle() {
|
||||
const ecDir = path.resolve('dist/public/element-call');
|
||||
if (!fs.existsSync(ecDir)) return;
|
||||
|
||||
const denoiseDir = path.join(ecDir, 'denoise');
|
||||
fs.mkdirSync(denoiseDir, { recursive: true });
|
||||
|
||||
const sapphi = path.resolve('node_modules/@sapphi-red/web-noise-suppressor/dist');
|
||||
const assets = [
|
||||
[
|
||||
path.join(sapphi, 'rnnoise/workletProcessor.js'),
|
||||
path.join(denoiseDir, 'rnnoiseWorklet.js'),
|
||||
],
|
||||
[path.join(sapphi, 'rnnoise.wasm'), path.join(denoiseDir, 'rnnoise.wasm')],
|
||||
[path.join(sapphi, 'rnnoise_simd.wasm'), path.join(denoiseDir, 'rnnoise_simd.wasm')],
|
||||
];
|
||||
assets.forEach(([s, d]) => {
|
||||
if (fs.existsSync(s)) fs.copyFileSync(s, d);
|
||||
});
|
||||
|
||||
const shimSrc = path.resolve('build/lotus-denoise.js');
|
||||
if (fs.existsSync(shimSrc)) fs.copyFileSync(shimSrc, path.join(ecDir, 'lotus-denoise.js'));
|
||||
|
||||
const indexPath = path.join(ecDir, 'index.html');
|
||||
if (fs.existsSync(indexPath)) {
|
||||
let html = fs.readFileSync(indexPath, 'utf8');
|
||||
if (!html.includes('lotus-denoise.js')) {
|
||||
// Classic (non-deferred) script runs before EC's deferred module entry.
|
||||
html = html.replace(
|
||||
/<script type="module"/,
|
||||
'<script src="./lotus-denoise.js"></script><script type="module"',
|
||||
);
|
||||
fs.writeFileSync(indexPath, html);
|
||||
}
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function serverMatrixSdkCryptoWasm(wasmFilePath) {
|
||||
return {
|
||||
name: 'vite-plugin-serve-matrix-sdk-crypto-wasm',
|
||||
@@ -133,6 +179,7 @@ export default defineConfig({
|
||||
wasm(),
|
||||
react(),
|
||||
copyPdfWorker(),
|
||||
lotusDenoise(),
|
||||
...(process.env.SENTRY_AUTH_TOKEN
|
||||
? [
|
||||
sentryVitePlugin({
|
||||
|
||||
Reference in New Issue
Block a user