- Model dropdown is now ordered by quality/CPU, best first (DeepFilterNet 3 →
DTLN → RNNoise → Speex); fix RNNoise's inaccurate "High" voice-quality label.
- When a user opts into the ML tier, default to the highest-quality model
(DeepFilterNet 3). The tier default stays browser-native (known-good, best
perceived in testing so far).
- Wire the "Series Suppression" (native-NS-before-ML) toggle into the real call
path — it was applied only in the settings tester, so the tester could sound
better than the actual call. Default it OFF (a single NS stage is best
practice; it's an opt-in test aid).
- isMLDenoiseSupported now also requires WebAssembly, so ML isn't offered on
strict-CSP shells where it would silently fall back to the raw mic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- scheduledMessages.test.ts (9): pins the MSC4140 request shape (PUT to the room
send endpoint with the org.matrix.msc4140.delay query, POST cancel/restart to
/delayed_events with the unstable prefix), the delay-floor math (Math.max(1000,
round(sendAt-now)) — "now"/past targets still yield a valid >=1000ms delay),
rounding, and url-encoding.
- lotusDenoiseUtils.test.ts (9): model-catalog data integrity + isMLDenoiseSupported
feature detection across AudioContext/webkit/getUserMedia.
- Bug found + fixed: isMLDenoiseSupported used `!!AudioWorkletNode`, a bare global
reference that throws ReferenceError (not returns false) on a browser with
AudioContext but no AudioWorkletNode binding. Switched to `typeof` so the
detection helper reports unsupported instead of throwing. Regression test proven
to fail on the old code.
Suite now 545 tests (4th real bug caught by the prevention work).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move the model comparison out of the always-visible Noise Suppression
description and into the ML-only sub-settings. Add a compact info card
for the selected model (CPU / voice quality / transients / download) plus
a collapsible 4-model comparison. Group ML sub-settings into Model,
Enhancements, and Test & calibrate sections with clear labels and
separators. Fix invented --lt-border-color token and hardcoded
rgba background to real TDS tokens. Build the model dropdown and
DenoiseTester labels/compare buttons from DENOISE_MODELS so
DeepFilterNet 3 is handled correctly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Integrate DeepFilterNet 3 (deepfilternet3-noise-filter@1.2.1) as a new
client-side denoise model id 'deepfilternet', mirroring the DTLN pattern.
The npm package ships only an ESM whose AudioWorklet processor + wasm-bindgen
glue are inlined as a string (loaded via a Blob URL — no CDN for the worklet).
Its only runtime fetches are a single-threaded df_bg.wasm and an ONNX model
tarball, which previously loaded from an external CDN. We now VENDOR both
(build/denoise-vendor/deepfilternet/v2/...) and self-host them under
denoise/deepfilternet/, overriding the package's cdnUrl so nothing hits the
upstream CDN — keeping it self-hosted / Tauri-CSP safe.
The wasm is single-threaded (no SharedArrayBuffer / atomics / imported shared
memory), so it needs no COOP/COEP cross-origin isolation and runs fine in EC's
non-isolated iframe. Runs at 48 kHz fullband. Any init/runtime failure falls
back to the raw mic, like the other models.
- vite.config.js: copy ESM + vendored wasm/model into the EC denoise dir with a
required-asset guard that aborts the build if any entry is missing.
- build/lotus-denoise.js: 'deepfilternet' branch — dynamic-import the ESM, build
a DeepFilterNet3Core pointed at the self-hosted base, await init, return the
worklet node; 48 kHz; raw-mic fail-safe preserved.
- denoisePipeline.ts: 'deepfilternet' branch for the in-app tester + sampleRate.
- settings.ts: add 'deepfilternet' to DenoiseModelId + getSettings whitelist.
- lotusDenoiseUtils.ts: add the comparison-chart row.
- General.tsx: add the "DeepFilterNet 3 (beta)" dropdown option.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The prior DTLN attempt (89a2321d) broke the build (missing dep, wrong
`cinny/` asset paths) and typecheck (`'dtln'` not in DenoiseModelId), and was
wired against an API the package doesn't expose. @workadventure/noise-
suppression is not a flat AudioWorklet — it's a self-contained ES module whose
processor name is `workadventure-noise-suppression` and which resolves its own
LiteRT WASM + TFLite models via import.meta.url. Driving it by hand-rolled
addModule + processorOptions cannot work.
- Re-add @workadventure/noise-suppression@0.0.4 (package.json + lockfile).
- vite: copy the package's whole dist/ tree intact to
denoise/workadventure/ (preserving assets/ + vendor/litert) so import.meta
resolution works at runtime; fail the build if the entry module is missing.
- shim: for the DTLN model, dynamic-import denoise/workadventure/audio-worklet
.js and use createNoiseSuppressionAudioWorklet(ctx, { bypassUntilReady })
to build the node; RNNoise/Speex keep their direct flat-worklet path. Async
init errors are logged + reported and fall back to the raw mic.
- Restore 'dtln' in DenoiseModelId (+ settings coercion), the model chart, and
the settings dropdown, labelled "(beta)".
DTLN builds and is fully self-hosted, but its in-call audio is UNVERIFIED in
this environment — needs a real-call test. DeepFilterNet stays excluded (CDN
asset loading, incompatible with self-hosting / Tauri CSP).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audit/repair of the multi-model denoise work so it actually builds and only
exposes working, self-hosted models.
- Complete the DTLN/DFN3 revert: uninstall @workadventure/noise-suppression
and deepfilternet3-noise-filter (package.json + lockfile), drop the unused
DTLN asset-copy block from vite.config.js (was shipping ~2MB of unused
tflite/wasm), and narrow DenoiseModelId to the bundled models (rnnoise,
speex). Coerce any retired persisted model value back to the default.
- Fix General.tsx CI typecheck failures introduced by the denoise UI: restore
three imports the rewrite deleted (useDateFormatItems, SequenceCardStyle,
useTauriUpdater), add the missing denoise/sound imports, and correct
hallucinated Folds props (Text has no variant/bold; Box uses
alignItems/justifyContent). tsc now passes with 0 errors.
- Harden the vite denoise plugin: required RNNoise/Speex/gate assets and the
shim now fail the build loudly if missing (instead of a silent warn that
shipped a broken ML feature), and the index.html shim injection is verified.
- CI: move the cinny-desktop submodule bump into ci.yml as a `trigger-desktop`
job gated on `needs: build`, and delete the standalone trigger-desktop.yml.
A failing push no longer kicks off the slow Tauri builds in parallel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implement a flexible, multi-model noise suppression pipeline for Element Call/LiveKit integration:
- ML Engines: Added support for RNNoise, Speex, DTLN, and DeepFilterNet 3 models.
- Pipeline Architecture: Implemented modular audio processing in lotus-denoise.js, supporting 'Series Suppression' (running browser-native NSNet2 before ML) and a hardware-style Noise Gate.
- UI & UX Enhancements:
- Settings UI: Added model comparison chart with CPU/Quality metadata.
- Tuning: Added Live Microphone Meter for calibrating Noise Gate thresholds.
- Reporting: Added LotusToast system to alert users when ML suppression fails or falls back to raw input.
- Robustness & Quality:
- Capture Fidelity: Removed forced 48kHz capture constraints to allow native-rate capture (solving static issues with high-end audio interfaces).
- Performance: Added WASM SIMD detection with transparent fallback.
- Capability Detection: Added browser feature detection to disable unsupported ML modes.
- Build Integration: Updated Vite config to self-host all model WASM/tflite assets in /denoise/ directory.
jaredandClaude Opus 4.8