5 Commits

Include renames

Author	SHA1	Message	Date
jaredandClaude Opus 4.8	cf7c66b99a	Reorganize ML noise suppression settings UI ... Move the model comparison out of the always-visible Noise Suppression description and into the ML-only sub-settings. Add a compact info card for the selected model (CPU / voice quality / transients / download) plus a collapsible 4-model comparison. Group ML sub-settings into Model, Enhancements, and Test & calibrate sections with clear labels and separators. Fix invented --lt-border-color token and hardcoded rgba background to real TDS tokens. Build the model dropdown and DenoiseTester labels/compare buttons from DENOISE_MODELS so DeepFilterNet 3 is handled correctly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-17 20:02:16 -04:00
jaredandClaude Opus 4.8	04b56ffacd	feat(denoise): add self-hosted DeepFilterNet 3 ML noise-suppression model ... Integrate DeepFilterNet 3 (deepfilternet3-noise-filter@1.2.1) as a new client-side denoise model id 'deepfilternet', mirroring the DTLN pattern. The npm package ships only an ESM whose AudioWorklet processor + wasm-bindgen glue are inlined as a string (loaded via a Blob URL — no CDN for the worklet). Its only runtime fetches are a single-threaded df_bg.wasm and an ONNX model tarball, which previously loaded from an external CDN. We now VENDOR both (build/denoise-vendor/deepfilternet/v2/...) and self-host them under denoise/deepfilternet/, overriding the package's cdnUrl so nothing hits the upstream CDN — keeping it self-hosted / Tauri-CSP safe. The wasm is single-threaded (no SharedArrayBuffer / atomics / imported shared memory), so it needs no COOP/COEP cross-origin isolation and runs fine in EC's non-isolated iframe. Runs at 48 kHz fullband. Any init/runtime failure falls back to the raw mic, like the other models. - vite.config.js: copy ESM + vendored wasm/model into the EC denoise dir with a required-asset guard that aborts the build if any entry is missing. - build/lotus-denoise.js: 'deepfilternet' branch — dynamic-import the ESM, build a DeepFilterNet3Core pointed at the self-hosted base, await init, return the worklet node; 48 kHz; raw-mic fail-safe preserved. - denoisePipeline.ts: 'deepfilternet' branch for the in-app tester + sampleRate. - settings.ts: add 'deepfilternet' to DenoiseModelId + getSettings whitelist. - lotusDenoiseUtils.ts: add the comparison-chart row. - General.tsx: add the "DeepFilterNet 3 (beta)" dropdown option. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-17 19:57:08 -04:00
jaredandClaude Opus 4.8	86272b6b08	fix(calls): wire DTLN ML denoise correctly via @workadventure JS API ... The prior DTLN attempt (89a2321d) broke the build (missing dep, wrong `cinny/` asset paths) and typecheck (`'dtln'` not in DenoiseModelId), and was wired against an API the package doesn't expose. @workadventure/noise- suppression is not a flat AudioWorklet — it's a self-contained ES module whose processor name is `workadventure-noise-suppression` and which resolves its own LiteRT WASM + TFLite models via import.meta.url. Driving it by hand-rolled addModule + processorOptions cannot work. - Re-add @workadventure/noise-suppression@0.0.4 (package.json + lockfile). - vite: copy the package's whole dist/ tree intact to denoise/workadventure/ (preserving assets/ + vendor/litert) so import.meta resolution works at runtime; fail the build if the entry module is missing. - shim: for the DTLN model, dynamic-import denoise/workadventure/audio-worklet .js and use createNoiseSuppressionAudioWorklet(ctx, { bypassUntilReady }) to build the node; RNNoise/Speex keep their direct flat-worklet path. Async init errors are logged + reported and fall back to the raw mic. - Restore 'dtln' in DenoiseModelId (+ settings coercion), the model chart, and the settings dropdown, labelled "(beta)". DTLN builds and is fully self-hosted, but its in-call audio is UNVERIFIED in this environment — needs a real-call test. DeepFilterNet stays excluded (CDN asset loading, incompatible with self-hosting / Tauri CSP). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 17:11:45 -04:00
jaredandClaude Opus 4.8	6634b2b8a2	fix(calls): make ML denoise build-honest + gate desktop trigger on CI ... Audit/repair of the multi-model denoise work so it actually builds and only exposes working, self-hosted models. - Complete the DTLN/DFN3 revert: uninstall @workadventure/noise-suppression and deepfilternet3-noise-filter (package.json + lockfile), drop the unused DTLN asset-copy block from vite.config.js (was shipping ~2MB of unused tflite/wasm), and narrow DenoiseModelId to the bundled models (rnnoise, speex). Coerce any retired persisted model value back to the default. - Fix General.tsx CI typecheck failures introduced by the denoise UI: restore three imports the rewrite deleted (useDateFormatItems, SequenceCardStyle, useTauriUpdater), add the missing denoise/sound imports, and correct hallucinated Folds props (Text has no variant/bold; Box uses alignItems/justifyContent). tsc now passes with 0 errors. - Harden the vite denoise plugin: required RNNoise/Speex/gate assets and the shim now fail the build loudly if missing (instead of a silent warn that shipped a broken ML feature), and the index.html shim injection is verified. - CI: move the cinny-desktop submodule bump into ci.yml as a `trigger-desktop` job gated on `needs: build`, and delete the standalone trigger-desktop.yml. A failing push no longer kicks off the slow Tauri builds in parallel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-16 01:42:21 -04:00
jared	5d5f5f4516	feat(calls): implement advanced multi-model ML noise suppression system ... Implement a flexible, multi-model noise suppression pipeline for Element Call/LiveKit integration: - ML Engines: Added support for RNNoise, Speex, DTLN, and DeepFilterNet 3 models. - Pipeline Architecture: Implemented modular audio processing in lotus-denoise.js, supporting 'Series Suppression' (running browser-native NSNet2 before ML) and a hardware-style Noise Gate. - UI & UX Enhancements: - Settings UI: Added model comparison chart with CPU/Quality metadata. - Tuning: Added Live Microphone Meter for calibrating Noise Gate thresholds. - Reporting: Added LotusToast system to alert users when ML suppression fails or falls back to raw input. - Robustness & Quality: - Capture Fidelity: Removed forced 48kHz capture constraints to allow native-rate capture (solving static issues with high-end audio interfaces). - Performance: Added WASM SIMD detection with transparent fallback. - Capability Detection: Added browser feature detection to disable unsupported ML modes. - Build Integration: Updated Vite config to self-host all model WASM/tflite assets in /denoise/ directory.	2026-06-16 00:50:12 -04:00