LotusGuild/cinny
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(denoise): autoGainControl=false for the ML tier + docs
CI / Build & Quality Checks (push) Successful in 11m16s
Details
CI / Trigger Desktop Build (push) Successful in 10s
Details

Browse Source 
- CallEmbed sets `autoGainControl=false` for the ML noise-suppression tier so
  the browser's auto gain control doesn't fight the in-source ML model; the
  browser/off tiers keep AGC on.
- Docs: refresh the LOTUS_FEATURES noise-suppression section (browser-native
  default, quality-ordered dropdown, DFN3 ML default, attenuation floor,
  gate-after-ML, DFN level 60, AGC-off, the reliability fixes) and LOTUS_TODO
  P5-30 (mark tuning/reliability/AGC done; record GTCRN as researched-and-deferred).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jared Vititoe
2026-07-01 00:46:39 -04:00
parent ebc782b16c
commit a0fcdf74da
3 changed files with 35 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LOTUS_TODO.md
+6 -2
View File
@@ -301,8 +301,12 @@ Features:
**Models — all in-source in the fork:**
- [x] **RNNoise** (48 kHz, default) · **Speex** (48 kHz) · **DTLN** (16 kHz) · **DeepFilterNet 3** (48 kHz) — all four wired and selectable.
- [ ] **Open verification:** real-call **audio-quality** comparison across the four models (RNNoise output is known-weak). Track under the denoise quality project, `LOTUS_TESTING.md` §D2-1 / J2.
- [x] **DeepFilterNet 3** (48 kHz, **ML default**) · **DTLN** (16 kHz) · **RNNoise** (48 kHz) · **Speex** (48 kHz) — all four wired and selectable; dropdown ordered best-quality first. Tier default is **Browser-native**.
- [x] **Quality tuning (2026-07):** dry/wet **attenuation floor** (~-16 dB, RNNoise/Speex only — the "robotic" fix; DTLN/DFN would comb-filter), **gate-after-ML**, **DFN level 80→60**. Floor tunable via `lotusDenoiseFloor`.
- [x] **AEC/AGC (2026-07):** echo-cancellation ON; **AGC OFF for the ML tier** (`autoGainControl=false`, threaded through EC `UrlParams`→`ConnectionFactory`) so browser AGC doesn't fight the model; playback confirmed no AEC-defeat.
- [x] **Reliability (2026-07):** never-silent watchdog, resume-timeout, WASM-cache reject-eviction, activate-off-local-participant, init/build leak fixes.
- [ ] **Open verification:** real-call by-ear **A/B** — model choice, floor value, AGC on/off (RNNoise known-weak historically). `LOTUS_TESTING.md` §D2-1 / J2.
- [ ] **GTCRN (RESEARCHED — DEFERRED):** tiny MIT 16 kHz model that beats RNNoise, but **no drop-in browser package** — needs a ~1-week from-scratch build: `onnxruntime-web` (WASM, 1 thread) in a **Web Worker** (ORT can't run in an AudioWorklet — issue #13072) behind a custom AudioWorklet ring-buffer node presenting as an `AudioNode`; model `gtcrn_simple.onnx` (~300 KB, stateful — thread `conv/tra/inter` caches per frame); we write STFT/iSTFT (n_fft 512/hop 256). Assets ~3–4 MB via the `lotusDenoise()` vite plugin. Registration checklist known (both repos, incl. the 2nd `denoisePipeline.ts` used by the DenoiseTester). **Revisit only if low-power quality is insufficient after validating the current tuning.**
- [ ] **Desktop-only / HW-gated (future):** FRCRN or NVIDIA Maxine (RTX/Tensor only) — impossible in-browser; would run in the Tauri Rust backend + bridge a virtual mic into the webview. Detect capability; web falls back to RNNoise.
- **Excluded:** Krisp (LiveKit Cloud only); FRCRN/Maxine on web (GPU/server-bound).