Commit Graph

2 Commits

Author SHA1 Message Date
jared abb7f743b8 fix(calls): DTLN at 16kHz + raw-capture A/B; explains weak/robotic results
CI / Build & Quality Checks (push) Successful in 10m23s
CI / Trigger Desktop Build (push) Successful in 5s
Two issues found from real testing of the in-app tester:

1. Raw ≈ RNNoise ≈ Speex sounded identical in Record & compare because the clip
   was captured with browser noise suppression ON (the user's native-NS
   setting), so "Raw" was already cleaned and the models had nothing left to
   remove. Record & compare now captures fully raw audio (noiseSuppression /
   AGC / echoCancellation off) so each model's effect on real noise is audible.
   (Friends still heard differences in calls — the models work; the test was
   feeding them pre-cleaned audio.)

2. DTLN was robotic/choppy/quiet because @workadventure/noise-suppression
   targets 16 kHz (AUDIO_CONFIG.sampleRate) and does NOT resample internally,
   while we ran it at 48 kHz. Run DTLN's whole graph in a 16 kHz context:
   - denoisePipeline: add sampleRateFor(model) (16k for dtln, 48k otherwise);
     tester live-monitor + playback contexts use it (bufferSource resamples the
     48k clip down for DTLN).
   - shim (build/lotus-denoise.js): SAMPLE_RATE is now model-aware, so DTLN is
     correct in real calls too (it was previously broken at 48 kHz). The 16 kHz
     processed track is still published to LiveKit (WebRTC/Opus resamples).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:27:15 -04:00
jared 14cfa021c5 feat(calls): in-app denoise tester to audition models + calibrate gate
CI / Build & Quality Checks (push) Successful in 10m24s
CI / Trigger Desktop Build (push) Successful in 17s
The previous "Test Microphone" meter only showed a raw 0-100% level bar — it
never ran the gate or any model, and its scale wasn't dBFS, so it couldn't tell
you which threshold to pick or let you hear the models solo. Replace it with a
real tester that reuses the shipped worklets (/public/element-call/denoise/) in
a main-app AudioContext, mirroring the call pipeline (source -> gate -> model).

- denoisePipeline.ts: shared loader for the RNNoise/Speex flat worklets and the
  DTLN @workadventure helper, the noise gate, and a dBFS RMS meter helper.
- DenoiseTester.tsx:
  - Live monitor: hear yourself through the selected model (+gate) in real time
    (headphones) with In/Out dBFS meters and a threshold marker on the In meter
    so the gate value is meaningful to calibrate.
  - Record & compare: capture a short clip, then A/B the same audio Raw vs
    RNNoise vs Speex vs DTLN.
- Wire it into the ML settings block; remove the old raw-only MicMeter. Use real
  TDS tokens (--accent-*, --border-color, --bg-card) instead of the invented
  --lt-* names + hardcoded hex the old meter used.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 17:53:57 -04:00