LotusGuild/cinny
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(denoise): add self-hosted DeepFilterNet 3 ML noise-suppression model

Browse Source 
Integrate DeepFilterNet 3 (deepfilternet3-noise-filter@1.2.1) as a new
client-side denoise model id 'deepfilternet', mirroring the DTLN pattern.

The npm package ships only an ESM whose AudioWorklet processor + wasm-bindgen
glue are inlined as a string (loaded via a Blob URL — no CDN for the worklet).
Its only runtime fetches are a single-threaded df_bg.wasm and an ONNX model
tarball, which previously loaded from an external CDN. We now VENDOR both
(build/denoise-vendor/deepfilternet/v2/...) and self-host them under
denoise/deepfilternet/, overriding the package's cdnUrl so nothing hits the
upstream CDN — keeping it self-hosted / Tauri-CSP safe.

The wasm is single-threaded (no SharedArrayBuffer / atomics / imported shared
memory), so it needs no COOP/COEP cross-origin isolation and runs fine in EC's
non-isolated iframe. Runs at 48 kHz fullband. Any init/runtime failure falls
back to the raw mic, like the other models.

- vite.config.js: copy ESM + vendored wasm/model into the EC denoise dir with a
  required-asset guard that aborts the build if any entry is missing.
- build/lotus-denoise.js: 'deepfilternet' branch — dynamic-import the ESM, build
  a DeepFilterNet3Core pointed at the self-hosted base, await init, return the
  worklet node; 48 kHz; raw-mic fail-safe preserved.
- denoisePipeline.ts: 'deepfilternet' branch for the in-app tester + sampleRate.
- settings.ts: add 'deepfilternet' to DenoiseModelId + getSettings whitelist.
- lotusDenoiseUtils.ts: add the comparison-chart row.
- General.tsx: add the "DeepFilterNet 3 (beta)" dropdown option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jared Vititoe
2026-06-17 19:57:08 -04:00
parent abb7f743b8
commit 04b56ffacd
10 changed files with 141 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vite.config.js
+37
View File
@@ -123,6 +123,43 @@ function lotusDenoise() {
}
fs.cpSync(dtlnSrc, path.join(denoiseDir, 'workadventure'), { recursive: true });
// DeepFilterNet 3 (deepfilternet3-noise-filter): the npm package ships only
// its ESM (index.esm.js) with the AudioWorklet processor + wasm-bindgen glue
// INLINED as a string (loaded via a Blob URL, no CDN for the worklet). The
// only runtime CDN fetches are its single-threaded `df_bg.wasm` and the
// ONNX `DeepFilterNet3_onnx.tar.gz` model — which we VENDOR locally (under
// build/denoise-vendor/deepfilternet/) and serve, overriding the package's
// cdnUrl to our self-hosted base. This keeps the feature CDN-free / Tauri-CSP
// safe. We copy the ESM (the shim dynamic-imports it, mirroring DTLN) plus
// the vendored assets, preserving the package's expected v2/... layout. All
// are required — a missing entry means a broken install, so fail the build.
const dfnEsm = path.resolve('node_modules/deepfilternet3-noise-filter/dist/index.esm.js');
const dfnVendor = path.resolve('build/denoise-vendor/deepfilternet');
const dfnAssets = [
[dfnEsm, path.join(denoiseDir, 'deepfilternet/index.esm.js')],
[
path.join(dfnVendor, 'v2/pkg/df_bg.wasm'),
path.join(denoiseDir, 'deepfilternet/v2/pkg/df_bg.wasm'),
],
[
path.join(dfnVendor, 'v2/models/DeepFilterNet3_onnx.tar.gz'),
path.join(denoiseDir, 'deepfilternet/v2/models/DeepFilterNet3_onnx.tar.gz'),
],
];
const dfnMissing = dfnAssets.filter(([s]) => !fs.existsSync(s)).map(([s]) => s);
if (dfnMissing.length > 0) {
throw new Error(
'[lotus-denoise] DeepFilterNet 3 asset(s) missing — build aborted to avoid ' +
'shipping a broken ML feature:\n ' +
dfnMissing.join('\n ') +
'\n(Run `npm ci`; the vendored wasm/model live under build/denoise-vendor/deepfilternet/.)',
);
}
dfnAssets.forEach(([s, d]) => {
fs.mkdirSync(path.dirname(d), { recursive: true });
fs.copyFileSync(s, d);
});
const shimSrc = path.resolve('build/lotus-denoise.js');
if (!fs.existsSync(shimSrc)) {
throw new Error(`[lotus-denoise] Missing shim source ${shimSrc} — build aborted.`);