Compare commits

...
Author SHA1 Message Date
jared 9e6fe75f14 Merge pull request 'fix(denoise): DeepFilterNet attenuation limit 60 → 25 dB (choppy word endings)' (#65) from fix/dfn-strength-25 into lotus
CI / Build embedded bundle (push) Successful in 5m59s
CI / Publish to Gitea npm registry (push) Successful in 4m12s
2026-10-30 18:58:29 -04:00
Lotus CIandClaude Opus 5.5 f180f76196 fix(denoise): DeepFilterNet attenuation limit 60 -> 25 dB
CI / Build embedded bundle (push) Skipped
CI / Build embedded bundle (pull_request) Successful in 8m5s
CI / Publish to Gitea npm registry (pull_request) Skipped
At 60 dB the background is muted outright (measured -61 to -64 dB), so
each word drops into dead silence and listeners hear the mic cutting out
at word ends. On room-echo speech (RT60 ~0.4 s) with fan and keyboard
noise at SNR 15 dB, 25 dB scored best:

  limit  PESQ clean/fan/keyboard   noise removed
  60     3.35 / 2.33 / 2.04        ~62 dB
  40     3.41 / 2.45 / 2.21        32-40 dB
  25     3.59 / 2.72 / 2.38        25 dB
  20     3.63 / 2.66 / 2.25        20 dB

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-30 18:50:06 -04:00
jared 4ca3b0695c Merge pull request 'fix(denoise): stop DeepFilterNet silently passing raw audio through' (#64) from fix/dfn-model-bytes into lotus
CI / Build embedded bundle (push) Successful in 8m41s
CI / Publish to Gitea npm registry (push) Skipped
2026-10-30 15:16:25 -04:00
Lotus CIandClaude Opus 5.5 283b231c7a style: oxfmt
CI / Build embedded bundle (push) Skipped
CI / Build embedded bundle (pull_request) Successful in 5m2s
CI / Publish to Gitea npm registry (pull_request) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-30 15:11:19 -04:00
Lotus CIandClaude Opus 5.5 16f5f3d1a3 fix(denoise): stop DeepFilterNet silently passing raw audio through
CI / Build embedded bundle (push) Skipped
CI / Build embedded bundle (pull_request) Failing after 5m38s
CI / Publish to Gitea npm registry (pull_request) Skipped
If the DeepFilterNet model is served with `Content-Encoding: gzip` (vite
preview does this for .tar.gz; some CDNs and nginx gzip_static setups do
too), fetch() hands back the already-unpacked tar. The library's worklet
then fails inside df_create, swallows the error and passes the raw mic
through, while Lotus Call reports DeepFilterNet as active.

Check the bytes after initialize(): a gzip model is used as-is, an
unpacked tar is re-gzipped in the browser, and anything else throws so
the rnnoise fallback runs and the host is told the truth.

Measured in a two-browser call (fan noise, SNR 15 dB): before, 4.9 dB of
noise removed, i.e. none beyond gain differences; after, 30.9 dB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-30 14:18:01 -04:00
jared 117467b1fb Merge pull request 'chore(deps): security fixes for dev/build tooling (Storybook, Vitest, Vite, transitive)' (#63) from chore/dev-tool-security into lotus
CI / Build embedded bundle (push) Successful in 5m37s
CI / Publish to Gitea npm registry (push) Skipped
2026-10-25 23:59:03 -04:00
Lotus CIandClaude Opus 5.5 43cbd8409d chore(deps): security fixes for dev/build tooling (Storybook, Vitest, Vite, transitive)
CI / Build embedded bundle (push) Skipped
CI / Build embedded bundle (pull_request) Successful in 4m6s
CI / Publish to Gitea npm registry (pull_request) Skipped
The npm advisory database flagged 19 packages in pnpm-lock.yaml. None of them
ship in the embedded call bundle (checked against the built dist); they run
on developer machines and in CI:
- storybook / @storybook/* 10.6.0 -> 10.6.3: critical, dev server code
  injection from the local network.
- vitest 4.1.12 (+ @vitest/ui unpinned to ^4.1.12), tinypool ^2.1.2:
  mock registration code injection, prototype-pollution RCE.
- vite ^8.3.5: dev-server file exposure, launch-editor issues.
- Overrides for transitive tooling: brace-expansion, browserslist,
  smol-toml, source-map-js, postcss-selector-parser, baseline-browser-
  mapping, fflate, pbkdf2, zod 4; existing qs/js-yaml/undici/dompurify
  overrides raised to the fixed versions.

Left: elliptic (no fixed release, low), zod 3 (no 3.x fix; a dependency
pins it). Lotus unit tests 144/144, build:embedded OK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-25 23:50:09 -04:00
jared 4c84c3a4f3 Merge pull request 'feat(lotus): expected-device check before joining (cinny #121)' (#62) from feat/expected-devices into lotus
CI / Build embedded bundle (push) Successful in 4m12s
CI / Publish to Gitea npm registry (push) Successful in 1m32s
2026-10-24 19:47:38 -04:00
5 changed files with 518 additions and 364 deletions
+8 -8
View File
@@ -64,9 +64,9 @@
"@rolldown/plugin-babel": "^0.2.3",
"@sentry/react": "^8.0.0",
"@sentry/vite-plugin": "^3.0.0",
"@storybook/addon-docs": "^10.3.6",
"@storybook/addon-vitest": "^10.3.6",
"@storybook/react-vite": "^10.3.6",
"@storybook/addon-docs": "^10.6.3",
"@storybook/addon-vitest": "^10.6.3",
"@storybook/react-vite": "^10.6.3",
"@testing-library/dom": "^10.1.0",
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^16.0.0",
@@ -86,9 +86,9 @@
"@vector-im/compound-design-tokens": "^11.0.0",
"@vector-im/compound-web": "^10.0.0",
"@vitejs/plugin-react": "^6.0.2",
"@vitest/browser-playwright": "^4.1.5",
"@vitest/browser-playwright": "^4.1.12",
"@vitest/coverage-v8": "^4.0.18",
"@vitest/ui": "4.1.11",
"@vitest/ui": "^4.1.12",
"babel-plugin-react-compiler": "^1.0.0",
"classnames": "^2.3.1",
"copy-to-clipboard": "^3.3.3",
@@ -123,18 +123,18 @@
"react-use-measure": "^2.1.1",
"rxjs": "^7.8.1",
"sass": "^1.42.1",
"storybook": "^10.3.6",
"storybook": "^10.6.3",
"typescript": "^5.8.3",
"unique-names-generator": "^4.6.0",
"uuid": "^14.0.0",
"vaul": "^1.0.0",
"vite": "^8.0.0",
"vite": "^8.3.5",
"vite-plugin-html": "^3.2.2",
"vite-plugin-node-polyfills": "^0.28.0",
"vite-plugin-node-stdlib-browser": "^0.2.1",
"vite-plugin-svgr": "^4.0.0",
"vite-plugin-wasm": "^3.6.0",
"vitest": "^4.1.5",
"vitest": "^4.1.12",
"vitest-axe": "^1.0.0-pre.3"
},
"packageManager": "pnpm@11.21.0+sha512.521705bce689924eac72f5a3587122f362689ef6571e55ba80076fd637c11132ecffada26fad4ea79c485bfddbfd3d5a2a5b05805a77e893de71ec8a6cca3bb1"
+418 -348
View File
File diff suppressed because it is too large Load Diff
+18 -4
View File
@@ -28,24 +28,38 @@ overrides:
# - https://github.com/advisories/GHSA-7r86-cg39-jmmj
# - https://github.com/advisories/GHSA-23c5-xmqv-rm74
"minimatch": "^10.2.3"
# Security fixes for transitive build/test tooling (none of these ship in the
# embedded call bundle; checked against the npm advisory database 2026-10-25).
"tinypool": "^2.1.2"
"@vitest/mocker": "^4.1.12"
"brace-expansion@>=4": "^5.0.12"
"browserslist": "^4.28.7"
"smol-toml": "^1.8.1"
"source-map-js": "^1.2.2"
"postcss-selector-parser@>=7": "^7.1.6"
"baseline-browser-mapping": "^2.11.0"
"fflate@<0.5": "^0.4.9"
"fflate@>=0.8": "^0.8.3"
"pbkdf2": "^3.1.7"
"zod@>=4": "^4.6.6"
# Security fix: https://github.com/element-hq/element-call/security/dependabot/109
"glob": "^10.5.0"
# Security fixes:
# - https://github.com/element-hq/element-call/security/dependabot/110
# - https://github.com/element-hq/element-call/security/dependabot/122
# - https://github.com/element-hq/element-call/security/dependabot/179
"qs": "^6.15.2"
"qs": "^6.16.0"
# Security fixes:
# - https://github.com/element-hq/element-call/security/dependabot/106
# - https://github.com/element-hq/element-call/security/dependabot/203
"js-yaml": "^4.2.0"
"js-yaml": "^4.3.2"
# Storybook declares support for 0.27.0 only but empirically works fine with 0.28.0.
"esbuild": "^0.28.0"
# Security fixes:
# - https://github.com/nodejs/undici/releases/tag/v6.24.0
# - https://github.com/element-hq/element-call/security/dependabot/201
# - https://github.com/element-hq/element-call/security/dependabot/199
"undici": "^6.27.0"
"undici": "^6.29.1"
# Security fix: https://github.com/advisories/GHSA-rf6f-7fwh-wjgh
"flatted": "^3.4.2"
# Security fix: https://github.com/element-hq/element-call/security/dependabot/184
@@ -59,4 +73,4 @@ overrides:
# - https://github.com/element-hq/element-call/security/dependabot/196
# - https://github.com/element-hq/element-call/security/dependabot/195
# - https://github.com/element-hq/element-call/security/dependabot/193
"dompurify": "^3.4.11"
"dompurify": "^3.4.17"
+35
View File
@@ -11,6 +11,7 @@ import { type AudioProcessorOptions } from "livekit-client";
import {
assertModuleExport,
type LotusDenoiseConfig,
normalizeDfnModelBytes,
LotusDenoiseProcessor,
prepareDenoiseAssets,
releasePreparedDenoiseAssets,
@@ -245,3 +246,37 @@ describe("assertModuleExport (#26)", () => {
);
});
});
describe("normalizeDfnModelBytes", () => {
const tar = (): Uint8Array => {
const b = new Uint8Array(1024);
b.set(new TextEncoder().encode("ustar"), 257);
b.set(new TextEncoder().encode("model.onnx"), 0);
return b;
};
const gunzip = async (buf: ArrayBuffer): Promise<Uint8Array> =>
new Uint8Array(
await new Response(
new Response(buf).body!.pipeThrough(new DecompressionStream("gzip")),
).arrayBuffer(),
);
test("passes a gzipped model through untouched", async () => {
const gz = new Uint8Array([0x1f, 0x8b, 8, 0, 1, 2, 3]).buffer;
expect(await normalizeDfnModelBytes(gz)).toBe(gz);
});
test("re-gzips a tar that a Content-Encoding server already unpacked", async () => {
const out = await normalizeDfnModelBytes(tar().buffer as ArrayBuffer);
const u8 = new Uint8Array(out);
expect([u8[0], u8[1]]).toEqual([0x1f, 0x8b]);
expect(await gunzip(out)).toEqual(tar());
});
test("rejects anything else (e.g. an HTML error page)", async () => {
const html = new TextEncoder().encode("<!doctype html><title>404</title>");
await expect(
normalizeDfnModelBytes(html.buffer as ArrayBuffer),
).rejects.toThrow(/neither gzip nor tar/);
});
});
+39 -4
View File
@@ -144,6 +144,8 @@ interface DfnCore {
initialize: () => Promise<void>;
createAudioWorkletNode: (ctx: AudioContext) => Promise<AudioNode>;
destroy: () => void;
/** Set by `initialize()`; the model bytes are handed to the worklet as-is. */
assets?: { wasmModule: WebAssembly.Module; modelBytes: ArrayBuffer } | null;
}
interface DfnModule {
DeepFilterNet3Core: new (opts: {
@@ -167,6 +169,33 @@ export function assertModuleExport<T>(
return mod as T;
}
/**
* Make sure the DeepFilterNet model is the gzipped tarball its wasm expects.
*
* A server or proxy that sends the `.tar.gz` with `Content-Encoding: gzip`
* (vite preview does; some CDNs and nginx `gzip_static` setups do too) makes
* `fetch()` hand back the inner tar. The library's worklet then fails inside
* `df_create`, swallows the error and passes the raw mic straight through,
* while we report DeepFilterNet as active. An already-decompressed tar is
* re-gzipped; anything else throws, so the rnnoise fallback runs and the host
* is told the truth.
*/
export async function normalizeDfnModelBytes(
bytes: ArrayBuffer,
): Promise<ArrayBuffer> {
const u8 = new Uint8Array(bytes);
if (u8.length > 2 && u8[0] === 0x1f && u8[1] === 0x8b) return bytes;
const isTar =
u8.length > 262 &&
String.fromCharCode(...u8.subarray(257, 262)) === "ustar";
if (!isTar)
throw new Error(
`denoise: DeepFilterNet model is neither gzip nor tar (${u8.length} bytes)`,
);
const gz = new Response(u8).body!.pipeThrough(new CompressionStream("gzip"));
return new Response(gz).arrayBuffer();
}
async function loadDfnCore(config: LotusDenoiseConfig): Promise<DfnCore> {
const base = config.assetBase;
const url = `${base}deepfilternet/index.esm.js`;
@@ -178,13 +207,19 @@ async function loadDfnCore(config: LotusDenoiseConfig): Promise<DfnCore> {
);
const core = new mod.DeepFilterNet3Core({
sampleRate: 48_000,
// 60, not 80: full-strength suppression is the main source of the
// "over-processed" character; a lower level keeps voice natural while
// the dry/wet floor handles the noise tail.
noiseReductionLevel: 60,
// Attenuation limit in dB. At 60 the background is muted outright
// (measured -61 to -64 dB), so every word drops into dead silence and
// listeners hear the mic "cutting out" at word ends. 25 keeps a quiet
// background bed and scored best on room-echo speech with fan and
// keyboard noise (PESQ 2.72 / 2.38 vs 2.33 / 2.04 at 60).
noiseReductionLevel: 25,
assetConfig: { cdnUrl: dfnBase },
});
await core.initialize();
if (core.assets)
core.assets.modelBytes = await normalizeDfnModelBytes(
core.assets.modelBytes,
);
return core;
}