Integrate DeepFilterNet 3 (deepfilternet3-noise-filter@1.2.1) as a new
client-side denoise model id 'deepfilternet', mirroring the DTLN pattern.
The npm package ships only an ESM whose AudioWorklet processor + wasm-bindgen
glue are inlined as a string (loaded via a Blob URL — no CDN for the worklet).
Its only runtime fetches are a single-threaded df_bg.wasm and an ONNX model
tarball, which previously loaded from an external CDN. We now VENDOR both
(build/denoise-vendor/deepfilternet/v2/...) and self-host them under
denoise/deepfilternet/, overriding the package's cdnUrl so nothing hits the
upstream CDN — keeping it self-hosted / Tauri-CSP safe.
The wasm is single-threaded (no SharedArrayBuffer / atomics / imported shared
memory), so it needs no COOP/COEP cross-origin isolation and runs fine in EC's
non-isolated iframe. Runs at 48 kHz fullband. Any init/runtime failure falls
back to the raw mic, like the other models.
- vite.config.js: copy ESM + vendored wasm/model into the EC denoise dir with a
required-asset guard that aborts the build if any entry is missing.
- build/lotus-denoise.js: 'deepfilternet' branch — dynamic-import the ESM, build
a DeepFilterNet3Core pointed at the self-hosted base, await init, return the
worklet node; 48 kHz; raw-mic fail-safe preserved.
- denoisePipeline.ts: 'deepfilternet' branch for the in-app tester + sampleRate.
- settings.ts: add 'deepfilternet' to DenoiseModelId + getSettings whitelist.
- lotusDenoiseUtils.ts: add the comparison-chart row.
- General.tsx: add the "DeepFilterNet 3 (beta)" dropdown option.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The prior DTLN attempt (89a2321d) broke the build (missing dep, wrong
`cinny/` asset paths) and typecheck (`'dtln'` not in DenoiseModelId), and was
wired against an API the package doesn't expose. @workadventure/noise-
suppression is not a flat AudioWorklet — it's a self-contained ES module whose
processor name is `workadventure-noise-suppression` and which resolves its own
LiteRT WASM + TFLite models via import.meta.url. Driving it by hand-rolled
addModule + processorOptions cannot work.
- Re-add @workadventure/noise-suppression@0.0.4 (package.json + lockfile).
- vite: copy the package's whole dist/ tree intact to
denoise/workadventure/ (preserving assets/ + vendor/litert) so import.meta
resolution works at runtime; fail the build if the entry module is missing.
- shim: for the DTLN model, dynamic-import denoise/workadventure/audio-worklet
.js and use createNoiseSuppressionAudioWorklet(ctx, { bypassUntilReady })
to build the node; RNNoise/Speex keep their direct flat-worklet path. Async
init errors are logged + reported and fall back to the raw mic.
- Restore 'dtln' in DenoiseModelId (+ settings coercion), the model chart, and
the settings dropdown, labelled "(beta)".
DTLN builds and is fully self-hosted, but its in-call audio is UNVERIFIED in
this environment — needs a real-call test. DeepFilterNet stays excluded (CDN
asset loading, incompatible with self-hosting / Tauri CSP).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audit/repair of the multi-model denoise work so it actually builds and only
exposes working, self-hosted models.
- Complete the DTLN/DFN3 revert: uninstall @workadventure/noise-suppression
and deepfilternet3-noise-filter (package.json + lockfile), drop the unused
DTLN asset-copy block from vite.config.js (was shipping ~2MB of unused
tflite/wasm), and narrow DenoiseModelId to the bundled models (rnnoise,
speex). Coerce any retired persisted model value back to the default.
- Fix General.tsx CI typecheck failures introduced by the denoise UI: restore
three imports the rewrite deleted (useDateFormatItems, SequenceCardStyle,
useTauriUpdater), add the missing denoise/sound imports, and correct
hallucinated Folds props (Text has no variant/bold; Box uses
alignItems/justifyContent). tsc now passes with 0 errors.
- Harden the vite denoise plugin: required RNNoise/Speex/gate assets and the
shim now fail the build loudly if missing (instead of a silent warn that
shipped a broken ML feature), and the index.html shim injection is verified.
- CI: move the cinny-desktop submodule bump into ci.yml as a `trigger-desktop`
job gated on `needs: build`, and delete the standalone trigger-desktop.yml.
A failing push no longer kicks off the slow Tauri builds in parallel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Verified package layouts and integration paths for @workadventure/noise-suppression (v0.0.4) and deepfilternet3-noise-filter (v1.2.1).
- Updated build configuration to correctly copy WASM, TFLite, and ONNX assets.
- Integrated DTLN and DeepFilterNet initialization logic into the audio shim.
- Enabled all four models (RNNoise, Speex, DTLN, DFN3) in Settings UI.
Implement a flexible, multi-model noise suppression pipeline for Element Call/LiveKit integration:
- ML Engines: Added support for RNNoise, Speex, DTLN, and DeepFilterNet 3 models.
- Pipeline Architecture: Implemented modular audio processing in lotus-denoise.js, supporting 'Series Suppression' (running browser-native NSNet2 before ML) and a hardware-style Noise Gate.
- UI & UX Enhancements:
- Settings UI: Added model comparison chart with CPU/Quality metadata.
- Tuning: Added Live Microphone Meter for calibrating Noise Gate thresholds.
- Reporting: Added LotusToast system to alert users when ML suppression fails or falls back to raw input.
- Robustness & Quality:
- Capture Fidelity: Removed forced 48kHz capture constraints to allow native-rate capture (solving static issues with high-end audio interfaces).
- Performance: Added WASM SIMD detection with transparent fallback.
- Capability Detection: Added browser feature detection to disable unsupported ML modes.
- Build Integration: Updated Vite config to self-host all model WASM/tflite assets in /denoise/ directory.
Replace the boolean call noise-suppression setting with a 3-way control
(Off / Browser-native / ML beta) in Settings -> General -> Calls.
- Off: noiseSuppression=false to Element Call
- Browser-native: EC's built-in WebRTC suppressor (prior default)
- ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor)
Element Call captures the mic inside its iframe and publishes to LiveKit,
so the host can't reach that track; LiveKit's Krisp filter is Cloud-only
(we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML
tier instead injects a same-origin pre-init shim into the vendored EC
index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin)
that patches getUserMedia and routes the captured mic through an RNNoise
AudioWorklet before LiveKit sees it -- the same post-capture pipeline as
#3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if
setup fails; keeps echoCancellation/AGC on the raw capture.
- settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy
boolean migration (true->browser, false->off)
- CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends
lotusDenoise=ml (native suppressor off in ML mode)
- vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and
inject the shim <script> before EC's module entry
- docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Point DECORATION_CDN at Lotus Nextcloud WebDAV share instead of external
avatardecoration.com; all 99 APNG files are self-hosted and served via
direct DAV URL (no CORS issue for <img> elements)
- Add onError handler to AvatarDecoration.tsx to silently hide the overlay
if a file is missing or the CDN is unreachable
- Rewrite scripts/syncDecorations.mjs: now sends HTTP HEAD requests to the
live Nextcloud CDN (batches of 16 in parallel) and removes catalog entries
for files that return non-2xx; empty categories are pruned automatically.
Workflow: delete files from Nextcloud → run `npm run sync:decorations` →
commit the updated avatarDecorations.ts. No local files needed.
- Add public/decorations/ to .gitignore; delete the 85 MB local APNG cache
that was downloaded during development (files live on Nextcloud now)
- Add sync:decorations script to package.json
- Update LOTUS_FEATURES.md, LOTUS_TODO.md (P5-13 + P5-14 ✓), README.md
with avatar decoration documentation and catalog sync workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove @esbuild-plugins/node-globals-polyfill (redundant since Vite 8
rolldownOptions.define handles globalThis). Add rolldownOptions.checks
to suppress PREFER_BUILTIN_FEATURE until Vite exposes output in rolldownOptions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GHSA-qjx8-664m-686j: prototype hijack in js-cookie <= 3.0.5 used
transitively via react-use in @giphy/react-components.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- react 18.2.0 to 19.2.6
- react-dom 18.2.0 to 19.2.6
- @types/react 18.2.39 to 19.2.15
- @types/react-dom 18.2.17 to 19.2.3
React 19 breaking changes fixed:
- useRef<T>(null) now returns RefObject<T | null>; cast to
RefObject<T> at 16 component call sites (safe, runtime unchanged)
- useRef<T>() without arg no longer valid; add | undefined>(undefined)
in useDebounce, useFileDrop, useThrottle, useVirtualPaginator hooks,
RoomInput, RoomTimeline, and ClientNonUIFeatures
- useReducer<typeof reducer> 1-arg form removed; drop explicit type arg
in useForceUpdate (inferred from reducer function)
- global JSX namespace removed; import type { JSX } from react in
react-custom-html-parser.tsx
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- eslint 8.57.1 to 9.39.4
- @typescript-eslint/eslint-plugin 7.18.0 to 8.59.4
- @typescript-eslint/parser 7.18.0 to 8.59.4
- globals 11.12.0 to 17.6.0
- @eslint/eslintrc and @eslint/js added for FlatCompat
- Replace .eslintrc.cjs + .eslintignore with eslint.config.mjs
- Use flat configs for react, react-hooks, typescript-eslint directly
- FlatCompat only for airbnb-base (no flat config support yet)
- Fix no-unused-vars override from airbnb and react/display-name: off
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- vite 6.4.2 to 8.0.14
- @vitejs/plugin-react 5.2.0 to 6.0.2
- Migrate optimizeDeps.esbuildOptions to rolldownOptions (Vite 8 uses rolldown)
- Remove @esbuild-plugins/node-globals-polyfill (no longer needed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- typescript 5.9.3 to 6.0.3
- moduleResolution Node to bundler (correct for Vite projects)
- target/lib ES2016 to ES2020 (enables flatMap, Promise.allSettled)
- Fix global to globalThis in initMatrix.ts (browser env)
- Fix EventEmitter default to named import in CallControl.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@giphy/react-components@10.1.2 imports noUUIDRandom from @giphy/js-util,
which was only added in 5.x. Previously the uuid override forced uuid@14
into js-util@4.4.2 breaking the noUUIDRandom export. Pin js-util@5.2.0
directly and drop the uuid override (moderate severity, not high).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Resolves all TS2345/TS2347/TS7006 type errors introduced by stricter TypeScript 5.x.
Fix Icons.Settings to Icons.Setting, cast account data returns, fix implicit any.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1.6.0 did not export SearchContextManager/SearchContext/SearchBar,
causing React error #130 (element type undefined) when opening GifPicker.
5.9.4 uses @emotion (not styled-components), supports React 16-18, and
exports all required components. Downgrade @giphy/js-fetch-api to 4.2.2
to match the peer dep range.
@giphy/react-components@10.x calls styled-components internals
(mergeAttributes) that do not exist in styled-components v6 — crashes
on open. Reverted to 1.6.0 until giphy publishes a v6-compatible release.
WelcomePage: remove Sentry test button (verified working), rename
Support -> Lotus Matrix Guide.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All newly flagged high-severity packages (lodash, js-cookie) are either
in dev-only tools (commitizen) or tree-shaken out of the deployed bundle
(react-use/js-cookie is unused). Zero deployed-bundle impact confirmed.
Being 9 major versions behind accumulates migration debt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lodash >= 4.18.0 patches prototype-pollution (GHSA-f23m-r3pf-42rh) and
code-injection (GHSA-r5fr-rjxr-66jc) used by slate-dom/slate-react in
the deployed bundle.
Attempted @giphy/react-components@10.1.2 upgrade but it pulled in new
high-severity lodash and js-cookie vulns — net regression, reverted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- dompurify updated to 3.4.5 to fix 7 XSS/prototype-pollution CVEs
- emojibase-data added to manualChunks: splits 856 kB out of the main
bundle, reducing it from 1.8 MB to 932 kB
- husky prepare script updated from deprecated "husky install" to "husky"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace broken vite-plugin-static-copy target for pdf.worker with a
custom closeBundle plugin that copies the file directly to dist root.
Also uninstall vite-plugin-top-level-await which was removed from
vite.config.js in the previous commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prettier: auto-formatted 103 files to fix baseline. Prettier check in CI
is now a hard gate (removed continue-on-error).
Brotli: installed libnginx-mod-http-brotli-filter/static. Enabled in nginx
with brotli_static on for pre-compressed assets and comp_level 6.
Sentry releases: deploy script now exports VITE_APP_VERSION=<git-short-sha>
before building so each Sentry release maps to an exact commit.
CI also passes github.sha as VITE_APP_VERSION.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
RoomSkeleton: shimmer skeleton matching Room header/timeline/input layout,
used as Suspense fallback for all three Room routes (home/direct/space)
Sentry source maps: @sentry/vite-plugin uploads 72 hidden source map files
to Sentry on each build then deletes them from dist — stack traces now show
real file/line numbers instead of minified bundle positions.
Auth token loaded from /etc/lotus-deploy.env (not in git).
Auto-deploy: webhook receiver on port 9001, nginx proxies
/hooks/lotus-deploy, HMAC-SHA256 verified, triggers on lotus branch push.
Deploy script: git reset --hard + npm ci + npm run build + rsync to webroot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Initialize Sentry SDK in index.tsx when VITE_SENTRY_DSN env var is set
- Wrap entire App with Sentry.ErrorBoundary (replaces the hard crash with a retry UI)
- 5% trace sample rate, sendDefaultPii disabled, strip events containing accessToken
- Add .env.production template with VITE_SENTRY_DSN placeholder
- Get your DSN from sentry.io -> Project Settings -> Client Keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add defensive check in folds Icon component so that if src is ever
undefined or non-function (root cause unknown, possibly data-dependent),
the SVG renders empty rather than throwing and crashing the whole app.
Also adds postinstall script to re-apply the patch after npm install.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes GHSA-q89c-q3h5-w34g: path traversal & URL injection via unsanitised
lng/ns parameters. Remaining open issues are all in devDependencies
(commitizen/lodash/tmp) or dev-server-only tools (esbuild/vite), with no
runtime impact on the production build.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Poll voting: PollContent sends m.poll.response on answer click
- Location: MLocation shows OSM map embed + share-location button in toolbar
- Image captions: caption field on media uploads sets message body
- Message forwarding: ForwardMessageDialog with searchable room picker
- Also includes ring timeout fix and earlier session patches
* add option to start video all in DM
* show speaker icon for dm's in call status name
* show call view if call is active in room
* add Atria call ringtone
* update element call and widget api
* add option to start voice/video call in dms
* only show call button if user have permission
* allow call widget to send call notification event
* show incoming call dialog and play sound
* fix call permission checks
* allow option to start call in all rooms
* send notification when starting call in non-voice rooms
* hide header call button from voice rooms
* prevent call join if call not supported and started by other party
* update call menu style
* show call not supported message on incoming call notification
* improve the incoming call layout
* video call with right click without opening menu
* allow call widget to fetch media url
* add webRTC missing error
* improve call permission label
---------
Co-authored-by: Krishan <33421343+kfiven@users.noreply.github.com>
* chore: install deps related to semantic release
* chore: add husky config
* ci: add a script to update version number on new release
* ci: update ci/cd to include semantic release changes
* chore: merge dev to semantic-release
jared
Krishan
Ajay Bura
Lotus Bot
root
renovate[bot]