- C-H1: forceState only on FIRST join; on EC reconnect re-arm the fork handlers
(resendForkState — deafen+quality only) instead of clobbering live mic/video/
deafen back to the join-time snapshot.
- C-H2: AFK auto-mute reads the fork's io.lotus.call_state VAD of the LOCAL
published track instead of getUserMedia on the browser DEFAULT mic (which could
measure silence while the user spoke on another device → auto-mute an active
speaker). Fails safe (never mutes) when call_state is null OR empty.
- C-H3: control observer re-binds after EC re-renders (body subtree:true + 100ms
debounce) with an early-return so unchanged state doesn't re-render.
- C-M3 setQuality join-gated; C-M4 hangup 4s fallback dispose (idempotent);
C-M5 PTT no longer silently un-deafens; C-M6 screenshare-audio mute resets on
stop; C-L4 deafen key works in the iframe; C-L6 setState-after-unmount guards.
Reviewed (C-H2 [] fail-safe + C-H3 re-render guard applied). tsc/eslint/prettier
clean, build OK, 677 tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- index.tsx: request navigator.storage.persist() for logged-in sessions so the
browser can't evict the IndexedDB rust-crypto store (eviction while the
localStorage session survives resurrects the device with a blank store → the
KE-1 "one time key already exists" upload storm). Guarded, checks persisted()
first, best-effort.
- Docs: remove HANDOFF_ELEMENT_CALL_FORK.md, LOTUS_E2EE_INVESTIGATION.md, and
LOTUS_BUGS.md. Port their live content into the three kept docs — verification
backlog → LOTUS_TESTING; open bugs + E2EE (KE-1..4) + an Element Call fork
operational reference (publish steps + io.lotus action catalog) → LOTUS_TODO.
Fix all dangling references (README, code comments, cross-doc links). Full
history of the removed docs remains in git.
Gates: tsc/eslint/prettier clean, build OK, 665 tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CallControl now sends the new io.lotus.set_deafen action (join-gated via
forceState) on every deafen / screenshare-audio-mute toggle + on join, ALONGSIDE
the retained iframe-DOM .muted hack (transitional). Against the current pinned
bundle the action is immediately error-replied + swallowed by .catch — inert, no
timeout. Reordered toggleSound() to commit state before setSound() so the sent
deafen value isn't inverted.
Phase 2 (after the fork is published): bump the pin lotus.1 -> lotus.2 and delete
the DOM hack. Docs: HANDOFF §12.4, LOTUS_TODO P6-2, LOTUS_BUGS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
HEAD-checks the copied denoise worklet/wasm/model assets for the selected model
and console.warns a single line listing anything missing — a silent asset skew
between the EC fork's expectations and vite's copied files would otherwise
disable noise suppression with no signal. Fire-and-forget; never blocks call
setup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CallEmbed sets `autoGainControl=false` for the ML noise-suppression tier so
the browser's auto gain control doesn't fight the in-source ML model; the
browser/off tiers keep AGC on.
- Docs: refresh the LOTUS_FEATURES noise-suppression section (browser-native
default, quality-ordered dropdown, DFN3 ML default, attenuation floor,
gate-after-ML, DFN level 60, AGC-off, the reliability fixes) and LOTUS_TODO
P5-30 (mark tuning/reliability/AGC done; record GTCRN as researched-and-deferred).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Model dropdown is now ordered by quality/CPU, best first (DeepFilterNet 3 →
DTLN → RNNoise → Speex); fix RNNoise's inaccurate "High" voice-quality label.
- When a user opts into the ML tier, default to the highest-quality model
(DeepFilterNet 3). The tier default stays browser-native (known-good, best
perceived in testing so far).
- Wire the "Series Suppression" (native-NS-before-ML) toggle into the real call
path — it was applied only in the settings tester, so the tester could sound
better than the actual call. Default it OFF (a single NS stage is best
practice; it's an opt-in test aid).
- isMLDenoiseSupported now also requires WebAssembly, so ML isn't offered on
strict-CSP shells where it would silently fall back to the raw mic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Element Call is now consumed as our self-built fork
(@lotusguild/element-call-embedded); wire up its previously-dormant
capabilities and document the fork as live.
Soundboard (P5-15): a call-bar button plays user-uploaded audio clips into the
call as a real published track (io.lotus.inject_audio) plus local playback.
Clips are uploadable like emoji/sticker packs, stored in io.lotus.soundboard
account data (synced across devices). Gated by a Settings toggle + volume.
Quality controls (P5-31): per-user mic/screenshare bitrate + screenshare
framerate (Settings -> Calls), applied via io.lotus.set_quality clamped to any
room cap. Room admins set caps and hard call-permissions (allow_screenshare /
allow_camera) in Room Settings -> Voice; the call bar hides blocked buttons.
- New: CallSoundboard, useSoundboard, soundboardClips; RoomQuality,
useCallQuality, callQuality (+ unit tests).
- Optimistic-write RoomQuality admin UI (no stale-state clobber).
- Docs: mark EC fork live across README/FEATURES/TODO/BUGS/TESTING; add D2
manual-test steps.
Numeric quality caps are client-cooperative; screenshare/camera permissions are
hard-enforced server-side (see LotusGuild/matrix voice-limit-guard).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the eslint/prettier failures from the previous commit (non-blocking
in CI, but real): drop the banned `void` operator on fire-and-forget
transport.send().catch() calls, prefix the now-unused _denoiseNativeNS
param, and run prettier on the touched files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Switch to @lotusguild/element-call-embedded@0.20.1-lotus.1 (our self-built
fork) and turn on the source-level features it adds:
- #1 denoise CUTOVER: in-source ML denoise (lotusDenoiseSource=1) replaces
the build-time getUserMedia shim — removed the shim injection from
vite.config.js (denoise/ assets still shipped; the processor loads them).
Survives reconnects (fixes A7).
- #2 call-state: CallEmbed consumes io.lotus.call_state; useCallSpeakers /
useRemoteAllMuted prefer it over scraping EC's DOM (DOM fallback kept;
empty payloads ignored).
- #4 focus: CallControl.focusCameraParticipant sends io.lotus.focus_participant
(works during screenshare), replacing the DOM tile-click hack.
- #5 theming: lotusTransparent=1 (native transparent background).
- #6 decorations: LotusDecorationPusher sends each member's decoration URL
via io.lotus.decorations -> rendered on in-call tiles.
#3 soundboard / #7 quality ship dormant (EC-ready; no host UI sends them yet).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- N122: setMediaState resolves on EC's transport ACK instead of waiting for a
DeviceMute state-echo that EC may elide or skip during teardown — which
previously stranded the promise forever and silently skipped the initial
deafen state + first StateUpdate on join. Dropped the single-slot
mediaStatePromiseResolver; onMediaState remains the authoritative sync path.
- N123: focusCameraParticipant now waits for a spotlight videoTile to mount via
a MutationObserver (with a 600ms hard-timeout fallback) instead of a fixed
2-frame delay that EC's React commit can exceed on slower devices.
- N126: PiP position restored from localStorage is shape+finiteness validated,
so corrupt data can't feed NaN into the position math (invalid 'NaNpx' CSS).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CallEmbed watchdog now SELF-HEALS: a genuine ready/joined signal arriving
after the 25s timeout clears the error and notifies subscribers with
undefined, so a slow-but-successful EC load no longer strands the user on
the recovery screen over a live call. Listener dispatch wrapped in try/catch.
- ringtones: synth notes route through a per-session master gain; stop() ramps
it to 0 so the ring is silenced instantly on answer instead of letting the
last scheduled phrase ring out over call audio.
- IncomingCallBanner: ping fires exactly once per incoming call (guarded by
refEventId) instead of re-pinging when ringtone settings change mid-banner.
- focusCameraParticipant: try multiple tile selectors (EC labels vary by
version), defer the tile click past EC's async spotlight layout switch
(rAF x2), and dev-warn when no tile matches so testers get signal.
- uploadContent: a cancelled upload (mx.cancelUpload -> AbortError) is no
longer treated as retryable — previously the retry loop could resurrect an
upload the user just cancelled. Also retry on 408.
- addRoomIdToMDirect/removeRoomIdFromMDirect: guard against a corrupt m.direct
whose values aren't arrays.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CallEmbed: 25s load watchdog that fails fast on iframe error / preparing-error /
timeout instead of hanging on a permanent spinner; additive onLoadError API,
cleared on ready/capabilities/joined.
- CallView: user-visible "call failed to load" overlay with Retry/Leave (folds +
tokens) via a new useCallLoadError hook.
- CallMemberCard: wrap the participant avatar in AvatarDecoration so decorations
render in the call roster (the tile rendered UserAvatar bare while member lists
already wrapped it).
Addresses LOTUS_BUGS item 3 (avatar decorations in calls) and EC iframe failure monitoring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure formatting reflows (multi-line wrapping of long lines/imports/tables);
no behavior change. Clears the working tree of pending prettier diffs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Four changes to match screenshare full-screen UX for camera feeds:
1. Fullscreen button always visible
CallControls.tsx: remove `screenshare &&` gate — the ⛶ fullscreen
button now appears in camera-only calls, not just during screenshare.
2. Per-participant camera focus (CallControl.focusCameraParticipant)
Finds the target's video tile in the EC iframe DOM via:
[data-testid="videoTile"] / [data-video-fit]
closest ancestor of [aria-label="${userId}"]
Enables spotlight mode if not already active, then clicks the tile
so EC's internal focus handler runs. Falls back gracefully if the
tile is not in the DOM (camera off).
3. MemberGlance participant popup
Clicking a participant avatar in the call status bar now shows a
small menu: "Focus camera" (calls focusCameraParticipant) and
"View profile" (existing behaviour). Previously it opened the
profile immediately with no way to focus the camera.
4. PiP fullscreen button
A ⛶/⊡ icon button appears in the PiP overlay top-right area,
letting users go fullscreen directly from PiP mode without
navigating back to the call room first.
UNTESTED — requires a real multi-participant call to verify tile
clicking behaviour and fullscreen transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace raw error object logging (which may contain Matrix event
payloads, user IDs, or message bodies) with e.message-only strings
in three files:
- CallEmbed.ts: state update and event widget feed errors
- msgContent.ts: image/video element load failures and thumb errors
- RoomInput.tsx: GIF send failure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement a flexible, multi-model noise suppression pipeline for Element Call/LiveKit integration:
- ML Engines: Added support for RNNoise, Speex, DTLN, and DeepFilterNet 3 models.
- Pipeline Architecture: Implemented modular audio processing in lotus-denoise.js, supporting 'Series Suppression' (running browser-native NSNet2 before ML) and a hardware-style Noise Gate.
- UI & UX Enhancements:
- Settings UI: Added model comparison chart with CPU/Quality metadata.
- Tuning: Added Live Microphone Meter for calibrating Noise Gate thresholds.
- Reporting: Added LotusToast system to alert users when ML suppression fails or falls back to raw input.
- Robustness & Quality:
- Capture Fidelity: Removed forced 48kHz capture constraints to allow native-rate capture (solving static issues with high-end audio interfaces).
- Performance: Added WASM SIMD detection with transparent fallback.
- Capability Detection: Added browser feature detection to disable unsupported ML modes.
- Build Integration: Updated Vite config to self-host all model WASM/tflite assets in /denoise/ directory.
Replace the boolean call noise-suppression setting with a 3-way control
(Off / Browser-native / ML beta) in Settings -> General -> Calls.
- Off: noiseSuppression=false to Element Call
- Browser-native: EC's built-in WebRTC suppressor (prior default)
- ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor)
Element Call captures the mic inside its iframe and publishes to LiveKit,
so the host can't reach that track; LiveKit's Krisp filter is Cloud-only
(we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML
tier instead injects a same-origin pre-init shim into the vendored EC
index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin)
that patches getUserMedia and routes the captured mic through an RNNoise
AudioWorklet before LiveKit sees it -- the same post-capture pipeline as
#3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if
setup fails; keeps echoCancellation/AGC on the raw capture.
- settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy
boolean migration (true->browser, false->off)
- CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends
lotusDenoise=ml (native suppressor off in ML mode)
- vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and
inject the shim <script> before EC's module entry
- docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Omitting sendNotificationType for call rooms caused Element Call to
default to ring behavior. Now all starting-call events explicitly set
notification (or ring for DMs). Voice channels always get notification.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add screenshareAudioMuted state to CallControlState and CallControl
- setSound() now preserves screenshare audio mute when un-deafening
- Add toggleScreenshareAudio() targeting audio[data-lk-source="screen_share_audio"]
- Add ScreenshareAudioButton (volume icon, warns when muted) to controls bar
- Fix unused prevScreenshare variable (ESLint error from prior commit)
- Run Prettier on Controls.tsx and CallControl.ts (CI formatting failures)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove revert-to-grid logic that was overriding EC's natural screenshare
spotlight, causing fullscreen to show user avatars instead of the screen
- Add fullscreen button to call controls (visible when screensharing) that
requests fullscreen on the call embed container
- Add FullscreenButton component with enter/exit SVG icons to Controls.tsx
- PIP mode: sync setPipMode to CallControl; auto-enable spotlight when
screenshare is active in pip so the screenshare fills the window
- Make useCallControlState accept undefined control for safe use in
CallEmbedProvider
- Add package-lock.json to .gitignore (generated by local npm install)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
downloadMedia: on 401 (SW session race or allow_redirect hop stripping
auth), retry via /_matrix/media/v3/ which is public on this homeserver
(allow_public_access_to_media_repo: true). Fixes images not loading
after sending, and avatar 401s in call prescreen tiles.
CallEmbed: inject flex-centering CSS for EC 0.19.4 participant avatar
container so the initial letter is correctly centered in its circle.
CSS class names are scoped to _avatarContainer_1mrho_40 in EC 0.19.4.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CallEmbed: inject :root { color-scheme } into iframe so EC respects Cinny
theme regardless of OS preference (fixes white background in dark mode)
- CallEmbed: store themeKind, update color-scheme CSS on live setTheme() calls
- CallEmbed: catch transport.send() rejection in setTheme() to prevent
unhandled promise rejection when widget not ready yet (fixes REACT-8)
- CallEmbed: html + body both set to background:none so wallpaper shows through
- CallEmbedProvider: apply chatBackground wallpaper style to call embed
container in full-view mode (not PiP) -- wallpapers carry over to calls
- useCallEmbed: pass themeKind through to CallEmbed constructor
- index.tsx: ignoreErrors: [Request timed out] to suppress matrixRTC
heartbeat timeouts (REACT-9) from Sentry noise
- README: document 0.19.4, positioning fix, dark mode fix, wallpaper,
millify Rolldown interop fix, Sentry noise filter
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the listenAction wrapper only called preventDefault() to stop
the switch default from firing an error, but it never sent a reply.
The widget transport would then wait for a response until it timed out.
Now the wrapper also calls transport.reply(ev.detail, {}) to return an
immediate success, fixing io.element.join, io.element.device_mute, and
set_always_on_screen.
The base WidgetDriver throws Failed to override function for these
methods. ClientWidgetApi routes update_delayed_event widget actions to
cancelScheduledDelayedEvent, restartScheduledDelayedEvent, or
sendScheduledDelayedEvent. Without these overrides every delayed-event
refresh from element-call fails, causing MembershipManager to drop the
call after retries.
Also make listenAction auto-call preventDefault so io.element.join and
other custom widget actions return success. Add set_always_on_screen
handler so element-call PiP requests are acknowledged.
- RoomTimeline.tsx: add eslint-disable comment for intentional eventsLength
dep on timelineSegments useMemo (needed to detect in-place timeline mutations)
- Remove ~47 stale eslint-disable-next-line comments across 28 files for rules
that are now off in the flat config (no-param-reassign, jsx-a11y/media-has-caption,
react/no-array-index-key, etc); run prettier to reformat
- vite.config.js: move manualChunks from rollupOptions.output to
rolldownOptions.output so Rolldown (Vite 8) actually applies it; main bundle
drops from 3.5 MB to 814 kB gzip-248 kB, matrix-sdk gets its own 1.16 MB
cacheable chunk
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- typescript 5.9.3 to 6.0.3
- moduleResolution Node to bundler (correct for Vite projects)
- target/lib ES2016 to ES2020 (enables flatMap, Promise.allSettled)
- Fix global to globalThis in initMatrix.ts (browser env)
- Fix EventEmitter default to named import in CallControl.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prettier: auto-formatted 103 files to fix baseline. Prettier check in CI
is now a hard gate (removed continue-on-error).
Brotli: installed libnginx-mod-http-brotli-filter/static. Enabled in nginx
with brotli_static on for pre-compressed assets and comp_level 6.
Sentry releases: deploy script now exports VITE_APP_VERSION=<git-short-sha>
before building so each Sentry release maps to an exact commit.
CI also passes github.sha as VITE_APP_VERSION.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CallEmbed: fix memory leak — mx event listeners were never removed
because dispose() called .bind(this) again, creating new function
objects. Now uses arrow class fields so start()/dispose() share the
exact same reference.
- callPreferences: toggleVideo is a no-op when cameraOnJoin=false,
preventing internal state drift from the returned value.
- CallControls: PTT key guard now blocks on SELECT elements and walks
the DOM for inherited contentEditable to prevent key interception
inside dropdowns and custom editors.
- RoomInput: GIF fetch validates Giphy CDN domain allow-list,
HTTP Content-Type header, and enforces 20 MB size cap.
EC 0.19.3 changed the toolbar layout. The old previousElementSibling
traversal from the leave button pointed at wrong elements:
- settingsButton was finding the raise-hand button
- reactionsButton was finding the screenshare button
Fix: use stable selectors instead:
- settingsButton: data-testid=settings-bottom-center (new in EC 0.19.3)
- reactionsButton: [class*=raiseHand] (CSS module class, consistent in 0.19.x)
- Camera no longer starts enabled when user disables it in prescreen
- When PTT mode is enabled, call starts muted so PTT works immediately
without requiring a manual mute first
- CallControlState also updated to match the forced-off audio for PTT
EC 0.16.x ignores io.element.device_mute for initial state at startup,
so audio= and video= URL params are the only reliable way to set the
initial device state before the call begins.
- Push to Talk: keydown/keyup binds mic to configurable key (default Space)
with visual PTT indicator and key-binding UI in Settings > General > Calls
- Camera always defaults OFF on join; cameraOnJoin setting for explicit opt-in
- Deafen button tooltip corrected to Deafen/Undeafen instead of Turn Off/On Sound
- Screenshare confirmation dialog before broadcasting to call participants
- Noise suppression toggle wired from settings through CallEmbed URL params
- CallControl.setMicrophone() public method for programmatic mic control
- Calls settings section added to General settings page
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* add option to start video all in DM
* show speaker icon for dm's in call status name
* show call view if call is active in room
* add Atria call ringtone
* update element call and widget api
* add option to start voice/video call in dms
* only show call button if user have permission
* allow call widget to send call notification event
* show incoming call dialog and play sound
* fix call permission checks
* allow option to start call in all rooms
* send notification when starting call in non-voice rooms
* hide header call button from voice rooms
* prevent call join if call not supported and started by other party
* update call menu style
* show call not supported message on incoming call notification
* improve the incoming call layout
* video call with right click without opening menu
* allow call widget to fetch media url
* add webRTC missing error
* improve call permission label
---------
Co-authored-by: Krishan <33421343+kfiven@users.noreply.github.com>
* add option to start video all in DM
* show speaker icon for dm's in call status name
* show call view if call is active in room
* add Atria call ringtone
* update element call and widget api
* add option to start voice/video call in dms
* only show call button if user have permission
* allow call widget to send call notification event
* show incoming call dialog and play sound
* fix call permission checks
* allow option to start call in all rooms
* send notification when starting call in non-voice rooms
* hide header call button from voice rooms
* prevent call join if call not supported and started by other party
* update call menu style
* show call not supported message on incoming call notification
* improve the incoming call layout
* video call with right click without opening menu
* allow call widget to fetch media url
* add webRTC missing error
* improve call permission label
---------
Co-authored-by: Krishan <33421343+kfiven@users.noreply.github.com>
* allow user to end call if error when loading
* show call support missing error if livekit server is not provided
* prevent joining from nav item double click if no livekit support
* allow user to end call if error when loading
* show call support missing error if livekit server is not provided
* prevent joining from nav item double click if no livekit support
* add mutation observer hok
* add hook to read speaking member by observing iframe content
* display speaking member name in call status bar and improve layout
* fix shrining
* add joined call control bar
* remove chat toggle from room header
* change member speaking icon to mic
* fix joined call control appear in other
* show spinner on end call button
* hide call statusbar for mobile view when room is selected
* make call statusbar more mobile friendly
* fix call status bar item align
* add mutation observer hok
* add hook to read speaking member by observing iframe content
* display speaking member name in call status bar and improve layout
* fix shrining
* add joined call control bar
* remove chat toggle from room header
* change member speaking icon to mic
* fix joined call control appear in other
* show spinner on end call button
* hide call statusbar for mobile view when room is selected
* make call statusbar more mobile friendly
* fix call status bar item align
* add mutation observer hok
* add hook to read speaking member by observing iframe content
* display speaking member name in call status bar and improve layout
* fix shrining
* add mutation observer hok
* add hook to read speaking member by observing iframe content
* display speaking member name in call status bar and improve layout
* fix shrining
jared
Krishan
Ajay Bura
Lotus Bot
root