Soundboard v2 — a near-parallel of the custom-emoji image-pack system for
in-call audio clips.
- Data model: 3-tier packs mirroring MSC2545 — room/space pack (state event
io.lotus.soundboard, inherited by child rooms via parent-space aggregation),
global refs (io.lotus.soundboard_rooms), and the personal pack
(io.lotus.soundboard account data; the v1 flat-list content is migrated to the
pack shape on read). New plugins/soundboard/ (readers, SoundboardPack, utils) +
hooks/useSoundboardPacks (useRelevantSoundboardPacks = user U global U room,
deduped). Unit-tested (migration + slug).
- Management: reusable SoundboardPackEditor (name + emoji + per-clip volume +
delete + upload + batched save), power-level-gated for room packs like emoji
packs; a Soundboard page wired into Room + Space settings.
- In-call: CallSoundboard rewritten as a Discord-style grid grouped by pack
(emoji + name tiles), sourcing room+parent-space U personal clips; a Manage
toggle embeds the editors; per-clip volume x master volume on playback.
- Spam guard: host gates on a playing key (fork enforces one clip at a time).
- Control bar: Mute-Screenshare moved next to the Screenshare button.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Renders LaTeX via spec data-mx-maths spans/divs (KaTeX render of the attr,
children as fallback) and conservative $…$ / $$…$$ text detection (escape-aware,
currency-guarded, never inside code/pre). KaTeX + CSS load lazily on first math
(ReactPrism pattern) — verified absent from the eager bundle. Sanitizer
unchanged by design (we render post-sanitize from attr/text; no incoming MathML
accepted). +14 unit tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CallEmbed sets `autoGainControl=false` for the ML noise-suppression tier so
the browser's auto gain control doesn't fight the in-source ML model; the
browser/off tiers keep AGC on.
- Docs: refresh the LOTUS_FEATURES noise-suppression section (browser-native
default, quality-ordered dropdown, DFN3 ML default, attenuation floor,
gate-after-ML, DFN level 60, AGC-off, the reliability fixes) and LOTUS_TODO
P5-30 (mark tuning/reliability/AGC done; record GTCRN as researched-and-deferred).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Model dropdown is now ordered by quality/CPU, best first (DeepFilterNet 3 →
DTLN → RNNoise → Speex); fix RNNoise's inaccurate "High" voice-quality label.
- When a user opts into the ML tier, default to the highest-quality model
(DeepFilterNet 3). The tier default stays browser-native (known-good, best
perceived in testing so far).
- Wire the "Series Suppression" (native-NS-before-ML) toggle into the real call
path — it was applied only in the settings tester, so the tester could sound
better than the actual call. Default it OFF (a single NS stage is best
practice; it's an opt-in test aid).
- isMLDenoiseSupported now also requires WebAssembly, so ML isn't offered on
strict-CSP shells where it would silently fall back to the raw mic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Element Call is now consumed as our self-built fork
(@lotusguild/element-call-embedded); wire up its previously-dormant
capabilities and document the fork as live.
Soundboard (P5-15): a call-bar button plays user-uploaded audio clips into the
call as a real published track (io.lotus.inject_audio) plus local playback.
Clips are uploadable like emoji/sticker packs, stored in io.lotus.soundboard
account data (synced across devices). Gated by a Settings toggle + volume.
Quality controls (P5-31): per-user mic/screenshare bitrate + screenshare
framerate (Settings -> Calls), applied via io.lotus.set_quality clamped to any
room cap. Room admins set caps and hard call-permissions (allow_screenshare /
allow_camera) in Room Settings -> Voice; the call bar hides blocked buttons.
- New: CallSoundboard, useSoundboard, soundboardClips; RoomQuality,
useCallQuality, callQuality (+ unit tests).
- Optimistic-write RoomQuality admin UI (no stale-state clobber).
- Docs: mark EC fork live across README/FEATURES/TODO/BUGS/TESTING; add D2
manual-test steps.
Numeric quality caps are client-cooperative; screenshare/camera permissions are
hard-enforced server-side (see LotusGuild/matrix voice-limit-guard).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coverage work found a 3rd real bug: isMacOS() compared os.name against the
legacy 'Mac OS' string, but ua-parser-js v2 reports 'macOS' — so it was dead,
and Mac users saw "Ctrl + k" instead of "⌘ + k" in the editor toolbar, search,
and settings shortcut hints. Now accepts both 'macOS' and 'Mac OS'.
Suites (via subagent, verified): via-servers (10 — power/popularity server
selection), bad-words (9), syntaxHighlight tokenize (14), plugins/utils
getEmoticonSearchStr (5), imageCompression formatFileSize/isCompressible (5),
user-agent (6, now asserting the fixed behavior).
Full suite now 501 tests, all passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve the eslint/prettier failures from the previous commit (non-blocking
in CI, but real): drop the banned `void` operator on fire-and-forget
transport.send().catch() calls, prefix the now-unused _denoiseNativeNS
param, and run prettier on the touched files.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Switch to @lotusguild/element-call-embedded@0.20.1-lotus.1 (our self-built
fork) and turn on the source-level features it adds:
- #1 denoise CUTOVER: in-source ML denoise (lotusDenoiseSource=1) replaces
the build-time getUserMedia shim — removed the shim injection from
vite.config.js (denoise/ assets still shipped; the processor loads them).
Survives reconnects (fixes A7).
- #2 call-state: CallEmbed consumes io.lotus.call_state; useCallSpeakers /
useRemoteAllMuted prefer it over scraping EC's DOM (DOM fallback kept;
empty payloads ignored).
- #4 focus: CallControl.focusCameraParticipant sends io.lotus.focus_participant
(works during screenshare), replacing the DOM tile-click hack.
- #5 theming: lotusTransparent=1 (native transparent background).
- #6 decorations: LotusDecorationPusher sends each member's decoration URL
via io.lotus.decorations -> rendered on in-call tiles.
#3 soundboard / #7 quality ship dormant (EC-ready; no host UI sends them yet).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- N122: setMediaState resolves on EC's transport ACK instead of waiting for a
DeviceMute state-echo that EC may elide or skip during teardown — which
previously stranded the promise forever and silently skipped the initial
deafen state + first StateUpdate on join. Dropped the single-slot
mediaStatePromiseResolver; onMediaState remains the authoritative sync path.
- N123: focusCameraParticipant now waits for a spotlight videoTile to mount via
a MutationObserver (with a 600ms hard-timeout fallback) instead of a fixed
2-frame delay that EC's React commit can exceed on slower devices.
- N126: PiP position restored from localStorage is shape+finiteness validated,
so corrupt data can't feed NaN into the position math (invalid 'NaNpx' CSS).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CallEmbed watchdog now SELF-HEALS: a genuine ready/joined signal arriving
after the 25s timeout clears the error and notifies subscribers with
undefined, so a slow-but-successful EC load no longer strands the user on
the recovery screen over a live call. Listener dispatch wrapped in try/catch.
- ringtones: synth notes route through a per-session master gain; stop() ramps
it to 0 so the ring is silenced instantly on answer instead of letting the
last scheduled phrase ring out over call audio.
- IncomingCallBanner: ping fires exactly once per incoming call (guarded by
refEventId) instead of re-pinging when ringtone settings change mid-banner.
- focusCameraParticipant: try multiple tile selectors (EC labels vary by
version), defer the tile click past EC's async spotlight layout switch
(rAF x2), and dev-warn when no tile matches so testers get signal.
- uploadContent: a cancelled upload (mx.cancelUpload -> AbortError) is no
longer treated as retryable — previously the retry loop could resurrect an
upload the user just cancelled. Also retry on 408.
- addRoomIdToMDirect/removeRoomIdFromMDirect: guard against a corrupt m.direct
whose values aren't arrays.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CallEmbed: 25s load watchdog that fails fast on iframe error / preparing-error /
timeout instead of hanging on a permanent spinner; additive onLoadError API,
cleared on ready/capabilities/joined.
- CallView: user-visible "call failed to load" overlay with Retry/Leave (folds +
tokens) via a new useCallLoadError hook.
- CallMemberCard: wrap the participant avatar in AvatarDecoration so decorations
render in the call roster (the tile rendered UserAvatar bare while member lists
already wrapped it).
Addresses LOTUS_BUGS item 3 (avatar decorations in calls) and EC iframe failure monitoring.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure formatting reflows (multi-line wrapping of long lines/imports/tables);
no behavior change. Clears the working tree of pending prettier diffs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Four changes to match screenshare full-screen UX for camera feeds:
1. Fullscreen button always visible
CallControls.tsx: remove `screenshare &&` gate — the ⛶ fullscreen
button now appears in camera-only calls, not just during screenshare.
2. Per-participant camera focus (CallControl.focusCameraParticipant)
Finds the target's video tile in the EC iframe DOM via:
[data-testid="videoTile"] / [data-video-fit]
closest ancestor of [aria-label="${userId}"]
Enables spotlight mode if not already active, then clicks the tile
so EC's internal focus handler runs. Falls back gracefully if the
tile is not in the DOM (camera off).
3. MemberGlance participant popup
Clicking a participant avatar in the call status bar now shows a
small menu: "Focus camera" (calls focusCameraParticipant) and
"View profile" (existing behaviour). Previously it opened the
profile immediately with no way to focus the camera.
4. PiP fullscreen button
A ⛶/⊡ icon button appears in the PiP overlay top-right area,
letting users go fullscreen directly from PiP mode without
navigating back to the call room first.
UNTESTED — requires a real multi-participant call to verify tile
clicking behaviour and fullscreen transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace raw error object logging (which may contain Matrix event
payloads, user IDs, or message bodies) with e.message-only strings
in three files:
- CallEmbed.ts: state update and event widget feed errors
- msgContent.ts: image/video element load failures and thumb errors
- RoomInput.tsx: GIF send failure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement a flexible, multi-model noise suppression pipeline for Element Call/LiveKit integration:
- ML Engines: Added support for RNNoise, Speex, DTLN, and DeepFilterNet 3 models.
- Pipeline Architecture: Implemented modular audio processing in lotus-denoise.js, supporting 'Series Suppression' (running browser-native NSNet2 before ML) and a hardware-style Noise Gate.
- UI & UX Enhancements:
- Settings UI: Added model comparison chart with CPU/Quality metadata.
- Tuning: Added Live Microphone Meter for calibrating Noise Gate thresholds.
- Reporting: Added LotusToast system to alert users when ML suppression fails or falls back to raw input.
- Robustness & Quality:
- Capture Fidelity: Removed forced 48kHz capture constraints to allow native-rate capture (solving static issues with high-end audio interfaces).
- Performance: Added WASM SIMD detection with transparent fallback.
- Capability Detection: Added browser feature detection to disable unsupported ML modes.
- Build Integration: Updated Vite config to self-host all model WASM/tflite assets in /denoise/ directory.
Replace the boolean call noise-suppression setting with a 3-way control
(Off / Browser-native / ML beta) in Settings -> General -> Calls.
- Off: noiseSuppression=false to Element Call
- Browser-native: EC's built-in WebRTC suppressor (prior default)
- ML (beta): on-device RNNoise (@sapphi-red/web-noise-suppressor)
Element Call captures the mic inside its iframe and publishes to LiveKit,
so the host can't reach that track; LiveKit's Krisp filter is Cloud-only
(we self-host the SFU) and EC's own RNNoise PR #3892 is unmerged. The ML
tier instead injects a same-origin pre-init shim into the vendored EC
index.html (build/lotus-denoise.js, wired by the lotusDenoise vite plugin)
that patches getUserMedia and routes the captured mic through an RNNoise
AudioWorklet before LiveKit sees it -- the same post-capture pipeline as
#3892, with no EC fork/AGPL/rebase burden. Falls back to the raw mic if
setup fails; keeps echoCancellation/AGC on the raw capture.
- settings.ts: callNoiseSuppression -> 'off'|'browser'|'ml' + legacy
boolean migration (true->browser, false->off)
- CallEmbed/useCallEmbed: tier maps to noiseSuppression param and appends
lotusDenoise=ml (native suppressor off in ML mode)
- vite.config.js: copy RNNoise worklet/wasm + shim into the EC bundle and
inject the shim <script> before EC's module entry
- docs: LOTUS_FEATURES.md, LOTUS_TODO.md (P5-30 done)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Omitting sendNotificationType for call rooms caused Element Call to
default to ring behavior. Now all starting-call events explicitly set
notification (or ring for DMs). Voice channels always get notification.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- useFileDrop: reset drag overlay when mouse leaves browser window
(relatedTarget === null signals viewport exit, counter was getting stuck)
- useDeviceVerificationStatus: add member count to useMemo deps so new
room members' devices get checked, not just initial joined members
- index.css: define --bg-surface-variant used by VoiceMessageRecorder,
MessageSearch, SearchFilters, UserRoomProfile (was falling back to transparent)
- syntaxHighlight: fix Python inline comments — # after space/tab was
treated as plain text; only start-of-line was recognised
- usePresenceUpdater: replace internal baseUrl cast with mx.getHomeserverUrl()
- useLocalMessageSearch: scan all linked timelines via getUnfilteredTimelineSet()
not just the live window, so scrolled-back history is included in E2EE search
- RoomViewHeader: show search button in encrypted rooms — local search is
implemented and handles them; the guard was a holdover from before it existed
- recent-emoji: return emojis in recency order (array is already unshifted on
use) instead of sorting by total usage count
Skipped: media gallery memory leak (needs virtualization refactor),
bookmark race condition (needs queue/lock), Night Light portal coverage
(position:fixed already covers full viewport — not a real bug).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add screenshareAudioMuted state to CallControlState and CallControl
- setSound() now preserves screenshare audio mute when un-deafening
- Add toggleScreenshareAudio() targeting audio[data-lk-source="screen_share_audio"]
- Add ScreenshareAudioButton (volume icon, warns when muted) to controls bar
- Fix unused prevScreenshare variable (ESLint error from prior commit)
- Run Prettier on Controls.tsx and CallControl.ts (CI formatting failures)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove revert-to-grid logic that was overriding EC's natural screenshare
spotlight, causing fullscreen to show user avatars instead of the screen
- Add fullscreen button to call controls (visible when screensharing) that
requests fullscreen on the call embed container
- Add FullscreenButton component with enter/exit SVG icons to Controls.tsx
- PIP mode: sync setPipMode to CallControl; auto-enable spotlight when
screenshare is active in pip so the screenshare fills the window
- Make useCallControlState accept undefined control for safe use in
CallEmbedProvider
- Add package-lock.json to .gitignore (generated by local npm install)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
downloadMedia: on 401 (SW session race or allow_redirect hop stripping
auth), retry via /_matrix/media/v3/ which is public on this homeserver
(allow_public_access_to_media_repo: true). Fixes images not loading
after sending, and avatar 401s in call prescreen tiles.
CallEmbed: inject flex-centering CSS for EC 0.19.4 participant avatar
container so the initial letter is correctly centered in its circle.
CSS class names are scoped to _avatarContainer_1mrho_40 in EC 0.19.4.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CallEmbed: inject :root { color-scheme } into iframe so EC respects Cinny
theme regardless of OS preference (fixes white background in dark mode)
- CallEmbed: store themeKind, update color-scheme CSS on live setTheme() calls
- CallEmbed: catch transport.send() rejection in setTheme() to prevent
unhandled promise rejection when widget not ready yet (fixes REACT-8)
- CallEmbed: html + body both set to background:none so wallpaper shows through
- CallEmbedProvider: apply chatBackground wallpaper style to call embed
container in full-view mode (not PiP) -- wallpapers carry over to calls
- useCallEmbed: pass themeKind through to CallEmbed constructor
- index.tsx: ignoreErrors: [Request timed out] to suppress matrixRTC
heartbeat timeouts (REACT-9) from Sentry noise
- README: document 0.19.4, positioning fix, dark mode fix, wallpaper,
millify Rolldown interop fix, Sentry noise filter
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- millify.ts: use named import { millify as millifyPlugin } instead of
default import to fix Rolldown CJS interop bug where zc.default gets
set to the whole module object instead of the function (mode=1 forces
default=n instead of default=n.default, breaking MembersDrawer)
- useCallEmbed.ts: use getBoundingClientRect() for accurate fixed
positioning; add useEffect to trigger syncCallEmbedPlacement on mount
so embed is positioned before the first resize event
- CallEmbedProvider.tsx: fix [pipMode, callVisible] effect to NOT clear
top/left/width/height when callVisible changes (previously cleared
position set by syncCallEmbedPlacement every time joined changed);
only clear pip-specific styles when actually exiting pip; add debug
console logging for positioning state
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the listenAction wrapper only called preventDefault() to stop
the switch default from firing an error, but it never sent a reply.
The widget transport would then wait for a response until it timed out.
Now the wrapper also calls transport.reply(ev.detail, {}) to return an
immediate success, fixing io.element.join, io.element.device_mute, and
set_always_on_screen.
The base WidgetDriver throws Failed to override function for these
methods. ClientWidgetApi routes update_delayed_event widget actions to
cancelScheduledDelayedEvent, restartScheduledDelayedEvent, or
sendScheduledDelayedEvent. Without these overrides every delayed-event
refresh from element-call fails, causing MembershipManager to drop the
call after retries.
Also make listenAction auto-call preventDefault so io.element.join and
other custom widget actions return success. Add set_always_on_screen
handler so element-call PiP requests are acknowledged.
- RoomTimeline.tsx: add eslint-disable comment for intentional eventsLength
dep on timelineSegments useMemo (needed to detect in-place timeline mutations)
- Remove ~47 stale eslint-disable-next-line comments across 28 files for rules
that are now off in the flat config (no-param-reassign, jsx-a11y/media-has-caption,
react/no-array-index-key, etc); run prettier to reformat
- vite.config.js: move manualChunks from rollupOptions.output to
rolldownOptions.output so Rolldown (Vite 8) actually applies it; main bundle
drops from 3.5 MB to 814 kB gzip-248 kB, matrix-sdk gets its own 1.16 MB
cacheable chunk
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- react 18.2.0 to 19.2.6
- react-dom 18.2.0 to 19.2.6
- @types/react 18.2.39 to 19.2.15
- @types/react-dom 18.2.17 to 19.2.3
React 19 breaking changes fixed:
- useRef<T>(null) now returns RefObject<T | null>; cast to
RefObject<T> at 16 component call sites (safe, runtime unchanged)
- useRef<T>() without arg no longer valid; add | undefined>(undefined)
in useDebounce, useFileDrop, useThrottle, useVirtualPaginator hooks,
RoomInput, RoomTimeline, and ClientNonUIFeatures
- useReducer<typeof reducer> 1-arg form removed; drop explicit type arg
in useForceUpdate (inferred from reducer function)
- global JSX namespace removed; import type { JSX } from react in
react-custom-html-parser.tsx
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- typescript 5.9.3 to 6.0.3
- moduleResolution Node to bundler (correct for Vite projects)
- target/lib ES2016 to ES2020 (enables flatMap, Promise.allSettled)
- Fix global to globalThis in initMatrix.ts (browser env)
- Fix EventEmitter default to named import in CallControl.ts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Resolves all TS2345/TS2347/TS7006 type errors introduced by stricter TypeScript 5.x.
Fix Icons.Settings to Icons.Setting, cast account data returns, fix implicit any.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prettier: auto-formatted 103 files to fix baseline. Prettier check in CI
is now a hard gate (removed continue-on-error).
Brotli: installed libnginx-mod-http-brotli-filter/static. Enabled in nginx
with brotli_static on for pre-compressed assets and comp_level 6.
Sentry releases: deploy script now exports VITE_APP_VERSION=<git-short-sha>
before building so each Sentry release maps to an exact commit.
CI also passes github.sha as VITE_APP_VERSION.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Lobby, Explore/FeaturedRooms/PublicRooms, Inbox/Notifications/Invites are
now lazy-loaded via React.lazy so they only enter the bundle when navigated
to. Main bundle: 2547 kB → 2472 kB (gzip 637 → 618 kB).
Spoiler aria-pressed was initialised to false (revealed); changed to true
so the spoiler starts hidden, matching CSS logic (aria-pressed=true →
color:transparent).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CallEmbed: fix memory leak — mx event listeners were never removed
because dispose() called .bind(this) again, creating new function
objects. Now uses arrow class fields so start()/dispose() share the
exact same reference.
- callPreferences: toggleVideo is a no-op when cameraOnJoin=false,
preventing internal state drift from the returned value.
- CallControls: PTT key guard now blocks on SELECT elements and walks
the DOM for inherited contentEditable to prevent key interception
inside dropdowns and custom editors.
- RoomInput: GIF fetch validates Giphy CDN domain allow-list,
HTTP Content-Type header, and enforces 20 MB size cap.
EC 0.19.3 changed the toolbar layout. The old previousElementSibling
traversal from the leave button pointed at wrong elements:
- settingsButton was finding the raise-hand button
- reactionsButton was finding the screenshare button
Fix: use stable selectors instead:
- settingsButton: data-testid=settings-bottom-center (new in EC 0.19.3)
- reactionsButton: [class*=raiseHand] (CSS module class, consistent in 0.19.x)
jared
Krishan
Ajay Bura
Lotus Bot
root