LotusGuild/cinny
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7939dc92d4c41c5b4b5c4be34d5392869894496f
cinny/LOTUS_TODO.md
T
jared 7939dc92d4 docs(call): cover soundboard/quality/permissions in user-facing docs 
- README "Calls & Voice": add the in-call soundboard, per-user call quality
  settings, and admin room call-permissions bullets.
- LOTUS_TODO: mark the soundboard UI as built (was "cinny UI remains / dormant").
- HANDOFF_ELEMENT_CALL_FORK: add a COMPLETE status banner to the §12.1 host
  checklist; fix stale denoise specifics (all four models are in-source;
  flag is lotusDenoiseSource=1, not lotusDenoise=ml).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 22:43:49 -04:00

53 KiB
Raw Blame History

Lotus Chat — Work Backlog

Repo: lotus branch at https://code.lotusguild.org/LotusGuild/cinny
Deploy: push to lotus → CI → auto-deploy to chat.lotusguild.org (~11 min)

⚠️ TDS DESIGN LAW — READ BEFORE TOUCHING ANY UI

ALL Lotus Terminal Design System (TDS) styling — colors, animations, glows, borders, fonts, spacing — MUST come exclusively from /root/code/web_template/base.css CSS variables. Do NOT hardcode hex values. Do NOT invent new variable names. Do NOT deviate from the design tokens defined in that file. The canonical variable reference: --lt-accent-orange, --lt-accent-cyan, --lt-accent-green, --lt-glow-orange, --lt-box-glow-*, --lt-border-color, etc. Reference implementation for code patterns: /root/code/tinker_tickets/ (markdown.js, base.js, ticket.css) This rule applies to EVERY task in this file without exception.

🧩 NATIVE-CINNY LAW — EVERY FEATURE MUST FEEL LIKE STOCK CINNY

Every feature we implement must feel native to the upstream Cinny app — indistinguishable from something the Cinny team would have shipped. Reference: https://github.com/cinnyapp/cinny.

Concretely this means:

  • Use the folds design system, not bespoke UI. Build with folds primitives (Button, Chip, IconButton, Menu, MenuItem, Dialog, Modal, Input, Switch, Badge, SettingTile, SequenceCard, etc.) and folds tokens (color.*, config.space.*, config.radii.*, config.borderWidth.*). No hardcoded hex/rgba() for UI chrome, no invented/undefined CSS variables.
  • Match Cinny's existing patterns. Before adding UI, find the closest existing Cinny component/flow and mirror it (e.g. a new dropdown uses Button+PopOut+Menu+MenuItem like the rest; a new modal has a Header with a close IconButton; a new setting is a SettingTile inside a SequenceCard). Consistency with stock Cinny beats personal style.
  • Lotus-custom additions should be unobtrusive and fit Cinny's visual language, spacing, and interaction conventions — a stranger using Cinny should not be able to tell which features are ours.

The ONE exception: explicit Lotus Terminal Design System (TDS) features, which intentionally have their own distinct look and follow the TDS Design Law above. TDS styling is opt-in (only active in Lotus Terminal mode); everything else must look and feel like native Cinny.

Completed features are documented in LOTUS_FEATURES.md.

✅ Done — Awaiting Verification

Built and gate-green; verify per LOTUS_TESTING.md, then they graduate to LOTUS_FEATURES.md. (Bug-side fixes awaiting verification live in LOTUS_BUGS.md.)

Feature Test guide
Full-Screen Camera Broadcasts (per-participant focus) A5 / G2
Advanced search filters (sender/date/has-link/has:image·file·video/pinned/recent) K2 / M1 / M2 / M4
Custom Accent Color Picker (non-TDS themes) M3
5 Color Theme Presets (Cyberpunk/Ocean/Blood Red/Matrix/Midnight) M5
Intersection-based lazy media loading H1
Context-aware thumbnail previews H2
Desktop — proactive update notifications (Tauri) J1
Remind Me Later K1
Mobile Bookmarks access E5
In-Call Soundboard (P5-15, uploadable clips → real call inject) D2-7
Call Quality Controls (P5-31, user + room-admin caps) D2-8
Call Permissions (P5-31, hard server-side screenshare/camera policy) D2-9

Legend:

  • [AUDIT REQUIRED] — at least one assumption needs code/server verification before implementing
  • [SERVER CHECK] — depends on a Synapse feature or MSC; verify on matrix.lotusguild.org
  • [LOW PRIORITY] — implement after all higher-priority items
  • [EXTREME COMPLEXITY] — multi-sprint, plan separately before touching
  • [BLOCKED] — cannot build until a server upgrade, upstream MSC, or dependency resolves
  • [IMPROVE] — feature exists in upstream Cinny; this task enhances it for Lotus Chat

Status: [ ] pending · [~] in progress · [x] completed

Server Capabilities (as of June 2026)

  • Homeserver: matrix.lotusguild.org
  • Synapse version: 1.155.0 (2026-06-18) — fully up to date; last version for Debian 12 (LXC 151 already on Debian 13 Trixie)
  • Matrix spec: up to v1.12 formally; newer MSC features via unstable_features

Confirmed facts

Finding Impact
MSC flags ON: msc4140 · msc3771 · msc3440.stable · msc4133.stable · simplified_msc3575 · msc4222 · msc3266 · msc3401_matrix_rtc All safe to use now
MSC flags OFF: msc4306 (thread subscriptions) · msc3882 · msc3912 · msc4155 These features are BLOCKED
MSC3266 room summary: flag msc3266_enabled: true set but GET /v1/rooms/{id}/summary still returns 404 (M_UNRECOGNIZED) Room Preview BLOCKED — endpoint not implemented in Synapse 1.155
MSC3892 relation redaction: not in flags Reaction Redaction feature BLOCKED
MSC4260 report user: POST /_matrix/client/v3/users/{userId}/report returns 200 ✅ Report User UNBLOCKED — endpoint live since Synapse 1.133; ready to build
MSC4151 report room: HTTP 405 on GET = endpoint exists (POST only) Report Room live ✅
folds AvatarImage does NOT accept children Add frame/overlay inside UserAvatar.tsx itself — optional frameName prop
No in-app toast system exists (was) Built ToastProvider + Jotai queue; at App.tsx:65
useUnverifiedDeviceCount() hook exists src/app/hooks/useDeviceVerificationStatus.ts:65-106
Voice player: AudioContent.tsx:44-223 Playback rate on hidden <audio> at line 217
CallControl.setMicrophone(bool) at CallControl.ts:206-212 For AFK auto-mute
CallControl.toggleSound() at CallControl.ts:230-251 Push-to-deafen — just wire a hotkey to this
matrix-js-sdk has NO arbitrary profile field methods Use mx.http.authedRequest() for MSC4133
Sanitizer (sanitize.ts) allows table, div, span, a, code, hr LFG HTML card is safe locally; test on Element/FluffyChat
Sanitizer STRIPS <math>/MathML tags Math/LaTeX task must also modify sanitizer
Service worker EXISTS at src/sw.ts Quick-reply task: add notificationclick handler
knockSupported() utility exists at matrix.ts:376-391 Knock UX: only need "Request to Join" in RoomIntro.tsx
KeywordMessages.tsx already has custom keyword push rules Full push rule editor: only non-keyword rule types need new UI
getMatrixToRoom() in matrix-to.ts generates invite URLs Invite link: just add QR code to room settings
Cindy CANNOT inject audio into EC call stream UNBLOCKED by EC fork — io.lotus.inject_audio widget action publishes a clip as a real call track In-call soundboard CAN now mix into the call (no longer local-only); needs cinny UI to drive the action
Folds uses vanilla-extract in non-TDS, NOT CSS custom properties Custom accent color: must create new vanilla-extract theme variant dynamically
Theme presets need ~50 CSS custom properties each Significant design work before coding
useCallSpeakers.ts CSS MutationObserver polling Visual speaking indicator: TDS ring animation on top of existing data
MSC3489/3672 live location: BOTH false on server Live Location BLOCKED

Key File Reference

What you need File Lines
Global keydown hook src/app/hooks/useKeyDown.ts whole file
Room navigation src/app/hooks/useRoomNavigate.ts 19-72
All room IDs atom src/app/state/room-list/roomList.ts allRoomsAtom
Room unread counts src/app/state/room/roomToUnread.ts roomToUnreadAtom
Overlay portal provider src/app/pages/App.tsx 65
Portal container div index.html 101
Room settings tabs src/app/features/room-settings/RoomSettings.tsx 27-56
State event read/write pattern src/app/features/common-settings/general/RoomEncryption.tsx 42-52
Power level checker src/app/hooks/usePowerLevels.ts whole file
Slash command registration src/app/hooks/useCommands.ts 140-537
Chat background picker src/app/features/settings/general/General.tsx 945-981
Chat backgrounds definition src/app/features/lotus/chatBackground.ts whole file
Matrix.to URL builder src/app/plugins/matrix-to.ts getMatrixToRoom()
Media event content types src/app/types/matrix/common.ts 46-91
Media URL conversion src/app/utils/matrix.ts mxcUrlToHttp()
Message pagination (search) src/app/features/message-search/useMessageSearch.ts 74-121
Infinite pagination pattern src/app/features/message-search/MessageSearch.tsx 234-365
Poll event format src/app/components/message/content/PollContent.tsx 1-320
Theme class application src/app/hooks/useTheme.ts 25-60
Animations file src/app/styles/Animations.css.ts whole file
Message status (EventStatus) src/app/features/room/message/Message.tsx 84-142
Call member change events src/app/hooks/useCall.ts 37-52
Mic control in calls src/app/plugins/call/CallControl.ts 206-212
Device verification hook src/app/hooks/useDeviceVerificationStatus.ts 65-106
Knock room support check src/app/utils/matrix.ts 376-391
Room join button location src/app/components/room-intro/RoomIntro.tsx 25-119
Notification mute via push rules src/app/hooks/useRoomsNotificationPreferences.ts 110-150
Message text body CSS src/app/components/message/layout/layout.css.ts 182-205

Priority 3 — Higher complexity / lower daily frequency

[ ] P3-4 · Accessibility Improvements (WCAG 2.1 AA)

What: Comprehensive audit and fix pass targeting the critical user paths:

  • Room list navigation (keyboard-only)
  • Reading messages in the timeline (screen reader announces new messages)
  • Composing and sending a reply
  • Opening and closing modals (focus trap, return focus)
  • ARIA labels on all icon-only buttons

Scope: Do NOT attempt to make every corner of the app AA-compliant in one pass — focus on the golden path (open app → find room → read → reply → send).
[AUDIT REQUIRED] — Run an automated audit first: npx axe-core or browser DevTools accessibility tree. Document every violation before writing a single line of code. Prioritize by severity (critical > serious > moderate).

Investigation Findings:

  • Root Cause: Inconsistent focus management, missing aria-live regions for dynamic timeline updates, and sparse global keyboard shortcuts.
  • Approach: Standardize focus-trap-react usage (reference RoomNavItem.tsx). Add aria-live regions to the timeline. Expand useKeyDown.ts for section navigation shortcuts.
  • Complexity: Medium-High (audit is the main work).

[ ] P3-8 · Thread Panel (full side drawer)

⚠️ LARGEST FEATURE — requires its own planning session before implementation.
What: A right-side drawer for threaded conversations. Currently "Reply in Thread" exists but there is no panel to read or write thread replies.

Features:

  • Click "Reply in Thread" → opens thread drawer on the right
  • Thread root event shown at the top of the panel
  • Full message rendering for all in-thread replies (reuse timeline components)
  • Reply input at the bottom (full composer with formatting, emoji, etc.)
  • Unread count badge on the thread button in the main timeline
  • Keyboard shortcut to close thread panel

Architecture:

  • New Jotai atom: activeThreadEventId: string | null
  • New component: src/app/features/room/thread/ThreadPanel.tsx
  • Rendered alongside RoomView as a conditional right panel (mirror the members drawer pattern)
  • Filter events in timeline to m.thread relation for the active root event ID
  • Shares the same mx client and room reference as the main timeline

[AUDIT REQUIRED] — Deeply audit how m.thread relation events are currently stored and retrieved in the matrix-js-sdk. Understand the thread aggregation API: GET /rooms/{roomId}/relations/{eventId}/m.thread. Check if RoomTimeline.tsx currently filters out thread replies from the main timeline (it should — confirm).

Investigation Findings:

  • Root Cause: Current m.thread events are treated as standard m.room.message events and rendered in the main timeline.
  • Approach: Introduce new Jotai atom activeThreadEventId. Create ThreadPanel.tsx. Update RoomTimeline.tsx to filter out thread relations (m.relates_to). Implement aggregation fetch using GET /rooms/{roomId}/relations/{eventId}/m.thread. Use thread.timelineSet directly for the most accurate thread view.
  • Complexity: High.

Priority 4 — Specialized, high complexity, or low priority

[ ] P4-7 · Virtualized Infinite Scroll for Search Results

What: Replace the manual "load more" button with an automated, virtualized infinite scroll for search results.
Approach: Utilize @tanstack/react-virtual in MessageSearch.tsx to handle the nextToken automatically as the user scrolls.

[ ] P4-8 · Encrypted Message Search Indexing & Caching

What: Implement a persistent local cache for search results, optimized for encrypted rooms.
Approach: Use IndexedDB to store search metadata (event IDs, timestamps) to prevent redundant server-side decryption/fetching.

[ ] P4-1 · Thread Notification Mode Per-Thread (MSC3771)

Spec: MSC3771 (stable). Depends on Thread Panel (#P3-8).
What: Per-thread notification toggle: "All messages" vs "Mentions only". Accessible from the thread panel header. Tracks unread counts separately per thread.
[AUDIT REQUIRED] — Implement after Thread Panel. Requires understanding how the SDK tracks per-thread unread counts.
Complexity: Medium (after thread panel exists).

[ ] P4-2 · Thread Subscriptions (MSC4306) [BLOCKED]

Spec: MSC4306 (Synapse experimental). Depends on Thread Panel (#P3-8).
What: "Follow thread" button to receive notifications for a thread you haven't posted in. Uses MSC4306 subscription endpoint.
[SERVER CHECK] — org.matrix.msc4306 = false on matrix.lotusguild.org — BLOCKED until server enables it.
Complexity: Medium (after thread panel exists).

[ ] P4-4 · Math / LaTeX Rendering in Messages (LOW PRIORITY)

Spec: CS-API §11.5 (stable) — formatted_body can contain LaTeX.
What: Render $...$ or $$...$$ LaTeX expressions in message bodies. Use KaTeX (lightweight, ~100KB, renders server-side-compatible CSS). Must gracefully fall back to raw LaTeX text if KaTeX fails.
Note: This is LOW PRIORITY — only useful for academic/technical communities. Implement last.
[AUDIT REQUIRED] — Confirm KaTeX bundle size impact on the Vite bundle. Check if matrix-js-sdk's HTML sanitizer strips LaTeX before it reaches the renderer. The formatted_body sanitization pipeline is the main risk here. (Confirmed: sanitizer STRIPS <math> tags — must be patched alongside the renderer.)
Complexity: Low-Medium.

[ ] P4-5 · Live Location Sharing (MSC3489 + MSC3672) (LOW PRIORITY, HIGH COMPLEXITY) [BLOCKED]

Spec: MSC3489 + MSC3672. Implemented in Element Web.
Note: Static location sharing is already implemented. This adds live/real-time GPS beacons. Very low priority per user preference.
What: Start sharing live location → creates m.beacon_info state event → client posts m.beacon events on a timer → other users see your position update live on a map.
[SERVER CHECK] — org.matrix.msc3489 = false AND org.matrix.msc3672 = false on matrix.lotusguild.org — BLOCKED.
Complexity: High. Requires background geolocation API + live map rendering.

[~] P4-6 · OIDC / SSO Next-Gen Auth (MSC3861) — CLIENT-SIDE BUILT, awaiting live verification

Spec: MSC3861 / MSC2965, Matrix spec v1.15. OAuth2-native auth via a Matrix Authentication Service (MAS).
Scope decision (2026-06): CLIENT-ONLY. We implemented OIDC login in the Lotus client so it can sign into next-gen homeservers (mozilla.org, eventually matrix.org). We deliberately did not convert lotusguild's own Synapse to MAS (no account migration; lotusguild keeps password + legacy Authelia SSO).
Built (matrix-js-sdk already ships the OIDC API; this was wiring):

  • Discovery: cs-api.ts getOidcIssuer() (stable m.authentication + msc2965). Flow hint: useParsedLoginFlows getOidcCompatibilityFlag() (MSC3824).
  • Login: pages/auth/oidc/{oidcConfig,oidcLoginUtil,oidcState}.ts (dynamic registration + cache, PKCE authorize), login/OidcLogin.tsx, issuer-gated Login.tsx.
  • Callback: oidc/OidcCallback.tsx + App.tsx short-circuit (non-hash redirect path).
  • Session/refresh: state/sessions.ts OIDC fields, client/{oidcTokenRefresher,oidcLogout}.ts, initMatrix.ts wiring.
  • Account mgmt: settings/account/OidcManageAccount.tsx.
  • 13 unit tests (discovery/flow/session/cache/callback parsing). All gates green. Awaiting verification (needs a real MSC3861 server — lotusguild is NOT one): deploy + log into mozilla.org (requires adding mozilla to the deployed config.json homeserverList + its domains to the CSP connect-src/img-src — see below), OR run a local matrix-authentication-service + Synapse msc3861 dev loop.
    To enable the mozilla.org test: add to matrix/cinny/config.json homeserverList "mozilla.org", and to the nginx CSP connect-src/img-src: https://mozilla.org https://mozilla.modular.im https://chat.mozilla.org https://vector.im.

Priority 5 — Gamer / Aesthetic / Customization

[MOVED] P5-9 · LFG (Looking for Group) Command → LotusBot

Decision: Implemented as !lfg in LotusBot rather than a client slash command. Bot-side rendering works consistently across all Matrix clients; client-side enhanced cards would only be visible to Lotus Chat users and require sanitizer auditing. The bot can also support richer flows (list active LFGs, DM interested players, auto-expire posts).

[~] P5-15 · In-Call Soundboard — IMPLEMENTED (⚠️ awaiting live verification, D2-7)

What: Soundboard button in the call controls bar → popout grid of the user's clips; clicking one plays it into the call as a real published track (peers hear it) and locally (presser hears it). Clips are user-uploadable, just like custom emojis/stickers.
🔱 [EC-FORK] Fork side + cinny side DONE. The fork ships io.lotus.inject_audio (LotusWidgetActions.InjectAudio, allow-listed in widget.ts), armed via the lotusAudioInject=1 flag; it publishes a clip as a separate LiveKit track — a real in-call soundboard mixed into the call, not local-only. cinny now drives it.
Shipped (cinny):

  • Clips stored in io.lotus.soundboard account data → synced across devices like emoji/sticker packs (useSoundboard hook; AccountDataEvent.LotusSoundboard).
  • Upload audio (≤1 MB, ≤40 clips) → mx.uploadContent → mxc; play resolves mxc → authed download → blob: object URL (the widget can't fetch authenticated media itself) → control.injectAudio(url, volume) + local playback.
  • CallSoundboard.tsx popout in the call bar (upload / play / delete), gated on the soundboardEnabled setting (Settings → General → Calls, + volume slider).
    Remaining: a dedicated Settings management page (optional — upload/delete already live in the popout); a small default clip set; live verification (D2-7). Files: utils/soundboardClips.ts, hooks/useSoundboard.ts, features/call/CallSoundboard.tsx, plugins/call/CallControl.ts#injectAudio.
    Complexity: Medium — done.

[~] P5-20 · Quick Reply from Browser Notification

What: Inline reply field in browser notification toasts via Notification Actions API. Reply sends as threaded reply to the triggering message.
[AUDIT REQUIRED] (1) Verify browser Notification Actions API support in target browsers. (2) Confirmed: service worker EXISTS at src/sw.ts — add notificationclick handler there.
Complexity: Medium-High.
Partial Fix Applied ⚠️ UNTESTED: Notifications now (a) show the real message body (username: message instead of "New inbox notification from..."), (b) click navigates directly to the room at the specific event (not the inbox), (c) window.focus() called on click so the tab comes to front, (d) reminder toasts also link to the specific event. Full inline-reply via Notification Actions API still needs the SW push+notificationclick pipeline (requires switching from new Notification() to showNotification() through the SW).

[x] P5-30 · Advanced ML Noise Suppression (Krisp-style)

What: High-end background noise cancellation using a pre-trained ML model (RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
Shipped: 3-tier setting (Off / Browser-native / ML) in Settings → General → Calls.
🔱 [EC-FORK] DONE — moved in-source (2026-06). ML denoise is now a first-class audio stage inside the forked Element Call: a LiveKit TrackProcessor<Audio> activated by lotusDenoiseSource=1 (cinny sets it when ML is selected). The old build-time getUserMedia/index.html monkeypatch is removed. Because EC re-runs the processor on every (re)publish, denoise now survives reconnects and mic-device switches — this is the A7 fix (see LOTUS_BUGS.md A7, LOTUS_TESTING.md §D2-1). The processor degrades to the raw mic rather than going silent.
Key decision: LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU); EC's own RNNoise PR #3892 is unmerged. Owning the fork let us implement the in-source stage directly.

Models — all in-source in the fork:

  • RNNoise (48 kHz, default) · Speex (48 kHz) · DTLN (16 kHz) · DeepFilterNet 3 (48 kHz) — all four wired and selectable.
  • Open verification: real-call audio-quality comparison across the four models (RNNoise output is known-weak). Track under the denoise quality project, LOTUS_TESTING.md §D2-1 / J2.
  • Desktop-only / HW-gated (future): FRCRN or NVIDIA Maxine (RTX/Tensor only) — impossible in-browser; would run in the Tauri Rust backend + bridge a virtual mic into the webview. Detect capability; web falls back to RNNoise.
  • Excluded: Krisp (LiveKit Cloud only); FRCRN/Maxine on web (GPU/server-bound).

[~] P5-31 · Granular Voice & Screenshare Quality Controls — IMPLEMENTED (⚠️ awaiting live verification, D2-8)

What: Let users (and room admins) adjust audio bitrate and screenshare bitrate/framerate.
🔱 [EC-FORK] Fork side + client side DONE. The fork ships io.lotus.set_quality (LotusWidgetActions.SetQuality) that applies audio/screenshare encoding params (RTCRtpSender.setParameters, all simulcast encodings, re-applied on TrackUnmuted/republish) inside EC. cinny now drives it.

Shipped (cinny):

  1. User settings (Settings → General → Calls): Microphone Bitrate, Screenshare Bitrate, Screenshare Framerate (callAudioBitrate / screenshareBitrate / screenshareFramerate).
  2. Room-admin caps: io.lotus.room_quality state event (StateEvent.LotusRoomQuality) + RoomQuality.tsx in Room Settings → General → Voice (mirrors RoomVoiceLimit).
  3. Apply logic: useCallQuality (wired in CallEmbedProvider's CallUtils) builds min(user setting, room cap) and sends io.lotus.set_quality on join / when settings change (utils/callQuality.ts, unit-tested).

Server-side enforcement (DONE — matrix repo): extended voice-limit-guard.py (LXC 151) to also read io.lotus.room_quality and hard-enforce a publish-source policy for ALL clients.

  • Reality (researched, primary-source, LiveKit 1.9.11): numeric bitrate/fps caps cannot be hard-enforced server-side — LiveKit is a pure SFU (forwards, never transcodes); there is NO bitrate/fps field in the JWT grant, RoomConfiguration, server limit: config, or any admin RPC, and stock Element Call ignores room metadata / custom claims for publish quality. So numeric caps stay cooperative (our fork honors them via min() → set_quality, already shipped).
  • What IS hard-enforced cross-client: VideoGrant.canPublishSources. The guard holds the LiveKit secret, so when io.lotus.room_quality sets allow_screenshare:false / allow_camera:false it re-signs the issued JWT with a narrowed source list → the SFU refuses those tracks for every client (Element, FluffyChat, our fork). Mic always kept. Fail-open; unit-tested (livekit/test_voice_limit_guard.py). Admin UI: Room Settings → Voice → Call Permissions switches. cinny also hides the blocked buttons.
  • Live (mid-call) enforcement — DONE: the JWT re-sign covers new joins; for participants already in the call, a background reconcile loop in the guard calls LiveKit UpdateParticipant every ~3 s to narrow canPublishSources, which unpublishes an in-progress screenshare/camera server-side for all clients and blocks re-publish (verified LiveKit 1.9.11 auto-unpublishes on permission narrowing). Only removes forbidden sources (never grants), preserves other permission flags, no-ops once compliant. So flipping a room audio-only kills live cameras/screenshares within ~one interval.
  • Not enforceable / deferred: numeric server enforcement (impossible — see above); screenshare resolution control (set_quality covers bitrate + framerate; resolution needs a getDisplayMedia hook inside the fork).

Complexity: DONE — client (cooperative numeric caps) + server (hard publish-source policy). Only the physically-impossible numeric server enforcement is out of scope.

[ ] P5-35 · Desktop — Notification Click Opens Room (DEFERRED)

What: Clicking a system tray notification navigates to the relevant room. Quick-reply from the notification toast would send the reply without opening the window.
Status: Deferred — tauri-plugin-notification has no Rust click/action callback API. Quick-reply would need a custom WinRT toast activator + COM registration, which can't be compile-tested without a Windows build environment.
Note: Tray icon and matrix: deep links already bring the window forward on most interactions. Revisit when tauri-plugin-notification gains click handler support upstream.
Complexity: High (platform-specific native code required).

[ ] P5-36 · Desktop — Windows Jump List (DEFERRED)

What: Right-clicking the taskbar icon shows a jump list with recent/favorite rooms for quick navigation.
Status: Deferred — implementing the Windows COM jump list API in Tauri requires iterating on C++/COM code that can only be compile-checked on Windows, making blind CI iteration impractical.
Action when unblocked: Revisit when a Tauri plugin abstracts the Windows Shell ICustomDestinationList interface, or when a Windows build environment is available for local iteration.
Complexity: High (Windows-only native COM).

[ ] P5-41 · Desktop — Native WinRT Toast Notifications

What: Replace emulated notifications with native WinRT Toast notifications.
Approach: Implement native WinRT Toast integration using windows-rs to enable full Action Center integration, including native Quick Reply functionality.

[ ] P5-42 · Desktop — Persistent Background Sync

What: Maintain light connection to homeserver when WebView2 is suspended.
Approach: Implement a headless Rust sidecar to fetch unread counts/notifications while the webview is suspended to ensure instant notification delivery.

[ ] P5-43 · Desktop — System Media Transport Controls (SMTC)

What: Integrate with Windows SMTC for volume flyout call/media control.
Approach: Use Windows SMTC API to expose call status, mic mute/unmute, and media controls to the Windows volume flyout/media overlay.

[ ] P5-44 · Desktop — Taskbar Thumbnail Toolbar

What: Add persistent call controls to the taskbar preview.
Approach: Implement a COM thumbnail toolbar in the application preview window, featuring Mute/Deafen/End Call buttons.

[ ] P5-46 · Desktop — System Power Management (Call Continuity)

What: Prevent system sleep/hibernate during active calls.
Approach: Use Tauri/Rust power-manager or platform-specific APIs to block system power saving states while a voice/video session is active.

[ ] P5-47 · Desktop — TDS-Styled Native Window Chrome

What: Replace system titlebar with custom Lotus TDS chrome.
Approach: Configure Tauri window (decorations: false) and implement custom, TDS-token compliant titlebar controls (Close/Max/Min) for a cohesive UI.

[ ] P5-48 · Desktop — Native File System Drag-and-Drop Improvements

What: Enhance drag-and-drop support for Windows.
Approach: Improve handling for Windows file shortcuts, recursive folder uploads, and shell-integrated "Send To" context menu actions.

[ ] P5-49 · Desktop — Network Awareness (NCSI Integration)

What: Proactively detect Windows network connectivity changes.
Approach: Integrate with the Windows Network Connectivity Status Indicator (NCSI) API to improve offline mode transition latency and network recovery.

[ ] P5-50 · Desktop — Windows Hardware-Accelerated Media Pipeline

What: Replace standard browser decoding with native Windows Media Foundation.
Approach: Leverage DirectShow/Media Foundation to offload video/audio decoding from the CPU to the GPU, significantly reducing power consumption and latency during calls.

[ ] P5-51 · Desktop — Federated "Identity Contexts" (Isolation Manager)

What: Compartmentalize sessions, local databases, and caches into isolated "Contexts."
Approach: Implement a zero-leak boundary for personas (e.g., Work vs. Personal) by isolating IndexedDB, filesystem caches, and session persistence per context.
Priority: Extreme Low (Multi-sprint/Architectural).

[~] P5-52 · Desktop — Room-Level Sync Governor (Performance Control) [STILL_CONSIDERING]

What: Granular sync tuning for individual rooms.
Approach: Allow per-room overrides for sync frequency and event type filtering (e.g., disable read receipts/typing in heavy rooms) to optimize performance. Implementation requires careful UX to prevent complexity fatigue.

[ ] P5-53 · Desktop — Local-Only "Scripting" Plugin System (Tampermonkey-like)

What: A sandboxed environment for local execution of user scripts on Matrix events.
Approach: Implement a WASM-based execution engine that allows users to write local-only, client-side scripts to interact with incoming Matrix events, trigger sounds/notifications, or inject custom UI elements based on event payload rules. Designed for privacy — all logic runs exclusively on the local machine.

[ ] P5-55 · Desktop — Composer Toolbar Drag-and-Drop Reordering

What: Allow users to reorder toolbar icons via drag-and-drop.
Approach: Extend the current settings-based toolbar toggle system to include a drag-and-drop UI mode in the composer settings, allowing users to personalize their icon order.

[ ] P5-56 · Desktop — Windows "Focus Assist" (DND) Sync

What: Automatically toggle notification state based on Windows Focus Assist.
Approach: Integrate with the Windows NotificationCenter / Focus state via Tauri/Rust to automatically enable/disable Lotus Chat's internal notification suppression mode when Windows Focus Assist is toggled.

[ ] P5-57 · Desktop — Visual Draft Persistence Indicator

🚀 Features to Add

  • Mobile Audit: Comprehensive audit of all features in LOTUS_FEATURES.md for mobile PWA usability and layout responsiveness.

Blocked Features

These features are confirmed desirable but cannot be built until the listed dependency is resolved. Check back after each Synapse upgrade — re-run /matrix/client/versions and unstable_features to see if they've become available.

[BLOCKED] · Live Location Sharing (MSC3489 + MSC3672)

Blocked by: org.matrix.msc3489 = false AND org.matrix.msc3672 = false on matrix.lotusguild.org (confirmed from unstable_features).
What it would do: Real-time GPS beacon streaming upgrading the existing static location share.
Action when unblocked: Both MSCs must be enabled on the homeserver before any client work.

[BLOCKED] · Reaction / Relation Redaction (MSC3892)

Blocked by: org.matrix.msc3892 = false on matrix.lotusguild.org
What it would do: Cleanly remove a reaction without redacting the parent message.
Current behavior: Full event redaction — acceptable fallback, no user-facing issue.
Action when unblocked: Find onReactionToggle redaction call site; swap in MSC3892 endpoint with fallback.

[BLOCKED] · Room Preview Before Joining (MSC3266)

Blocked by: GET /_matrix/client/v1/rooms/{roomId}/summary returns M_UNRECOGNIZED 404 — endpoint not implemented in Synapse 1.155. Config flag msc3266_enabled: true is set but has no effect; Synapse appears not to have shipped a stable implementation at the v1 path. Verified 2026-06-18.
What it would do: Show room name, topic, avatar, member count before joining.
Action when unblocked: Re-test after each future Synapse upgrade.

[BLOCKED] · Thread Subscriptions (MSC4306)

Blocked by: org.matrix.msc4306 = false on matrix.lotusguild.org
What it would do: Follow a thread without posting; get notifications for replies.
Action when unblocked: Add "Follow thread" button in the thread panel header (depends on #P3-8 Thread Panel).

[DONE] · Report User (MSC4260) ✅

Previously blocked by: Server spec v1.12, but POST /_matrix/client/v3/users/{userId}/report was confirmed 200 on 2026-06-18 (live since Synapse 1.133.0).
What it does: Reports a specific user to homeserver admins (separate from reporting a message).
Note: Report Message already exists in upstream Cinny. This adds Report User to the profile panel.
Implemented 2026-06-18: ReportUserModal.tsx added at src/app/features/room/ReportUserModal.tsx. Button wired into UserRoomProfile.tsx between UserModeration and UserDeviceSessions (hidden for own profile). Category dropdown + reason text, inline success/error feedback, auto-close 1500ms after success.

Pending Audits

[ ] Audit-3 · Profile banner image — Matrix protocol support

Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banner field. uk.tcpip.msc4133.stable = true on our server — check if a banner_url or similar field is defined. If no cross-client standard exists, do not implement.

📚 Implementation Reference

Exhaustive, low-level implementation details for backlog items. Follow these patterns to ensure code is "Lotus-perfect" (idiomatic, performant, and TDS-compliant).

P3-8 · Thread Panel (Full Side Drawer)

Architecture: Mirror the MembersDrawer pattern but with a specialized timeline.

  • State (src/app/state/room/thread.ts): 
    export const activeThreadIdAtom = atom<string | null>(null);
  • Layout (src/app/features/room/Room.tsx): Insert ThreadPanel conditionally alongside RoomTimeline: 
    {
  activeThreadId && (
    <>
      <Line variant="Background" direction="Vertical" size="300" />
      <ThreadPanel roomId={roomId} threadId={activeThreadId} />
    </>
  );
}
  • Component (src/app/features/room/thread/ThreadPanel.tsx): Use room.getThread(threadId) from the SDK. Render a Header with a "Close" button that sets activeThreadIdAtom to null. Reuse RoomTimeline but pass a filtered EventTimelineSet. Use thread.timelineSet directly for the most accurate thread view.

P4-4 · Math / LaTeX Rendering

Mechanism: KaTeX injection into the HTML parser.

  • Sanitizer (src/app/utils/sanitize.ts): Allow KaTeX-specific tags and classes (e.g., span, annotation, math). Use a specialized allowed list for math blocks.

    [Gemini_Found] sanitize.ts uses sanitize-html (not DOMPurify) with an explicit allowlist (allowedTags) and disallowedTagsMode: 'discard'. All MathML tags are currently absent from the allowlist and are silently stripped. Update permittedHtmlTags to include: <math>, <mi>, <mo>, <mn>, <ms>, <mtext>, <mspace>, <mrow>, <mfrac>, <msqrt>, <mroot>, <mstyle>, <merror>, <mpadded>, <mphantom>, <mfenced>, <menclose>, <msub>, <msup>, <msubsup>, <munder>, <mover>, <munderover>, <mmultiscripts>, <mtable>, <mtr>, <mtd>, <maligngroup>, <malignmark>, and annotation. Also add the required MathML attributes (e.g. xmlns, display, mathvariant) to permittedTagToAttributes.

  • Parser (src/app/plugins/react-custom-html-parser.tsx): Detect $ ... $ and $$ ... $$ patterns in text nodes: 
    if (node.type === 'text') {
  const parts = node.data.split(/(\$\$.*?\$\$|\$.*?\$)/g);
  return parts.map((p) => {
    if (p.startsWith('$')) return <KaTeX math={p.replace(/\$/g, '')} />;
    return p;
  });
}
  • CSS (src/app/styles/CustomHtml.css.ts): Import katex/dist/katex.min.css only when a math block is rendered to save initial bundle size.

P4-6 · OIDC / SSO Next-Gen Auth (MSC3861)

Mechanism: Matrix Authentication Service (MAS) Integration.

  • Architecture: Shift from password-based /login to OAuth2 authorization_code flow.
  • Key Files: src/app/pages/auth/Login.tsx and src/app/hooks/useAuth.ts.
  • Implementation: Use oidc-client-ts or a similar lightweight OIDC library. Check for m.authentication in /.well-known/matrix/client. Redirect to the MAS authorization endpoint. Handle the callback in a new OidcCallback route and store the OIDC refresh_token.

P5-1 · Custom Accent Color Picker (Non-TDS only)

Mechanism: Dynamic CSS variable injection.

  • Setting (src/app/state/settings.ts): Add customAccentColor: string (hex).
  • Manager (src/app/pages/ThemeManager.tsx): Inside the useEffect that monitors theme changes: 
    if (!lotusTerminal && customAccentColor) {
  document.documentElement.style.setProperty('--lt-accent-orange', customAccentColor);
  document.documentElement.style.setProperty('--lt-accent-orange-glow', `${customAccentColor}80`);
}
  • UI (src/app/features/settings/general/General.tsx): Use <Input type="color">. Hide this section if lotusTerminal is true.

P5-15 · In-Call Soundboard

Mechanism: Local-to-Global Audio Bridge via Web Audio API.

  • Create an AudioContext and a MediaStreamDestinationNode.
  • Create an AudioBufferSourceNode for each clip.
  • Route the mic MediaStream and the clip source to the destination node.
  • Pass the destination's .stream to the call bridge.

⚠️ [Gemini_Found — CORRECTED] Gemini originally suggested using LiveKit's LocalAudioTrack.replaceTrack() to mix audio into the call stream. This is not possible from Lotus Chat's realm: Element Call runs in a cross-origin iframe controlled via matrix-widget-api (postMessage). LiveKit's JS SDK and its LocalAudioTrack live inside EC's sandboxed context — inaccessible from our code. This directly contradicts the confirmed constraint already listed in the Server Capabilities table: "Cindy CANNOT inject audio into EC call stream — In-call soundboard must be redesigned as local-only." The soundboard must be a local-playback-only feature (output through the user's speakers, not mixed into the call audio stream).

🔱 [EC-FORK — RESOLVED] Both the original claim and the earlier "practical blocker still holds" correction are now outdated. EC is same-origin and we own the source, so we no longer reach into EC's module scope from cinny — instead the fork exposes the inject point itself: the io.lotus.inject_audio widget action (LotusWidgetActions.InjectAudio) publishes a clip as a separate LiveKit track from inside EC. A real in-call soundboard (mixed into the call, not local-only) is therefore unblocked, and the cinny-side soundboard UI is now built (P5-15 above): uploadable clips played into the call via this action, stored in io.lotus.soundboard account data.

P5-20 · Quick Reply from Browser Notification

Mechanism: Service Worker notificationclick Action.

[Gemini_Found] Implementation detail: serviceWorkerRegistration.showNotification() should be used instead of new Notification() so that the service worker can listen to the notificationclick event. new Notification() creates notifications that are bound to the client page, not the SW.

// src/sw.ts
self.addEventListener('notificationclick', (event) => {
  if (event.action === 'reply' && event.reply) {
    const { roomId, threadId } = event.notification.data;
    const session = sessions.get(event.clientId);
    fetch(`${session.baseUrl}/_matrix/client/v3/rooms/${roomId}/send/m.room.message`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${session.accessToken}` },
      body: JSON.stringify({
        msgtype: 'm.text',
        body: event.reply,
        'm.relates_to': threadId ? { rel_type: 'm.thread', event_id: threadId } : undefined,
      }),
    });
  }
});

P5-30 · Advanced ML Noise Suppression — Model Roadmap

See shipped implementation in LOTUS_FEATURES.md → "Noise Suppression (Advanced Multi-Tier)".

Models status:

  • RNNoise (sapphi, 48 kHz) — ✅ working, default fallback. Keep — runs on any hardware.
  • Speex (sapphi, 48 kHz) — ✅ working, low value; candidate to drop.
  • DTLN (@workadventure, 16 kHz) — 🟡 wired; sample-rate fix applied (was robotic at 48 kHz). TODO: verify in a real call. Narrowband (16 kHz) = slightly telephone-y even when correct.

Constraints: client-side AudioWorklet, fully self-hosted, no GPU, self-hosted SFU (no LiveKit Cloud).

Roadmap:

  • Verify DTLN 16 kHz fix in a real call.
  • DeepFilterNet 3 — best self-hostable upgrade: Rust→WASM, CPU real-time, 48 kHz fullband. Self-host df_bg.wasm + DFN3 ONNX model; wire a 48 kHz worklet. Audio quality unverifiable without a real-call test.
  • Desktop-only / HW-gated: FRCRN (Alibaba) or NVIDIA Maxine (RTX/Tensor only). Runs in Tauri Rust backend + bridges a virtual mic into the webview. Must detect capability; web + weak HW falls back to RNNoise/DTLN.

P5-31 · Granular Voice & Screenshare Quality Controls

Mechanism: WebRTC Encoding Parameters + Backend Quality Guard.

  • State Event: io.lotus.room_quality (state key "") containing: 
    { "audio_bitrate": 128000, "screen_max_res": "1080p", "screen_max_fps": 60 }
  • Screenshare: In src/app/plugins/call/CallControl.ts, map the "Quality" setting to getDisplayMedia constraints.
  • Audio Bitrate: After the call joins, find the RTCRtpSender for the audio track: 
    const sender = peerConnection.getSenders().find((s) => s.track?.kind === 'audio');
const params = sender.getParameters();
params.encodings[0].maxBitrate = roomBitrate || 128000;
await sender.setParameters(params);
  • Backend Sidecar: Extend voice-limit-guard.py (LXC 151) to fetch io.lotus.room_quality and inject limits into the LiveKit JWT or return them as an authorized config packet.

P5-40 · Desktop — Proactive Update Notifications (Tauri)

Key Files: src/app/hooks/useTauriUpdater.ts, src/app/pages/client/ClientNonUIFeatures.tsx, src/app/features/toast/LotusToastContainer.tsx.

  1. Create a TauriUpdateFeature component. Use useTauriUpdater() to get the check function and status.
  2. In a useEffect, call check() on mount and then on a setInterval (every 12 hours).
  3. When status transitions to { state: 'available', version: '...' }, fire a Lotus Toast: "Lotus Chat v[version] is available!" with an "Update" button that calls install().
  4. Store lastCheck timestamp in localStorage to prevent redundant checks on refresh.

Mobile Bookmarks Visibility Fix

Issue: ClientLayout.tsx explicitly restricts BookmarksPanel to ScreenSize.Desktop (lines 51-56).

// ClientLayout.tsx
{
  bookmarksOpen && (
    <BookmarksPanel
      onClose={() => setBookmarksOpen(false)}
      isMobile={screenSize !== ScreenSize.Desktop}
    />
  );
}

BookmarksPanel.tsx already supports the isMobile prop (line 127) to enable full-screen absolute positioning. No other changes required.

Remind Me Later (Slack-style)

Mechanism: Account Data + Timer/Service Worker.

  • Storage (src/app/hooks/useReminders.ts): Store in account data io.lotus.reminders as Array<{ id: string, roomId: string, eventId: string, timestamp: number }>.
  • Context Menu (src/app/features/room/message/MessageContextMenu.tsx): Add "Remind me" option → opens date/time picker modal (reuse JumpToTime.tsx logic).
  • Trigger (foreground): setTimeout in a hook inside ReminderMonitor in ClientNonUIFeatures.tsx → pushes to toastQueueAtom in state/toast.ts when due.
  • Trigger (background): Use Service Worker — setTimeout in the main thread will not fire when the PWA is suspended.

Mobile Usability Audit — Methodology

  1. Viewport & Touch: All interactive elements must have at least 44px × 44px touch targets. Audit for horizontal overflow (horizontal scrolling must be disabled).
  2. Modal Responsiveness: All modals (Settings, Profile, etc.) MUST cover the full screen on mobile, not float as overlays.
  3. Sidebar / Panels: On mobile, sidebar panels (Members, Bookmarks, Media) must become full-screen overlays (using a Drawer or Modal pattern) rather than side-by-side flexbox panels.
  4. Input & Composer: Ensure the composer doesn't get obscured by the mobile keyboard. Test focus trap and blur behaviors.

Implementation Notes

⚠️ TDS DESIGN LAW (repeated here for emphasis)

Every TDS color, animation, glow, border, shadow, and font value MUST come from /root/code/web_template/base.css.
Never hardcode hex values. Never invent CSS variable names.
Key variables: --lt-accent-orange · --lt-accent-cyan · --lt-accent-green · --lt-glow-* · --lt-box-glow-* · --lt-border-color · --lt-font-mono
Reference implementation: /root/code/tinker_tickets/ (markdown.js, base.js, ticket.css)
This applies without exception to every task marked [IMPROVE], [Build], or any UI change.

Design Rules

  • All new components must respect both TDS dark (LotusTerminalTheme) and TDS light (LotusTerminalLightTheme) modes
  • Non-TDS theme work (custom accent color, theme presets) uses vanilla-extract theme files — match the pattern in src/lotus-terminal.css.ts
  • Code syntax highlighting token classes: .tok-kw .tok-str .tok-num .tok-cmt .tok-fn (defined in web_template/base.css)
  • folds AvatarImage does NOT accept children — wrap Avatar components externally for overlays/frames/borders

CI/CD Pipeline

edit → commit → git push origin lotus
→ Gitea Actions: tsc --noEmit, eslint, prettier (~3 min)
→ Webhook: lotus_deploy.sh on LXC 106 polls CI, then npm ci && npm run build → rsync
→ Live at chat.lotusguild.org (~11 min total)

Per-Feature Checklist (before marking complete)

  • npx tsc --noEmit — zero TypeScript errors
  • npx eslint src/ — zero new errors (warnings OK if pre-existing)
  • npx prettier --check src/ — formatting passes
  • README.md updated (Lotus-custom features only — not upstream Cinny features)
  • landing/index.html updated if the feature appears in the comparison table
  • Visually tested at chat.lotusguild.org after CI deploys

Homeserver Access (for server audits)

  • Synapse (Matrix): LXC 151 on compute-storage-01 — pct exec 151 -- bash
  • Config: /etc/matrix-synapse/homeserver.yaml
  • Version check: curl -s https://matrix.lotusguild.org/_matrix/client/versions
Reference in New Issue View Git Blame Copy Permalink