LotusGuild/cinny
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
1b4c6cab6d87dd0d262483d139fbd7ea17a09c59
cinny/LOTUS_TODO.md
T
jared 3df95adc52 perf(media): defer image/video decryption until near-viewport (P5-5) 
Creates useNearViewport hook (IntersectionObserver, 200px rootMargin,
one-shot disconnect after first trigger). ImageContent and VideoContent
now gate loadSrc() on nearViewport — when autoPlay is enabled, encrypted
media is not decrypted until the element is within 200px of the visible
area, reducing initial page load cost on long timelines.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-18 17:56:34 -04:00

46 KiB
Raw Blame History

Lotus Chat — Work Backlog

Repo: lotus branch at https://code.lotusguild.org/LotusGuild/cinny
Deploy: push to lotus → CI → auto-deploy to chat.lotusguild.org (~11 min)

🏗️ Infrastructure & Maintenance

  • Upgrade Synapse to v1.155.0 ✅ Done 2026-06-18
    • Context: 1.155.0 is the last version supporting Debian 12 Bookworm. LXC 151 is already on Debian 13 Trixie — OS migration was completed prior to this upgrade.
    • What changed (1.154→1.155): No breaking changes, no config changes, no DB migrations. Bugfixes: to-device EDU size limiting, restricted room joins, sliding sync subscription response timing. Rust port of more internal classes (perf only).
    • MSC4452 (Preview URL capabilities) shipped in 1.154 — opt-in via msc4452_enabled, not enabled.

📱 Quick Feature Additions

  • Full-Screen Camera Broadcasts
    • Context: Element Call currently supports full-screening screenshares. We need to parity this functionality for camera broadcasts.
    • Goal: Users should be able to toggle any camera feed to full-screen mode, similar to the existing screenshare full-screen implementation.

⚠️ TDS DESIGN LAW — READ BEFORE TOUCHING ANY UI

ALL Lotus Terminal Design System (TDS) styling — colors, animations, glows, borders, fonts, spacing — MUST come exclusively from /root/code/web_template/base.css CSS variables. Do NOT hardcode hex values. Do NOT invent new variable names. Do NOT deviate from the design tokens defined in that file. The canonical variable reference: --lt-accent-orange, --lt-accent-cyan, --lt-accent-green, --lt-glow-orange, --lt-box-glow-*, --lt-border-color, etc. Reference implementation for code patterns: /root/code/tinker_tickets/ (markdown.js, base.js, ticket.css) This rule applies to EVERY task in this file without exception.

Completed features are documented in LOTUS_FEATURES.md.

Legend:

  • [AUDIT REQUIRED] — at least one assumption needs code/server verification before implementing
  • [SERVER CHECK] — depends on a Synapse feature or MSC; verify on matrix.lotusguild.org
  • [LOW PRIORITY] — implement after all higher-priority items
  • [EXTREME COMPLEXITY] — multi-sprint, plan separately before touching
  • [BLOCKED] — cannot build until a server upgrade, upstream MSC, or dependency resolves
  • [IMPROVE] — feature exists in upstream Cinny; this task enhances it for Lotus Chat

Status: [ ] pending · [~] in progress · [x] completed

Server Capabilities (as of June 2026)

  • Homeserver: matrix.lotusguild.org
  • Synapse version: 1.155.0 (2026-06-18) — fully up to date; last version for Debian 12 (LXC 151 already on Debian 13 Trixie)
  • Matrix spec: up to v1.12 formally; newer MSC features via unstable_features

Confirmed facts

Finding Impact
MSC flags ON: msc4140 · msc3771 · msc3440.stable · msc4133.stable · simplified_msc3575 · msc4222 · msc3266 · msc3401_matrix_rtc All safe to use now
MSC flags OFF: msc4306 (thread subscriptions) · msc3882 · msc3912 · msc4155 These features are BLOCKED
MSC3266 room summary: flag msc3266_enabled: true set but GET /v1/rooms/{id}/summary still returns 404 (M_UNRECOGNIZED) Room Preview BLOCKED — endpoint not implemented in Synapse 1.155
MSC3892 relation redaction: not in flags Reaction Redaction feature BLOCKED
MSC4260 report user: POST /_matrix/client/v3/users/{userId}/report returns 200 ✅ Report User UNBLOCKED — endpoint live since Synapse 1.133; ready to build
MSC4151 report room: HTTP 405 on GET = endpoint exists (POST only) Report Room live ✅
folds AvatarImage does NOT accept children Add frame/overlay inside UserAvatar.tsx itself — optional frameName prop
No in-app toast system exists (was) Built ToastProvider + Jotai queue; at App.tsx:65
useUnverifiedDeviceCount() hook exists src/app/hooks/useDeviceVerificationStatus.ts:65-106
Voice player: AudioContent.tsx:44-223 Playback rate on hidden <audio> at line 217
CallControl.setMicrophone(bool) at CallControl.ts:206-212 For AFK auto-mute
CallControl.toggleSound() at CallControl.ts:230-251 Push-to-deafen — just wire a hotkey to this
matrix-js-sdk has NO arbitrary profile field methods Use mx.http.authedRequest() for MSC4133
Sanitizer (sanitize.ts) allows table, div, span, a, code, hr LFG HTML card is safe locally; test on Element/FluffyChat
Sanitizer STRIPS <math>/MathML tags Math/LaTeX task must also modify sanitizer
Service worker EXISTS at src/sw.ts Quick-reply task: add notificationclick handler
knockSupported() utility exists at matrix.ts:376-391 Knock UX: only need "Request to Join" in RoomIntro.tsx
KeywordMessages.tsx already has custom keyword push rules Full push rule editor: only non-keyword rule types need new UI
getMatrixToRoom() in matrix-to.ts generates invite URLs Invite link: just add QR code to room settings
Cindy CANNOT inject audio into EC call stream In-call soundboard must be redesigned as local-only
Folds uses vanilla-extract in non-TDS, NOT CSS custom properties Custom accent color: must create new vanilla-extract theme variant dynamically
Theme presets need ~50 CSS custom properties each Significant design work before coding
useCallSpeakers.ts CSS MutationObserver polling Visual speaking indicator: TDS ring animation on top of existing data
MSC3489/3672 live location: BOTH false on server Live Location BLOCKED

Key File Reference

What you need File Lines
Global keydown hook src/app/hooks/useKeyDown.ts whole file
Room navigation src/app/hooks/useRoomNavigate.ts 19-72
All room IDs atom src/app/state/room-list/roomList.ts allRoomsAtom
Room unread counts src/app/state/room/roomToUnread.ts roomToUnreadAtom
Overlay portal provider src/app/pages/App.tsx 65
Portal container div index.html 101
Room settings tabs src/app/features/room-settings/RoomSettings.tsx 27-56
State event read/write pattern src/app/features/common-settings/general/RoomEncryption.tsx 42-52
Power level checker src/app/hooks/usePowerLevels.ts whole file
Slash command registration src/app/hooks/useCommands.ts 140-537
Chat background picker src/app/features/settings/general/General.tsx 945-981
Chat backgrounds definition src/app/features/lotus/chatBackground.ts whole file
Matrix.to URL builder src/app/plugins/matrix-to.ts getMatrixToRoom()
Media event content types src/app/types/matrix/common.ts 46-91
Media URL conversion src/app/utils/matrix.ts mxcUrlToHttp()
Message pagination (search) src/app/features/message-search/useMessageSearch.ts 74-121
Infinite pagination pattern src/app/features/message-search/MessageSearch.tsx 234-365
Poll event format src/app/components/message/content/PollContent.tsx 1-320
Theme class application src/app/hooks/useTheme.ts 25-60
Animations file src/app/styles/Animations.css.ts whole file
Message status (EventStatus) src/app/features/room/message/Message.tsx 84-142
Call member change events src/app/hooks/useCall.ts 37-52
Mic control in calls src/app/plugins/call/CallControl.ts 206-212
Device verification hook src/app/hooks/useDeviceVerificationStatus.ts 65-106
Knock room support check src/app/utils/matrix.ts 376-391
Room join button location src/app/components/room-intro/RoomIntro.tsx 25-119
Notification mute via push rules src/app/hooks/useRoomsNotificationPreferences.ts 110-150
Message text body CSS src/app/components/message/layout/layout.css.ts 182-205

Priority 3 — Higher complexity / lower daily frequency

[ ] P3-4 · Accessibility Improvements (WCAG 2.1 AA)

What: Comprehensive audit and fix pass targeting the critical user paths:

  • Room list navigation (keyboard-only)
  • Reading messages in the timeline (screen reader announces new messages)
  • Composing and sending a reply
  • Opening and closing modals (focus trap, return focus)
  • ARIA labels on all icon-only buttons

Scope: Do NOT attempt to make every corner of the app AA-compliant in one pass — focus on the golden path (open app → find room → read → reply → send).
[AUDIT REQUIRED] — Run an automated audit first: npx axe-core or browser DevTools accessibility tree. Document every violation before writing a single line of code. Prioritize by severity (critical > serious > moderate).

Investigation Findings:

  • Root Cause: Inconsistent focus management, missing aria-live regions for dynamic timeline updates, and sparse global keyboard shortcuts.
  • Approach: Standardize focus-trap-react usage (reference RoomNavItem.tsx). Add aria-live regions to the timeline. Expand useKeyDown.ts for section navigation shortcuts.
  • Complexity: Medium-High (audit is the main work).

[ ] P3-8 · Thread Panel (full side drawer)

⚠️ LARGEST FEATURE — requires its own planning session before implementation.
What: A right-side drawer for threaded conversations. Currently "Reply in Thread" exists but there is no panel to read or write thread replies.

Features:

  • Click "Reply in Thread" → opens thread drawer on the right
  • Thread root event shown at the top of the panel
  • Full message rendering for all in-thread replies (reuse timeline components)
  • Reply input at the bottom (full composer with formatting, emoji, etc.)
  • Unread count badge on the thread button in the main timeline
  • Keyboard shortcut to close thread panel

Architecture:

  • New Jotai atom: activeThreadEventId: string | null
  • New component: src/app/features/room/thread/ThreadPanel.tsx
  • Rendered alongside RoomView as a conditional right panel (mirror the members drawer pattern)
  • Filter events in timeline to m.thread relation for the active root event ID
  • Shares the same mx client and room reference as the main timeline

[AUDIT REQUIRED] — Deeply audit how m.thread relation events are currently stored and retrieved in the matrix-js-sdk. Understand the thread aggregation API: GET /rooms/{roomId}/relations/{eventId}/m.thread. Check if RoomTimeline.tsx currently filters out thread replies from the main timeline (it should — confirm).

Investigation Findings:

  • Root Cause: Current m.thread events are treated as standard m.room.message events and rendered in the main timeline.
  • Approach: Introduce new Jotai atom activeThreadEventId. Create ThreadPanel.tsx. Update RoomTimeline.tsx to filter out thread relations (m.relates_to). Implement aggregation fetch using GET /rooms/{roomId}/relations/{eventId}/m.thread. Use thread.timelineSet directly for the most accurate thread view.
  • Complexity: High.

Priority 4 — Specialized, high complexity, or low priority

[ ] P4-7 · Virtualized Infinite Scroll for Search Results

What: Replace the manual "load more" button with an automated, virtualized infinite scroll for search results.
Approach: Utilize @tanstack/react-virtual in MessageSearch.tsx to handle the nextToken automatically as the user scrolls.

[ ] P4-8 · Encrypted Message Search Indexing & Caching

What: Implement a persistent local cache for search results, optimized for encrypted rooms.
Approach: Use IndexedDB to store search metadata (event IDs, timestamps) to prevent redundant server-side decryption/fetching.

[ ] P4-9 · Advanced Search Filter UI

What: Introduce a more robust search filter UI in SearchFilters.tsx.
Approach: Add UI components for easier filtering and a visual date-range picker that correctly maps to fromTs and toTs.

[ ] P4-1 · Thread Notification Mode Per-Thread (MSC3771)

Spec: MSC3771 (stable). Depends on Thread Panel (#P3-8).
What: Per-thread notification toggle: "All messages" vs "Mentions only". Accessible from the thread panel header. Tracks unread counts separately per thread.
[AUDIT REQUIRED] — Implement after Thread Panel. Requires understanding how the SDK tracks per-thread unread counts.
Complexity: Medium (after thread panel exists).

[ ] P4-2 · Thread Subscriptions (MSC4306) [BLOCKED]

Spec: MSC4306 (Synapse experimental). Depends on Thread Panel (#P3-8).
What: "Follow thread" button to receive notifications for a thread you haven't posted in. Uses MSC4306 subscription endpoint.
[SERVER CHECK] — org.matrix.msc4306 = false on matrix.lotusguild.org — BLOCKED until server enables it.
Complexity: Medium (after thread panel exists).

[ ] P4-4 · Math / LaTeX Rendering in Messages (LOW PRIORITY)

Spec: CS-API §11.5 (stable) — formatted_body can contain LaTeX.
What: Render $...$ or $$...$$ LaTeX expressions in message bodies. Use KaTeX (lightweight, ~100KB, renders server-side-compatible CSS). Must gracefully fall back to raw LaTeX text if KaTeX fails.
Note: This is LOW PRIORITY — only useful for academic/technical communities. Implement last.
[AUDIT REQUIRED] — Confirm KaTeX bundle size impact on the Vite bundle. Check if matrix-js-sdk's HTML sanitizer strips LaTeX before it reaches the renderer. The formatted_body sanitization pipeline is the main risk here. (Confirmed: sanitizer STRIPS <math> tags — must be patched alongside the renderer.)
Complexity: Low-Medium.

[ ] P4-5 · Live Location Sharing (MSC3489 + MSC3672) (LOW PRIORITY, HIGH COMPLEXITY) [BLOCKED]

Spec: MSC3489 + MSC3672. Implemented in Element Web.
Note: Static location sharing is already implemented. This adds live/real-time GPS beacons. Very low priority per user preference.
What: Start sharing live location → creates m.beacon_info state event → client posts m.beacon events on a timer → other users see your position update live on a map.
[SERVER CHECK] — org.matrix.msc3489 = false AND org.matrix.msc3672 = false on matrix.lotusguild.org — BLOCKED.
Complexity: High. Requires background geolocation API + live map rendering.

[ ] P4-6 · OIDC / SSO Next-Gen Auth (MSC3861) (EXTREME COMPLEXITY, LOW PRIORITY)

Spec: MSC3861, merged Matrix spec v1.15. Uses Matrix Authentication Service (MAS).
Context: ~80% of homeserver users have LLDAP/Authelia/SSO accounts. SSO is currently enabled on matrix.lotusguild.org but accounts are not yet linked. This would allow users to log in via their SSO credentials.
What: OAuth 2.0 / OIDC login flow, token refresh, account management page linking Matrix identity to SSO identity.
EXTREME COMPLEXITY — requires: MAS deployment/configuration on the homeserver, significant auth flow changes in the client, token refresh handling, session management overhaul.
[SERVER CHECK] — Before any client work, audit whether MAS is already deployed on compute-storage-01. Check: pct exec 151 -- systemctl status matrix-authentication-service or similar.
Complexity: Extreme. Multi-sprint project. Plan separately.

Priority 5 — Gamer / Aesthetic / Customization

[ ] P5-1 · Custom Accent Color Picker (non-TDS mode only)

What: A hex/HSL color picker in Settings → Appearance. Chosen color replaces the primary accent throughout the UI: buttons, badges, active states, highlights, presence dot, links. Applied via a CSS custom property override injected into <head>.
IMPORTANT: This feature is completely inactive when TDS is enabled — TDS has its own fixed palette. Add this setting under a "Non-TDS Themes" section that is hidden when TDS is active.
[AUDIT REQUIRED] Identify all CSS custom properties that constitute the "accent color" in non-TDS mode. Map them to the folds/vanilla-extract token names. (Confirmed: folds uses vanilla-extract, NOT CSS custom properties — must create a new vanilla-extract theme variant dynamically.)
Complexity: Medium.

[ ] P5-2 · Additional Color Theme Presets

What: 5 new one-click theme presets alongside TDS. Each must be a complete, polished system with proper contrast ratios (WCAG AA). All implemented as vanilla-extract themes matching the existing TDS pattern.

Themes:

  1. Cyberpunk — deep navy bg (#0a0015), electric purple (#bf5fff) + hot pink (#ff2d9b) accents, neon glow
  2. Ocean — deep sea blue bg (#020b18), teal (#00c9b1) + aqua (#0096d6) accents, soft feel
  3. Blood Red — near-black bg (#0d0203), deep crimson (#7a0010) + bright red (#ff2233) accents
  4. Classic Matrix — pure black bg (#000000), phosphor green (#00ff41) text + accents
  5. Midnight — dark charcoal (#111827), cool blue-grey (#6b7ca8) accents, clean minimal

[AUDIT REQUIRED] Study src/lotus-terminal.css.ts for the full token list before designing themes. All tokens must be covered (~50 CSS custom properties each).
Complexity: Medium (design effort is the main cost).

[MOVED] P5-9 · LFG (Looking for Group) Command → LotusBot

Decision: Implemented as !lfg in LotusBot rather than a client slash command. Bot-side rendering works consistently across all Matrix clients; client-side enhanced cards would only be visible to Lotus Chat users and require sanitizer auditing. The bot can also support richer flows (list active LFGs, DM interested players, auto-expire posts).

[x] P5-5 · Intersection-Based Lazy Loading ⚠️ UNTESTED — needs verification in timeline with many images

What: Use IntersectionObserver to trigger media decryption and loading only when components approach the viewport.
Approach: Reduce initial memory footprint and improve timeline load times by deferring decryption of images/videos until they are visible.

[ ] P5-6 · Context-Aware Thumbnail Previews

What: Enhance thumbnail rendering in the timeline for consistent, polished aesthetics.
Approach: Use CSS object-fit: cover with improved focal-point centering within ThumbnailContent to prevent media stretching or awkward aspect-ratio cropping.

[ ] P5-15 · In-Call Soundboard

What: Grid of short audio clips playable into the call audio stream via Web Audio API (AudioBufferSourceNode → MediaStreamDestinationNode → mixed with mic). Built-in clips + user-uploadable custom clips (stored as mxc://). Accessible from call controls bar.
[AUDIT REQUIRED] Verify the Element Call integration exposes the mic MediaStream for mixing. This is the highest-risk part of this feature.
Complexity: High.

[ ] P5-20 · Quick Reply from Browser Notification

What: Inline reply field in browser notification toasts via Notification Actions API. Reply sends as threaded reply to the triggering message.
[AUDIT REQUIRED] (1) Verify browser Notification Actions API support in target browsers. (2) Confirmed: service worker EXISTS at src/sw.ts — add notificationclick handler there.
Complexity: Medium-High.

[x] P5-30 · Advanced ML Noise Suppression (Krisp-style)

What: High-end background noise cancellation using a pre-trained ML model (RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
Shipped: 3-tier setting (Off / Browser-native / ML) in Settings → General → Calls. ML tier injects a same-origin pre-init shim into the vendored Element Call index.html that monkeypatches getUserMedia and routes the captured mic through an RNNoise AudioWorklet before LiveKit publishes — no EC fork required. See LOTUS_FEATURES.md → "Noise Suppression (Advanced Multi-Tier)".
Key decision: LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU); EC's own RNNoise PR #3892 is unmerged. The shim is the same post-capture pipeline #3892 uses, executed from the realm we control, so it survives EC version bumps.
AEC note (resolved-as-accepted): WebAudio capture routing can weaken browser AEC — same tradeoff as EC's upstream feature; mitigated by keeping echoCancellation/autoGainControl on the raw capture and labeling the tier "beta".

Model Roadmap (priority order):

  • Verify DTLN (16 kHz narrowband fix) in a real call before investing further — wired but unverified.
  • DeepFilterNet 3 — best self-hostable upgrade: Rust→WASM, CPU real-time, 48 kHz fullband. Effort: self-host df_bg.wasm + DFN3 ONNX model, wire a 48 kHz worklet.
  • Desktop-only / HW-gated: FRCRN or NVIDIA Maxine (RTX/Tensor only) — impossible in-browser; would run in Tauri Rust backend + bridge a virtual mic into the webview. Must detect capability and only offer on supported hardware; web falls back to RNNoise.
  • Excluded: Krisp (LiveKit Cloud only); FRCRN/Maxine on web (GPU/server-bound).

[ ] P5-31 · Granular Voice & Screenshare Quality Controls (Discord-style)

What: Let users (or room admins via room settings) adjust audio bitrates (e.g., 64kbps to 512kbps) and screenshare quality (resolution: 720p/1080p/Source, framerate: 15/30/60fps).
Note: Requires tight integration with the LiveKit SFU and custom state events for per-room quality caps.
[AUDIT REQUIRED] Must verify if current lk-jwt-service can be extended with custom bitrate/resolution claims or if a new sidecar (similar to voice-limit-guard) is needed for server-side enforcement.
Complexity: Extreme.

[ ] P5-35 · Desktop — Notification Click Opens Room (DEFERRED)

What: Clicking a system tray notification navigates to the relevant room. Quick-reply from the notification toast would send the reply without opening the window.
Status: Deferred — tauri-plugin-notification has no Rust click/action callback API. Quick-reply would need a custom WinRT toast activator + COM registration, which can't be compile-tested without a Windows build environment.
Note: Tray icon and matrix: deep links already bring the window forward on most interactions. Revisit when tauri-plugin-notification gains click handler support upstream.
Complexity: High (platform-specific native code required).

[ ] P5-36 · Desktop — Windows Jump List (DEFERRED)

What: Right-clicking the taskbar icon shows a jump list with recent/favorite rooms for quick navigation.
Status: Deferred — implementing the Windows COM jump list API in Tauri requires iterating on C++/COM code that can only be compile-checked on Windows, making blind CI iteration impractical.
Action when unblocked: Revisit when a Tauri plugin abstracts the Windows Shell ICustomDestinationList interface, or when a Windows build environment is available for local iteration.
Complexity: High (Windows-only native COM).

[x] P5-40 · Desktop — Proactive Update Notifications (Tauri) ⚠️ UNTESTED (requires Tauri build)

What: Automatically check for app updates on launch and periodically during long sessions. If an update is available, show an in-app toast or badge (e.g., on the Settings icon) to alert the user without requiring a manual check in settings.
Mechanism: Use the useTauriUpdater hook in a global component like ClientNonUIFeatures.tsx.
Note: Ensure the check is throttled (e.g., once every 12 hours) to avoid redundant Tauri commands.
Complexity: Low-Medium.

[ ] P5-41 · Desktop — Native WinRT Toast Notifications

What: Replace emulated notifications with native WinRT Toast notifications.
Approach: Implement native WinRT Toast integration using windows-rs to enable full Action Center integration, including native Quick Reply functionality.

[ ] P5-42 · Desktop — Persistent Background Sync

What: Maintain light connection to homeserver when WebView2 is suspended.
Approach: Implement a headless Rust sidecar to fetch unread counts/notifications while the webview is suspended to ensure instant notification delivery.

[ ] P5-43 · Desktop — System Media Transport Controls (SMTC)

What: Integrate with Windows SMTC for volume flyout call/media control.
Approach: Use Windows SMTC API to expose call status, mic mute/unmute, and media controls to the Windows volume flyout/media overlay.

[ ] P5-44 · Desktop — Taskbar Thumbnail Toolbar

What: Add persistent call controls to the taskbar preview.
Approach: Implement a COM thumbnail toolbar in the application preview window, featuring Mute/Deafen/End Call buttons.

[ ] P5-46 · Desktop — System Power Management (Call Continuity)

What: Prevent system sleep/hibernate during active calls.
Approach: Use Tauri/Rust power-manager or platform-specific APIs to block system power saving states while a voice/video session is active.

[ ] P5-47 · Desktop — TDS-Styled Native Window Chrome

What: Replace system titlebar with custom Lotus TDS chrome.
Approach: Configure Tauri window (decorations: false) and implement custom, TDS-token compliant titlebar controls (Close/Max/Min) for a cohesive UI.

[ ] P5-48 · Desktop — Native File System Drag-and-Drop Improvements

What: Enhance drag-and-drop support for Windows.
Approach: Improve handling for Windows file shortcuts, recursive folder uploads, and shell-integrated "Send To" context menu actions.

[ ] P5-49 · Desktop — Network Awareness (NCSI Integration)

What: Proactively detect Windows network connectivity changes.
Approach: Integrate with the Windows Network Connectivity Status Indicator (NCSI) API to improve offline mode transition latency and network recovery.

[ ] P5-50 · Desktop — Windows Hardware-Accelerated Media Pipeline

What: Replace standard browser decoding with native Windows Media Foundation.
Approach: Leverage DirectShow/Media Foundation to offload video/audio decoding from the CPU to the GPU, significantly reducing power consumption and latency during calls.

[ ] P5-51 · Desktop — Federated "Identity Contexts" (Isolation Manager)

What: Compartmentalize sessions, local databases, and caches into isolated "Contexts."
Approach: Implement a zero-leak boundary for personas (e.g., Work vs. Personal) by isolating IndexedDB, filesystem caches, and session persistence per context.
Priority: Extreme Low (Multi-sprint/Architectural).

[~] P5-52 · Desktop — Room-Level Sync Governor (Performance Control) [STILL_CONSIDERING]

What: Granular sync tuning for individual rooms.
Approach: Allow per-room overrides for sync frequency and event type filtering (e.g., disable read receipts/typing in heavy rooms) to optimize performance. Implementation requires careful UX to prevent complexity fatigue.

[ ] P5-53 · Desktop — Local-Only "Scripting" Plugin System (Tampermonkey-like)

What: A sandboxed environment for local execution of user scripts on Matrix events.
Approach: Implement a WASM-based execution engine that allows users to write local-only, client-side scripts to interact with incoming Matrix events, trigger sounds/notifications, or inject custom UI elements based on event payload rules. Designed for privacy — all logic runs exclusively on the local machine.

[ ] P5-55 · Desktop — Composer Toolbar Drag-and-Drop Reordering

What: Allow users to reorder toolbar icons via drag-and-drop.
Approach: Extend the current settings-based toolbar toggle system to include a drag-and-drop UI mode in the composer settings, allowing users to personalize their icon order.

[ ] P5-56 · Desktop — Windows "Focus Assist" (DND) Sync

What: Automatically toggle notification state based on Windows Focus Assist.
Approach: Integrate with the Windows NotificationCenter / Focus state via Tauri/Rust to automatically enable/disable Lotus Chat's internal notification suppression mode when Windows Focus Assist is toggled.

[ ] P5-57 · Desktop — Visual Draft Persistence Indicator

🚀 Features to Add

  • Mobile Audit: Comprehensive audit of all features in LOTUS_FEATURES.md for mobile PWA usability and layout responsiveness.
  • Remind Me Later: Slack-style reminders for messages — fully implemented ⚠️ UNTESTED end-to-end
    • Storage: useReminders.ts — persists to io.lotus.reminders account data with addReminder / removeReminder / getReminders
    • UI: RemindMeDialog.tsx — 4 presets (20 min, 1 hr, 3 hr, tomorrow 9am); wired into Message.tsx context menu via remindOpen state; useModalStyle(320) for mobile fullscreen
    • Monitor: ReminderMonitor in ClientNonUIFeatures.tsx — polls every 30s + on tab visibility; fires Lotus toast when due and calls removeReminder
  • Mobile Bookmarks: Fixed ⚠️ UNTESTED — bookmarks now accessible from within any room on mobile
    • Root Cause: BookmarksPanel renders correctly on mobile but BookmarksTab lives in SidebarNav, which is hidden when inside a room on mobile (MobileFriendlyClientNav returns null). No trigger existed.
    • Fix: Added "Saved Messages" MenuItem to the RoomMenu (···More Options) in RoomViewHeader.tsx. Toggles bookmarksPanelAtom and closes the menu. Works on all screen sizes — desktop users see it as a duplicate of the sidebar star, mobile users now have their only in-room access point.

Blocked Features

These features are confirmed desirable but cannot be built until the listed dependency is resolved. Check back after each Synapse upgrade — re-run /matrix/client/versions and unstable_features to see if they've become available.

[BLOCKED] · Live Location Sharing (MSC3489 + MSC3672)

Blocked by: org.matrix.msc3489 = false AND org.matrix.msc3672 = false on matrix.lotusguild.org (confirmed from unstable_features).
What it would do: Real-time GPS beacon streaming upgrading the existing static location share.
Action when unblocked: Both MSCs must be enabled on the homeserver before any client work.

[BLOCKED] · Reaction / Relation Redaction (MSC3892)

Blocked by: org.matrix.msc3892 = false on matrix.lotusguild.org
What it would do: Cleanly remove a reaction without redacting the parent message.
Current behavior: Full event redaction — acceptable fallback, no user-facing issue.
Action when unblocked: Find onReactionToggle redaction call site; swap in MSC3892 endpoint with fallback.

[BLOCKED] · Room Preview Before Joining (MSC3266)

Blocked by: GET /_matrix/client/v1/rooms/{roomId}/summary returns M_UNRECOGNIZED 404 — endpoint not implemented in Synapse 1.155. Config flag msc3266_enabled: true is set but has no effect; Synapse appears not to have shipped a stable implementation at the v1 path. Verified 2026-06-18.
What it would do: Show room name, topic, avatar, member count before joining.
Action when unblocked: Re-test after each future Synapse upgrade.

[BLOCKED] · Thread Subscriptions (MSC4306)

Blocked by: org.matrix.msc4306 = false on matrix.lotusguild.org
What it would do: Follow a thread without posting; get notifications for replies.
Action when unblocked: Add "Follow thread" button in the thread panel header (depends on #P3-8 Thread Panel).

[DONE] · Report User (MSC4260) ✅

Previously blocked by: Server spec v1.12, but POST /_matrix/client/v3/users/{userId}/report was confirmed 200 on 2026-06-18 (live since Synapse 1.133.0).
What it does: Reports a specific user to homeserver admins (separate from reporting a message).
Note: Report Message already exists in upstream Cinny. This adds Report User to the profile panel.
Implemented 2026-06-18: ReportUserModal.tsx added at src/app/features/room/ReportUserModal.tsx. Button wired into UserRoomProfile.tsx between UserModeration and UserDeviceSessions (hidden for own profile). Category dropdown + reason text, inline success/error feedback, auto-close 1500ms after success.

Pending Audits

[ ] Audit-3 · Profile banner image — Matrix protocol support

Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banner field. uk.tcpip.msc4133.stable = true on our server — check if a banner_url or similar field is defined. If no cross-client standard exists, do not implement.

📚 Implementation Reference

Exhaustive, low-level implementation details for backlog items. Follow these patterns to ensure code is "Lotus-perfect" (idiomatic, performant, and TDS-compliant).

P3-8 · Thread Panel (Full Side Drawer)

Architecture: Mirror the MembersDrawer pattern but with a specialized timeline.

  • State (src/app/state/room/thread.ts): 
    export const activeThreadIdAtom = atom<string | null>(null);
  • Layout (src/app/features/room/Room.tsx): Insert ThreadPanel conditionally alongside RoomTimeline: 
    {
  activeThreadId && (
    <>
      <Line variant="Background" direction="Vertical" size="300" />
      <ThreadPanel roomId={roomId} threadId={activeThreadId} />
    </>
  );
}
  • Component (src/app/features/room/thread/ThreadPanel.tsx): Use room.getThread(threadId) from the SDK. Render a Header with a "Close" button that sets activeThreadIdAtom to null. Reuse RoomTimeline but pass a filtered EventTimelineSet. Use thread.timelineSet directly for the most accurate thread view.

P4-4 · Math / LaTeX Rendering

Mechanism: KaTeX injection into the HTML parser.

  • Sanitizer (src/app/utils/sanitize.ts): Allow KaTeX-specific tags and classes (e.g., span, annotation, math). Use a specialized allowed list for math blocks.
  • Parser (src/app/plugins/react-custom-html-parser.tsx): Detect $ ... $ and $$ ... $$ patterns in text nodes: 
    if (node.type === 'text') {
  const parts = node.data.split(/(\$\$.*?\$\$|\$.*?\$)/g);
  return parts.map((p) => {
    if (p.startsWith('$')) return <KaTeX math={p.replace(/\$/g, '')} />;
    return p;
  });
}
  • CSS (src/app/styles/CustomHtml.css.ts): Import katex/dist/katex.min.css only when a math block is rendered to save initial bundle size.

P4-6 · OIDC / SSO Next-Gen Auth (MSC3861)

Mechanism: Matrix Authentication Service (MAS) Integration.

  • Architecture: Shift from password-based /login to OAuth2 authorization_code flow.
  • Key Files: src/app/pages/auth/Login.tsx and src/app/hooks/useAuth.ts.
  • Implementation: Use oidc-client-ts or a similar lightweight OIDC library. Check for m.authentication in /.well-known/matrix/client. Redirect to the MAS authorization endpoint. Handle the callback in a new OidcCallback route and store the OIDC refresh_token.

P5-1 · Custom Accent Color Picker (Non-TDS only)

Mechanism: Dynamic CSS variable injection.

  • Setting (src/app/state/settings.ts): Add customAccentColor: string (hex).
  • Manager (src/app/pages/ThemeManager.tsx): Inside the useEffect that monitors theme changes: 
    if (!lotusTerminal && customAccentColor) {
  document.documentElement.style.setProperty('--lt-accent-orange', customAccentColor);
  document.documentElement.style.setProperty('--lt-accent-orange-glow', `${customAccentColor}80`);
}
  • UI (src/app/features/settings/general/General.tsx): Use <Input type="color">. Hide this section if lotusTerminal is true.

P5-15 · In-Call Soundboard

Mechanism: Local-to-Global Audio Bridge via Web Audio API.

  • Create an AudioContext and a MediaStreamDestinationNode.
  • Create an AudioBufferSourceNode for each clip.
  • Route the mic MediaStream and the clip source to the destination node.
  • Pass the destination's .stream to the call bridge.

P5-20 · Quick Reply from Browser Notification

Mechanism: Service Worker notificationclick Action.

// src/sw.ts
self.addEventListener('notificationclick', (event) => {
  if (event.action === 'reply' && event.reply) {
    const { roomId, threadId } = event.notification.data;
    const session = sessions.get(event.clientId);
    fetch(`${session.baseUrl}/_matrix/client/v3/rooms/${roomId}/send/m.room.message`, {
      method: 'POST',
      headers: { Authorization: `Bearer ${session.accessToken}` },
      body: JSON.stringify({
        msgtype: 'm.text',
        body: event.reply,
        'm.relates_to': threadId ? { rel_type: 'm.thread', event_id: threadId } : undefined,
      }),
    });
  }
});

P5-30 · Advanced ML Noise Suppression — Model Roadmap

See shipped implementation in LOTUS_FEATURES.md → "Noise Suppression (Advanced Multi-Tier)".

Models status:

  • RNNoise (sapphi, 48 kHz) — ✅ working, default fallback. Keep — runs on any hardware.
  • Speex (sapphi, 48 kHz) — ✅ working, low value; candidate to drop.
  • DTLN (@workadventure, 16 kHz) — 🟡 wired; sample-rate fix applied (was robotic at 48 kHz). TODO: verify in a real call. Narrowband (16 kHz) = slightly telephone-y even when correct.

Constraints: client-side AudioWorklet, fully self-hosted, no GPU, self-hosted SFU (no LiveKit Cloud).

Roadmap:

  • Verify DTLN 16 kHz fix in a real call.
  • DeepFilterNet 3 — best self-hostable upgrade: Rust→WASM, CPU real-time, 48 kHz fullband. Self-host df_bg.wasm + DFN3 ONNX model; wire a 48 kHz worklet. Audio quality unverifiable without a real-call test.
  • Desktop-only / HW-gated: FRCRN (Alibaba) or NVIDIA Maxine (RTX/Tensor only). Runs in Tauri Rust backend + bridges a virtual mic into the webview. Must detect capability; web + weak HW falls back to RNNoise/DTLN.

P5-31 · Granular Voice & Screenshare Quality Controls

Mechanism: WebRTC Encoding Parameters + Backend Quality Guard.

  • State Event: io.lotus.room_quality (state key "") containing: 
    { "audio_bitrate": 128000, "screen_max_res": "1080p", "screen_max_fps": 60 }
  • Screenshare: In src/app/plugins/call/CallControl.ts, map the "Quality" setting to getDisplayMedia constraints.
  • Audio Bitrate: After the call joins, find the RTCRtpSender for the audio track: 
    const sender = peerConnection.getSenders().find((s) => s.track?.kind === 'audio');
const params = sender.getParameters();
params.encodings[0].maxBitrate = roomBitrate || 128000;
await sender.setParameters(params);
  • Backend Sidecar: Extend voice-limit-guard.py (LXC 151) to fetch io.lotus.room_quality and inject limits into the LiveKit JWT or return them as an authorized config packet.

P5-40 · Desktop — Proactive Update Notifications (Tauri)

Key Files: src/app/hooks/useTauriUpdater.ts, src/app/pages/client/ClientNonUIFeatures.tsx, src/app/features/toast/LotusToastContainer.tsx.

  1. Create a TauriUpdateFeature component. Use useTauriUpdater() to get the check function and status.
  2. In a useEffect, call check() on mount and then on a setInterval (every 12 hours).
  3. When status transitions to { state: 'available', version: '...' }, fire a Lotus Toast: "Lotus Chat v[version] is available!" with an "Update" button that calls install().
  4. Store lastCheck timestamp in localStorage to prevent redundant checks on refresh.

Mobile Bookmarks Visibility Fix

Issue: ClientLayout.tsx explicitly restricts BookmarksPanel to ScreenSize.Desktop (lines 51-56).

// ClientLayout.tsx
{
  bookmarksOpen && (
    <BookmarksPanel
      onClose={() => setBookmarksOpen(false)}
      isMobile={screenSize !== ScreenSize.Desktop}
    />
  );
}

BookmarksPanel.tsx already supports the isMobile prop (line 127) to enable full-screen absolute positioning. No other changes required.

Remind Me Later (Slack-style)

Mechanism: Account Data + Timer/Service Worker.

  • Storage (src/app/hooks/useReminders.ts): Store in account data io.lotus.reminders as Array<{ id: string, roomId: string, eventId: string, timestamp: number }>.
  • Context Menu (src/app/features/room/message/MessageContextMenu.tsx): Add "Remind me" option → opens date/time picker modal (reuse JumpToTime.tsx logic).
  • Trigger (foreground): setTimeout in a hook inside ReminderMonitor in ClientNonUIFeatures.tsx → pushes to toastQueueAtom in state/toast.ts when due.
  • Trigger (background): Use Service Worker — setTimeout in the main thread will not fire when the PWA is suspended.

Mobile Usability Audit — Methodology

  1. Viewport & Touch: All interactive elements must have at least 44px × 44px touch targets. Audit for horizontal overflow (horizontal scrolling must be disabled).
  2. Modal Responsiveness: All modals (Settings, Profile, etc.) MUST cover the full screen on mobile, not float as overlays.
  3. Sidebar / Panels: On mobile, sidebar panels (Members, Bookmarks, Media) must become full-screen overlays (using a Drawer or Modal pattern) rather than side-by-side flexbox panels.
  4. Input & Composer: Ensure the composer doesn't get obscured by the mobile keyboard. Test focus trap and blur behaviors.

Implementation Notes

⚠️ TDS DESIGN LAW (repeated here for emphasis)

Every TDS color, animation, glow, border, shadow, and font value MUST come from /root/code/web_template/base.css.
Never hardcode hex values. Never invent CSS variable names.
Key variables: --lt-accent-orange · --lt-accent-cyan · --lt-accent-green · --lt-glow-* · --lt-box-glow-* · --lt-border-color · --lt-font-mono
Reference implementation: /root/code/tinker_tickets/ (markdown.js, base.js, ticket.css)
This applies without exception to every task marked [IMPROVE], [Build], or any UI change.

Design Rules

  • All new components must respect both TDS dark (LotusTerminalTheme) and TDS light (LotusTerminalLightTheme) modes
  • Non-TDS theme work (custom accent color, theme presets) uses vanilla-extract theme files — match the pattern in src/lotus-terminal.css.ts
  • Code syntax highlighting token classes: .tok-kw .tok-str .tok-num .tok-cmt .tok-fn (defined in web_template/base.css)
  • folds AvatarImage does NOT accept children — wrap Avatar components externally for overlays/frames/borders

CI/CD Pipeline

edit → commit → git push origin lotus
→ Gitea Actions: tsc --noEmit, eslint, prettier (~3 min)
→ Webhook: lotus_deploy.sh on LXC 106 polls CI, then npm ci && npm run build → rsync
→ Live at chat.lotusguild.org (~11 min total)

Per-Feature Checklist (before marking complete)

  • npx tsc --noEmit — zero TypeScript errors
  • npx eslint src/ — zero new errors (warnings OK if pre-existing)
  • npx prettier --check src/ — formatting passes
  • README.md updated (Lotus-custom features only — not upstream Cinny features)
  • landing/index.html updated if the feature appears in the comparison table
  • Visually tested at chat.lotusguild.org after CI deploys

Homeserver Access (for server audits)

  • Synapse (Matrix): LXC 151 on compute-storage-01 — pct exec 151 -- bash
  • Config: /etc/matrix-synapse/homeserver.yaml
  • Version check: curl -s https://matrix.lotusguild.org/_matrix/client/versions
Reference in New Issue View Git Blame Copy Permalink