jared
7939dc92d4
- README "Calls & Voice": add the in-call soundboard, per-user call quality settings, and admin room call-permissions bullets. - LOTUS_TODO: mark the soundboard UI as built (was "cinny UI remains / dormant"). - HANDOFF_ELEMENT_CALL_FORK: add a COMPLETE status banner to the §12.1 host checklist; fix stale denoise specifics (all four models are in-source; flag is lotusDenoiseSource=1, not lotusDenoise=ml). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
725 lines
53 KiB
Markdown
725 lines
53 KiB
Markdown
# Lotus Chat — Work Backlog
|
||
|
||
**Repo:** `lotus` branch at `https://code.lotusguild.org/LotusGuild/cinny`
|
||
**Deploy:** push to `lotus` → CI → auto-deploy to `chat.lotusguild.org` (~11 min)
|
||
|
||
---
|
||
|
||
## ⚠️ TDS DESIGN LAW — READ BEFORE TOUCHING ANY UI
|
||
|
||
> **ALL Lotus Terminal Design System (TDS) styling — colors, animations, glows, borders, fonts, spacing — MUST come exclusively from `/root/code/web_template/base.css` CSS variables.**
|
||
> Do NOT hardcode hex values. Do NOT invent new variable names. Do NOT deviate from the design tokens defined in that file.
|
||
> The canonical variable reference: `--lt-accent-orange`, `--lt-accent-cyan`, `--lt-accent-green`, `--lt-glow-orange`, `--lt-box-glow-*`, `--lt-border-color`, etc.
|
||
> Reference implementation for code patterns: `/root/code/tinker_tickets/` (markdown.js, base.js, ticket.css)
|
||
> This rule applies to EVERY task in this file without exception.
|
||
|
||
---
|
||
|
||
## 🧩 NATIVE-CINNY LAW — EVERY FEATURE MUST FEEL LIKE STOCK CINNY
|
||
|
||
> **Every feature we implement must feel native to the upstream Cinny app — indistinguishable from something the Cinny team would have shipped.** Reference: <https://github.com/cinnyapp/cinny>.
|
||
>
|
||
> Concretely this means:
|
||
>
|
||
> - **Use the `folds` design system, not bespoke UI.** Build with folds primitives (`Button`, `Chip`, `IconButton`, `Menu`, `MenuItem`, `Dialog`, `Modal`, `Input`, `Switch`, `Badge`, `SettingTile`, `SequenceCard`, etc.) and folds tokens (`color.*`, `config.space.*`, `config.radii.*`, `config.borderWidth.*`). No hardcoded hex/`rgba()` for UI chrome, no invented/undefined CSS variables.
|
||
> - **Match Cinny's existing patterns.** Before adding UI, find the closest existing Cinny component/flow and mirror it (e.g. a new dropdown uses `Button`+`PopOut`+`Menu`+`MenuItem` like the rest; a new modal has a `Header` with a close `IconButton`; a new setting is a `SettingTile` inside a `SequenceCard`). Consistency with stock Cinny beats personal style.
|
||
> - **Lotus-custom additions should be unobtrusive** and fit Cinny's visual language, spacing, and interaction conventions — a stranger using Cinny should not be able to tell which features are ours.
|
||
>
|
||
> **The ONE exception:** explicit **Lotus Terminal Design System (TDS)** features, which intentionally have their own distinct look and follow the **TDS Design Law** above. TDS styling is opt-in (only active in Lotus Terminal mode); everything else must look and feel like native Cinny.
|
||
|
||
---
|
||
|
||
Completed features are documented in [LOTUS_FEATURES.md](./LOTUS_FEATURES.md).
|
||
|
||
---
|
||
|
||
## ✅ Done — Awaiting Verification
|
||
|
||
Built and gate-green; verify per [LOTUS_TESTING.md](./LOTUS_TESTING.md), then they graduate to LOTUS_FEATURES.md. (Bug-side fixes awaiting verification live in LOTUS_BUGS.md.)
|
||
|
||
| Feature | Test guide |
|
||
| :-------------------------------------------------------------------------------- | :---------------- |
|
||
| Full-Screen Camera Broadcasts (per-participant focus) | A5 / G2 |
|
||
| Advanced search filters (sender/date/has-link/has:image·file·video/pinned/recent) | K2 / M1 / M2 / M4 |
|
||
| Custom Accent Color Picker (non-TDS themes) | M3 |
|
||
| 5 Color Theme Presets (Cyberpunk/Ocean/Blood Red/Matrix/Midnight) | M5 |
|
||
| Intersection-based lazy media loading | H1 |
|
||
| Context-aware thumbnail previews | H2 |
|
||
| Desktop — proactive update notifications (Tauri) | J1 |
|
||
| Remind Me Later | K1 |
|
||
| Mobile Bookmarks access | E5 |
|
||
| In-Call Soundboard (P5-15, uploadable clips → real call inject) | D2-7 |
|
||
| Call Quality Controls (P5-31, user + room-admin caps) | D2-8 |
|
||
| Call Permissions (P5-31, hard server-side screenshare/camera policy) | D2-9 |
|
||
|
||
---
|
||
|
||
Legend:
|
||
|
||
- `[AUDIT REQUIRED]` — at least one assumption needs code/server verification before implementing
|
||
- `[SERVER CHECK]` — depends on a Synapse feature or MSC; verify on `matrix.lotusguild.org`
|
||
- `[LOW PRIORITY]` — implement after all higher-priority items
|
||
- `[EXTREME COMPLEXITY]` — multi-sprint, plan separately before touching
|
||
- `[BLOCKED]` — cannot build until a server upgrade, upstream MSC, or dependency resolves
|
||
- `[IMPROVE]` — feature exists in upstream Cinny; this task enhances it for Lotus Chat
|
||
|
||
Status: `[ ]` pending · `[~]` in progress · `[x]` completed
|
||
|
||
---
|
||
|
||
## Server Capabilities (as of June 2026)
|
||
|
||
- **Homeserver:** `matrix.lotusguild.org`
|
||
- **Synapse version:** `1.155.0` (2026-06-18) — fully up to date; last version for Debian 12 (LXC 151 already on Debian 13 Trixie)
|
||
- **Matrix spec:** up to `v1.12` formally; newer MSC features via `unstable_features`
|
||
|
||
### Confirmed facts
|
||
|
||
| Finding | Impact |
|
||
| -------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
|
||
| **MSC flags ON:** `msc4140` · `msc3771` · `msc3440.stable` · `msc4133.stable` · `simplified_msc3575` · `msc4222` · `msc3266` · `msc3401_matrix_rtc` | All safe to use now |
|
||
| **MSC flags OFF:** `msc4306` (thread subscriptions) · `msc3882` · `msc3912` · `msc4155` | These features are BLOCKED |
|
||
| **MSC3266** room summary: flag `msc3266_enabled: true` set but `GET /v1/rooms/{id}/summary` still returns 404 (M_UNRECOGNIZED) | Room Preview BLOCKED — endpoint not implemented in Synapse 1.155 |
|
||
| **MSC3892** relation redaction: not in flags | Reaction Redaction feature BLOCKED |
|
||
| **MSC4260** report user: `POST /_matrix/client/v3/users/{userId}/report` returns **200** ✅ | **Report User UNBLOCKED** — endpoint live since Synapse 1.133; ready to build |
|
||
| **MSC4151** report room: HTTP 405 on GET = endpoint exists (POST only) | Report Room live ✅ |
|
||
| `folds AvatarImage` does NOT accept children | Add frame/overlay inside `UserAvatar.tsx` itself — optional `frameName` prop |
|
||
| No in-app toast system exists (was) | Built `ToastProvider` + Jotai queue; at `App.tsx:65` |
|
||
| `useUnverifiedDeviceCount()` hook exists | `src/app/hooks/useDeviceVerificationStatus.ts:65-106` |
|
||
| Voice player: `AudioContent.tsx:44-223` | Playback rate on hidden `<audio>` at line 217 |
|
||
| `CallControl.setMicrophone(bool)` at `CallControl.ts:206-212` | For AFK auto-mute |
|
||
| `CallControl.toggleSound()` at `CallControl.ts:230-251` | Push-to-deafen — just wire a hotkey to this |
|
||
| matrix-js-sdk has NO arbitrary profile field methods | Use `mx.http.authedRequest()` for MSC4133 |
|
||
| Sanitizer (`sanitize.ts`) allows table, div, span, a, code, hr | LFG HTML card is safe locally; test on Element/FluffyChat |
|
||
| Sanitizer STRIPS `<math>`/MathML tags | Math/LaTeX task must also modify sanitizer |
|
||
| Service worker EXISTS at `src/sw.ts` | Quick-reply task: add `notificationclick` handler |
|
||
| `knockSupported()` utility exists at `matrix.ts:376-391` | Knock UX: only need "Request to Join" in `RoomIntro.tsx` |
|
||
| `KeywordMessages.tsx` already has custom keyword push rules | Full push rule editor: only non-keyword rule types need new UI |
|
||
| `getMatrixToRoom()` in `matrix-to.ts` generates invite URLs | Invite link: just add QR code to room settings |
|
||
| ~~Cindy CANNOT inject audio into EC call stream~~ **UNBLOCKED by EC fork** — `io.lotus.inject_audio` widget action publishes a clip as a real call track | In-call soundboard CAN now mix into the call (no longer local-only); needs cinny UI to drive the action |
|
||
| Folds uses vanilla-extract in non-TDS, NOT CSS custom properties | Custom accent color: must create new vanilla-extract theme variant dynamically |
|
||
| Theme presets need ~50 CSS custom properties each | Significant design work before coding |
|
||
| `useCallSpeakers.ts` CSS MutationObserver polling | Visual speaking indicator: TDS ring animation on top of existing data |
|
||
| MSC3489/3672 live location: BOTH false on server | Live Location BLOCKED |
|
||
|
||
---
|
||
|
||
## Key File Reference
|
||
|
||
| What you need | File | Lines |
|
||
| -------------------------------- | ------------------------------------------------------------- | ------------------- |
|
||
| Global keydown hook | `src/app/hooks/useKeyDown.ts` | whole file |
|
||
| Room navigation | `src/app/hooks/useRoomNavigate.ts` | 19-72 |
|
||
| All room IDs atom | `src/app/state/room-list/roomList.ts` | `allRoomsAtom` |
|
||
| Room unread counts | `src/app/state/room/roomToUnread.ts` | `roomToUnreadAtom` |
|
||
| Overlay portal provider | `src/app/pages/App.tsx` | 65 |
|
||
| Portal container div | `index.html` | 101 |
|
||
| Room settings tabs | `src/app/features/room-settings/RoomSettings.tsx` | 27-56 |
|
||
| State event read/write pattern | `src/app/features/common-settings/general/RoomEncryption.tsx` | 42-52 |
|
||
| Power level checker | `src/app/hooks/usePowerLevels.ts` | whole file |
|
||
| Slash command registration | `src/app/hooks/useCommands.ts` | 140-537 |
|
||
| Chat background picker | `src/app/features/settings/general/General.tsx` | 945-981 |
|
||
| Chat backgrounds definition | `src/app/features/lotus/chatBackground.ts` | whole file |
|
||
| Matrix.to URL builder | `src/app/plugins/matrix-to.ts` | `getMatrixToRoom()` |
|
||
| Media event content types | `src/app/types/matrix/common.ts` | 46-91 |
|
||
| Media URL conversion | `src/app/utils/matrix.ts` | `mxcUrlToHttp()` |
|
||
| Message pagination (search) | `src/app/features/message-search/useMessageSearch.ts` | 74-121 |
|
||
| Infinite pagination pattern | `src/app/features/message-search/MessageSearch.tsx` | 234-365 |
|
||
| Poll event format | `src/app/components/message/content/PollContent.tsx` | 1-320 |
|
||
| Theme class application | `src/app/hooks/useTheme.ts` | 25-60 |
|
||
| Animations file | `src/app/styles/Animations.css.ts` | whole file |
|
||
| Message status (EventStatus) | `src/app/features/room/message/Message.tsx` | 84-142 |
|
||
| Call member change events | `src/app/hooks/useCall.ts` | 37-52 |
|
||
| Mic control in calls | `src/app/plugins/call/CallControl.ts` | 206-212 |
|
||
| Device verification hook | `src/app/hooks/useDeviceVerificationStatus.ts` | 65-106 |
|
||
| Knock room support check | `src/app/utils/matrix.ts` | 376-391 |
|
||
| Room join button location | `src/app/components/room-intro/RoomIntro.tsx` | 25-119 |
|
||
| Notification mute via push rules | `src/app/hooks/useRoomsNotificationPreferences.ts` | 110-150 |
|
||
| Message text body CSS | `src/app/components/message/layout/layout.css.ts` | 182-205 |
|
||
|
||
---
|
||
|
||
## Priority 3 — Higher complexity / lower daily frequency
|
||
|
||
### [ ] P3-4 · Accessibility Improvements (WCAG 2.1 AA)
|
||
|
||
**What:** Comprehensive audit and fix pass targeting the critical user paths:
|
||
|
||
- Room list navigation (keyboard-only)
|
||
- Reading messages in the timeline (screen reader announces new messages)
|
||
- Composing and sending a reply
|
||
- Opening and closing modals (focus trap, return focus)
|
||
- ARIA labels on all icon-only buttons
|
||
|
||
**Scope:** Do NOT attempt to make every corner of the app AA-compliant in one pass — focus on the golden path (open app → find room → read → reply → send).
|
||
**[AUDIT REQUIRED]** — Run an automated audit first: `npx axe-core` or browser DevTools accessibility tree. Document every violation before writing a single line of code. Prioritize by severity (critical > serious > moderate).
|
||
|
||
**Investigation Findings:**
|
||
|
||
- **Root Cause:** Inconsistent focus management, missing `aria-live` regions for dynamic timeline updates, and sparse global keyboard shortcuts.
|
||
- **Approach:** Standardize `focus-trap-react` usage (reference `RoomNavItem.tsx`). Add `aria-live` regions to the timeline. Expand `useKeyDown.ts` for section navigation shortcuts.
|
||
- **Complexity:** Medium-High (audit is the main work).
|
||
|
||
---
|
||
|
||
### [ ] P3-8 · Thread Panel (full side drawer)
|
||
|
||
**⚠️ LARGEST FEATURE — requires its own planning session before implementation.**
|
||
**What:** A right-side drawer for threaded conversations. Currently "Reply in Thread" exists but there is no panel to read or write thread replies.
|
||
|
||
Features:
|
||
|
||
- Click "Reply in Thread" → opens thread drawer on the right
|
||
- Thread root event shown at the top of the panel
|
||
- Full message rendering for all in-thread replies (reuse timeline components)
|
||
- Reply input at the bottom (full composer with formatting, emoji, etc.)
|
||
- Unread count badge on the thread button in the main timeline
|
||
- Keyboard shortcut to close thread panel
|
||
|
||
**Architecture:**
|
||
|
||
- New Jotai atom: `activeThreadEventId: string | null`
|
||
- New component: `src/app/features/room/thread/ThreadPanel.tsx`
|
||
- Rendered alongside `RoomView` as a conditional right panel (mirror the members drawer pattern)
|
||
- Filter events in timeline to `m.thread` relation for the active root event ID
|
||
- Shares the same `mx` client and room reference as the main timeline
|
||
|
||
**[AUDIT REQUIRED]** — Deeply audit how `m.thread` relation events are currently stored and retrieved in the matrix-js-sdk. Understand the thread aggregation API: `GET /rooms/{roomId}/relations/{eventId}/m.thread`. Check if `RoomTimeline.tsx` currently filters out thread replies from the main timeline (it should — confirm).
|
||
|
||
**Investigation Findings:**
|
||
|
||
- **Root Cause:** Current `m.thread` events are treated as standard `m.room.message` events and rendered in the main timeline.
|
||
- **Approach:** Introduce new Jotai atom `activeThreadEventId`. Create `ThreadPanel.tsx`. Update `RoomTimeline.tsx` to filter out thread relations (`m.relates_to`). Implement aggregation fetch using `GET /rooms/{roomId}/relations/{eventId}/m.thread`. Use `thread.timelineSet` directly for the most accurate thread view.
|
||
- **Complexity:** High.
|
||
|
||
---
|
||
|
||
## Priority 4 — Specialized, high complexity, or low priority
|
||
|
||
### [ ] P4-7 · Virtualized Infinite Scroll for Search Results
|
||
|
||
**What:** Replace the manual "load more" button with an automated, virtualized infinite scroll for search results.
|
||
**Approach:** Utilize `@tanstack/react-virtual` in `MessageSearch.tsx` to handle the `nextToken` automatically as the user scrolls.
|
||
|
||
### [ ] P4-8 · Encrypted Message Search Indexing & Caching
|
||
|
||
**What:** Implement a persistent local cache for search results, optimized for encrypted rooms.
|
||
**Approach:** Use `IndexedDB` to store search metadata (event IDs, timestamps) to prevent redundant server-side decryption/fetching.
|
||
|
||
### [ ] P4-1 · Thread Notification Mode Per-Thread (MSC3771)
|
||
|
||
**Spec:** MSC3771 (stable). Depends on Thread Panel (#P3-8).
|
||
**What:** Per-thread notification toggle: "All messages" vs "Mentions only". Accessible from the thread panel header. Tracks unread counts separately per thread.
|
||
**[AUDIT REQUIRED]** — Implement after Thread Panel. Requires understanding how the SDK tracks per-thread unread counts.
|
||
**Complexity:** Medium (after thread panel exists).
|
||
|
||
---
|
||
|
||
### [ ] P4-2 · Thread Subscriptions (MSC4306) [BLOCKED]
|
||
|
||
**Spec:** MSC4306 (Synapse experimental). Depends on Thread Panel (#P3-8).
|
||
**What:** "Follow thread" button to receive notifications for a thread you haven't posted in. Uses MSC4306 subscription endpoint.
|
||
**[SERVER CHECK]** — `org.matrix.msc4306 = false` on `matrix.lotusguild.org` — BLOCKED until server enables it.
|
||
**Complexity:** Medium (after thread panel exists).
|
||
|
||
---
|
||
|
||
### [ ] P4-4 · Math / LaTeX Rendering in Messages (LOW PRIORITY)
|
||
|
||
**Spec:** CS-API §11.5 (stable) — `formatted_body` can contain LaTeX.
|
||
**What:** Render `$...$` or `$$...$$` LaTeX expressions in message bodies. Use KaTeX (lightweight, ~100KB, renders server-side-compatible CSS). Must gracefully fall back to raw LaTeX text if KaTeX fails.
|
||
**Note:** This is LOW PRIORITY — only useful for academic/technical communities. Implement last.
|
||
**[AUDIT REQUIRED]** — Confirm KaTeX bundle size impact on the Vite bundle. Check if matrix-js-sdk's HTML sanitizer strips LaTeX before it reaches the renderer. The formatted_body sanitization pipeline is the main risk here. (Confirmed: sanitizer STRIPS `<math>` tags — must be patched alongside the renderer.)
|
||
**Complexity:** Low-Medium.
|
||
|
||
---
|
||
|
||
### [ ] P4-5 · Live Location Sharing (MSC3489 + MSC3672) (LOW PRIORITY, HIGH COMPLEXITY) [BLOCKED]
|
||
|
||
**Spec:** MSC3489 + MSC3672. Implemented in Element Web.
|
||
**Note:** Static location sharing is already implemented. This adds live/real-time GPS beacons. Very low priority per user preference.
|
||
**What:** Start sharing live location → creates `m.beacon_info` state event → client posts `m.beacon` events on a timer → other users see your position update live on a map.
|
||
**[SERVER CHECK]** — `org.matrix.msc3489 = false` AND `org.matrix.msc3672 = false` on `matrix.lotusguild.org` — BLOCKED.
|
||
**Complexity:** High. Requires background geolocation API + live map rendering.
|
||
|
||
---
|
||
|
||
### [~] P4-6 · OIDC / SSO Next-Gen Auth (MSC3861) — CLIENT-SIDE BUILT, awaiting live verification
|
||
|
||
**Spec:** MSC3861 / MSC2965, Matrix spec v1.15. OAuth2-native auth via a Matrix Authentication Service (MAS).
|
||
**Scope decision (2026-06):** CLIENT-ONLY. We implemented OIDC login _in the Lotus client_ so it can sign into next-gen homeservers (mozilla.org, eventually matrix.org). We deliberately did **not** convert lotusguild's own Synapse to MAS (no account migration; lotusguild keeps password + legacy Authelia SSO).
|
||
**Built (matrix-js-sdk already ships the OIDC API; this was wiring):**
|
||
|
||
- Discovery: `cs-api.ts` `getOidcIssuer()` (stable `m.authentication` + msc2965). Flow hint: `useParsedLoginFlows` `getOidcCompatibilityFlag()` (MSC3824).
|
||
- Login: `pages/auth/oidc/{oidcConfig,oidcLoginUtil,oidcState}.ts` (dynamic registration + cache, PKCE authorize), `login/OidcLogin.tsx`, issuer-gated `Login.tsx`.
|
||
- Callback: `oidc/OidcCallback.tsx` + `App.tsx` short-circuit (non-hash redirect path).
|
||
- Session/refresh: `state/sessions.ts` OIDC fields, `client/{oidcTokenRefresher,oidcLogout}.ts`, `initMatrix.ts` wiring.
|
||
- Account mgmt: `settings/account/OidcManageAccount.tsx`.
|
||
- 13 unit tests (discovery/flow/session/cache/callback parsing). All gates green.
|
||
**Awaiting verification (needs a real MSC3861 server — lotusguild is NOT one):** deploy + log into **mozilla.org** (requires adding mozilla to the deployed `config.json` homeserverList + its domains to the CSP `connect-src`/`img-src` — see below), OR run a local `matrix-authentication-service` + Synapse `msc3861` dev loop.
|
||
**To enable the mozilla.org test:** add to `matrix/cinny/config.json` homeserverList `"mozilla.org"`, and to the nginx CSP `connect-src`/`img-src`: `https://mozilla.org https://mozilla.modular.im https://chat.mozilla.org https://vector.im`.
|
||
|
||
---
|
||
|
||
## Priority 5 — Gamer / Aesthetic / Customization
|
||
|
||
### [MOVED] P5-9 · LFG (Looking for Group) Command → LotusBot
|
||
|
||
**Decision:** Implemented as `!lfg` in LotusBot rather than a client slash command. Bot-side rendering works consistently across all Matrix clients; client-side enhanced cards would only be visible to Lotus Chat users and require sanitizer auditing. The bot can also support richer flows (list active LFGs, DM interested players, auto-expire posts).
|
||
|
||
---
|
||
|
||
### [~] P5-15 · In-Call Soundboard — IMPLEMENTED (⚠️ awaiting live verification, D2-7)
|
||
|
||
**What:** Soundboard button in the call controls bar → popout grid of the user's clips; clicking one plays it **into the call** as a real published track (peers hear it) and locally (presser hears it). Clips are **user-uploadable, just like custom emojis/stickers**.
|
||
**🔱 [EC-FORK] Fork side + cinny side DONE.** The fork ships `io.lotus.inject_audio` (`LotusWidgetActions.InjectAudio`, allow-listed in `widget.ts`), armed via the `lotusAudioInject=1` flag; it publishes a clip as a separate LiveKit track — a **real** in-call soundboard mixed into the call, not local-only. cinny now drives it.
|
||
**Shipped (cinny):**
|
||
|
||
- Clips stored in `io.lotus.soundboard` account data → **synced across devices like emoji/sticker packs** (`useSoundboard` hook; `AccountDataEvent.LotusSoundboard`).
|
||
- Upload audio (≤1 MB, ≤40 clips) → `mx.uploadContent` → mxc; play resolves mxc → authed download → `blob:` object URL (the widget can't fetch authenticated media itself) → `control.injectAudio(url, volume)` + local playback.
|
||
- `CallSoundboard.tsx` popout in the call bar (upload / play / delete), gated on the `soundboardEnabled` setting (Settings → General → Calls, + volume slider).
|
||
**Remaining:** a dedicated Settings management page (optional — upload/delete already live in the popout); a small default clip set; live verification (D2-7). Files: `utils/soundboardClips.ts`, `hooks/useSoundboard.ts`, `features/call/CallSoundboard.tsx`, `plugins/call/CallControl.ts#injectAudio`.
|
||
**Complexity:** Medium — done.
|
||
|
||
---
|
||
|
||
### [~] P5-20 · Quick Reply from Browser Notification
|
||
|
||
**What:** Inline reply field in browser notification toasts via Notification Actions API. Reply sends as threaded reply to the triggering message.
|
||
**[AUDIT REQUIRED]** (1) Verify browser Notification Actions API support in target browsers. (2) Confirmed: service worker EXISTS at `src/sw.ts` — add `notificationclick` handler there.
|
||
**Complexity:** Medium-High.
|
||
**Partial Fix Applied ⚠️ UNTESTED:** Notifications now (a) show the real message body (`username: message` instead of "New inbox notification from..."), (b) click navigates directly to the room at the specific event (not the inbox), (c) `window.focus()` called on click so the tab comes to front, (d) reminder toasts also link to the specific event. Full inline-reply via Notification Actions API still needs the SW `push`+`notificationclick` pipeline (requires switching from `new Notification()` to `showNotification()` through the SW).
|
||
|
||
---
|
||
|
||
### [x] P5-30 · Advanced ML Noise Suppression (Krisp-style)
|
||
|
||
**What:** High-end background noise cancellation using a pre-trained ML model (RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
|
||
**Shipped:** 3-tier setting (Off / Browser-native / ML) in Settings → General → Calls.
|
||
**🔱 [EC-FORK] DONE — moved in-source (2026-06).** ML denoise is now a first-class audio stage **inside** the forked Element Call: a LiveKit `TrackProcessor<Audio>` activated by `lotusDenoiseSource=1` (cinny sets it when ML is selected). The old build-time `getUserMedia`/`index.html` monkeypatch is **removed**. Because EC re-runs the processor on every (re)publish, denoise now **survives reconnects and mic-device switches** — this is the A7 fix (see `LOTUS_BUGS.md` A7, `LOTUS_TESTING.md` §D2-1). The processor degrades to the raw mic rather than going silent.
|
||
**Key decision:** LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU); EC's own RNNoise PR #3892 is unmerged. Owning the fork let us implement the in-source stage directly.
|
||
|
||
**Models — all in-source in the fork:**
|
||
|
||
- [x] **RNNoise** (48 kHz, default) · **Speex** (48 kHz) · **DTLN** (16 kHz) · **DeepFilterNet 3** (48 kHz) — all four wired and selectable.
|
||
- [ ] **Open verification:** real-call **audio-quality** comparison across the four models (RNNoise output is known-weak). Track under the denoise quality project, `LOTUS_TESTING.md` §D2-1 / J2.
|
||
- [ ] **Desktop-only / HW-gated (future):** FRCRN or NVIDIA Maxine (RTX/Tensor only) — impossible in-browser; would run in the Tauri Rust backend + bridge a virtual mic into the webview. Detect capability; web falls back to RNNoise.
|
||
- **Excluded:** Krisp (LiveKit Cloud only); FRCRN/Maxine on web (GPU/server-bound).
|
||
|
||
---
|
||
|
||
### [~] P5-31 · Granular Voice & Screenshare Quality Controls — IMPLEMENTED (⚠️ awaiting live verification, D2-8)
|
||
|
||
**What:** Let users (and room admins) adjust audio bitrate and screenshare bitrate/framerate.
|
||
**🔱 [EC-FORK] Fork side + client side DONE.** The fork ships `io.lotus.set_quality` (`LotusWidgetActions.SetQuality`) that applies audio/screenshare encoding params (`RTCRtpSender.setParameters`, all simulcast encodings, re-applied on `TrackUnmuted`/republish) inside EC. cinny now drives it.
|
||
|
||
**Shipped (cinny):**
|
||
|
||
1. **User settings** (Settings → General → Calls): Microphone Bitrate, Screenshare Bitrate, Screenshare Framerate (`callAudioBitrate` / `screenshareBitrate` / `screenshareFramerate`).
|
||
2. **Room-admin caps**: `io.lotus.room_quality` state event (`StateEvent.LotusRoomQuality`) + `RoomQuality.tsx` in Room Settings → General → Voice (mirrors `RoomVoiceLimit`).
|
||
3. **Apply logic**: `useCallQuality` (wired in `CallEmbedProvider`'s `CallUtils`) builds `min(user setting, room cap)` and sends `io.lotus.set_quality` on join / when settings change (`utils/callQuality.ts`, unit-tested).
|
||
|
||
**Server-side enforcement (DONE — matrix repo):** extended `voice-limit-guard.py` (LXC 151) to also read `io.lotus.room_quality` and hard-enforce a **publish-source policy** for ALL clients.
|
||
|
||
- **Reality (researched, primary-source, LiveKit 1.9.11):** numeric bitrate/fps caps **cannot** be hard-enforced server-side — LiveKit is a pure SFU (forwards, never transcodes); there is NO bitrate/fps field in the JWT grant, `RoomConfiguration`, server `limit:` config, or any admin RPC, and stock Element Call ignores room metadata / custom claims for publish quality. So numeric caps stay **cooperative** (our fork honors them via `min()` → `set_quality`, already shipped).
|
||
- **What IS hard-enforced cross-client:** `VideoGrant.canPublishSources`. The guard holds the LiveKit secret, so when `io.lotus.room_quality` sets `allow_screenshare:false` / `allow_camera:false` it re-signs the issued JWT with a narrowed source list → the SFU refuses those tracks for **every** client (Element, FluffyChat, our fork). Mic always kept. Fail-open; unit-tested (`livekit/test_voice_limit_guard.py`). Admin UI: Room Settings → Voice → **Call Permissions** switches. cinny also hides the blocked buttons.
|
||
- **Live (mid-call) enforcement — DONE:** the JWT re-sign covers new joins; for participants **already in the call**, a background reconcile loop in the guard calls LiveKit `UpdateParticipant` every ~3 s to narrow `canPublishSources`, which unpublishes an in-progress screenshare/camera **server-side for all clients** and blocks re-publish (verified LiveKit 1.9.11 auto-unpublishes on permission narrowing). Only removes forbidden sources (never grants), preserves other permission flags, no-ops once compliant. So flipping a room audio-only kills live cameras/screenshares within ~one interval.
|
||
- **Not enforceable / deferred:** numeric server enforcement (impossible — see above); screenshare **resolution** control (`set_quality` covers bitrate + framerate; resolution needs a `getDisplayMedia` hook inside the fork).
|
||
|
||
**Complexity:** DONE — client (cooperative numeric caps) + server (hard publish-source policy). Only the physically-impossible numeric server enforcement is out of scope.
|
||
|
||
---
|
||
|
||
### [ ] P5-35 · Desktop — Notification Click Opens Room (DEFERRED)
|
||
|
||
**What:** Clicking a system tray notification navigates to the relevant room. Quick-reply from the notification toast would send the reply without opening the window.
|
||
**Status:** Deferred — `tauri-plugin-notification` has no Rust click/action callback API. Quick-reply would need a custom WinRT toast activator + COM registration, which can't be compile-tested without a Windows build environment.
|
||
**Note:** Tray icon and `matrix:` deep links already bring the window forward on most interactions. Revisit when tauri-plugin-notification gains click handler support upstream.
|
||
**Complexity:** High (platform-specific native code required).
|
||
|
||
---
|
||
|
||
### [ ] P5-36 · Desktop — Windows Jump List (DEFERRED)
|
||
|
||
**What:** Right-clicking the taskbar icon shows a jump list with recent/favorite rooms for quick navigation.
|
||
**Status:** Deferred — implementing the Windows COM jump list API in Tauri requires iterating on C++/COM code that can only be compile-checked on Windows, making blind CI iteration impractical.
|
||
**Action when unblocked:** Revisit when a Tauri plugin abstracts the Windows Shell `ICustomDestinationList` interface, or when a Windows build environment is available for local iteration.
|
||
**Complexity:** High (Windows-only native COM).
|
||
|
||
---
|
||
|
||
### [ ] P5-41 · Desktop — Native WinRT Toast Notifications
|
||
|
||
**What:** Replace emulated notifications with native WinRT Toast notifications.
|
||
**Approach:** Implement native WinRT Toast integration using `windows-rs` to enable full Action Center integration, including native Quick Reply functionality.
|
||
|
||
### [ ] P5-42 · Desktop — Persistent Background Sync
|
||
|
||
**What:** Maintain light connection to homeserver when WebView2 is suspended.
|
||
**Approach:** Implement a headless Rust sidecar to fetch unread counts/notifications while the webview is suspended to ensure instant notification delivery.
|
||
|
||
### [ ] P5-43 · Desktop — System Media Transport Controls (SMTC)
|
||
|
||
**What:** Integrate with Windows SMTC for volume flyout call/media control.
|
||
**Approach:** Use Windows SMTC API to expose call status, mic mute/unmute, and media controls to the Windows volume flyout/media overlay.
|
||
|
||
### [ ] P5-44 · Desktop — Taskbar Thumbnail Toolbar
|
||
|
||
**What:** Add persistent call controls to the taskbar preview.
|
||
**Approach:** Implement a COM thumbnail toolbar in the application preview window, featuring Mute/Deafen/End Call buttons.
|
||
|
||
### [ ] P5-46 · Desktop — System Power Management (Call Continuity)
|
||
|
||
**What:** Prevent system sleep/hibernate during active calls.
|
||
**Approach:** Use Tauri/Rust `power-manager` or platform-specific APIs to block system power saving states while a voice/video session is active.
|
||
|
||
### [ ] P5-47 · Desktop — TDS-Styled Native Window Chrome
|
||
|
||
**What:** Replace system titlebar with custom Lotus TDS chrome.
|
||
**Approach:** Configure Tauri window (`decorations: false`) and implement custom, TDS-token compliant titlebar controls (Close/Max/Min) for a cohesive UI.
|
||
|
||
### [ ] P5-48 · Desktop — Native File System Drag-and-Drop Improvements
|
||
|
||
**What:** Enhance drag-and-drop support for Windows.
|
||
**Approach:** Improve handling for Windows file shortcuts, recursive folder uploads, and shell-integrated "Send To" context menu actions.
|
||
|
||
### [ ] P5-49 · Desktop — Network Awareness (NCSI Integration)
|
||
|
||
**What:** Proactively detect Windows network connectivity changes.
|
||
**Approach:** Integrate with the Windows Network Connectivity Status Indicator (NCSI) API to improve offline mode transition latency and network recovery.
|
||
|
||
### [ ] P5-50 · Desktop — Windows Hardware-Accelerated Media Pipeline
|
||
|
||
**What:** Replace standard browser decoding with native Windows Media Foundation.
|
||
**Approach:** Leverage DirectShow/Media Foundation to offload video/audio decoding from the CPU to the GPU, significantly reducing power consumption and latency during calls.
|
||
|
||
### [ ] P5-51 · Desktop — Federated "Identity Contexts" (Isolation Manager)
|
||
|
||
**What:** Compartmentalize sessions, local databases, and caches into isolated "Contexts."
|
||
**Approach:** Implement a zero-leak boundary for personas (e.g., Work vs. Personal) by isolating `IndexedDB`, filesystem caches, and session persistence per context.
|
||
**Priority:** Extreme Low (Multi-sprint/Architectural).
|
||
|
||
### [~] P5-52 · Desktop — Room-Level Sync Governor (Performance Control) [STILL_CONSIDERING]
|
||
|
||
**What:** Granular sync tuning for individual rooms.
|
||
**Approach:** Allow per-room overrides for sync frequency and event type filtering (e.g., disable read receipts/typing in heavy rooms) to optimize performance. Implementation requires careful UX to prevent complexity fatigue.
|
||
|
||
### [ ] P5-53 · Desktop — Local-Only "Scripting" Plugin System (Tampermonkey-like)
|
||
|
||
**What:** A sandboxed environment for local execution of user scripts on Matrix events.
|
||
**Approach:** Implement a WASM-based execution engine that allows users to write local-only, client-side scripts to interact with incoming Matrix events, trigger sounds/notifications, or inject custom UI elements based on event payload rules. Designed for privacy — all logic runs exclusively on the local machine.
|
||
|
||
### [ ] P5-55 · Desktop — Composer Toolbar Drag-and-Drop Reordering
|
||
|
||
**What:** Allow users to reorder toolbar icons via drag-and-drop.
|
||
**Approach:** Extend the current settings-based toolbar toggle system to include a drag-and-drop UI mode in the composer settings, allowing users to personalize their icon order.
|
||
|
||
### [ ] P5-56 · Desktop — Windows "Focus Assist" (DND) Sync
|
||
|
||
**What:** Automatically toggle notification state based on Windows Focus Assist.
|
||
**Approach:** Integrate with the Windows `NotificationCenter` / `Focus` state via Tauri/Rust to automatically enable/disable Lotus Chat's internal notification suppression mode when Windows Focus Assist is toggled.
|
||
|
||
### [ ] P5-57 · Desktop — Visual Draft Persistence Indicator
|
||
|
||
---
|
||
|
||
## 🚀 Features to Add
|
||
|
||
- [ ] **Mobile Audit:** Comprehensive audit of all features in LOTUS_FEATURES.md for mobile PWA usability and layout responsiveness.
|
||
|
||
---
|
||
|
||
## Blocked Features
|
||
|
||
These features are confirmed desirable but cannot be built until the listed dependency is resolved.
|
||
Check back after each Synapse upgrade — re-run `/matrix/client/versions` and `unstable_features` to see if they've become available.
|
||
|
||
### [BLOCKED] · Live Location Sharing (MSC3489 + MSC3672)
|
||
|
||
**Blocked by:** `org.matrix.msc3489 = false` AND `org.matrix.msc3672 = false` on `matrix.lotusguild.org` (confirmed from unstable_features).
|
||
**What it would do:** Real-time GPS beacon streaming upgrading the existing static location share.
|
||
**Action when unblocked:** Both MSCs must be enabled on the homeserver before any client work.
|
||
|
||
### [BLOCKED] · Reaction / Relation Redaction (MSC3892)
|
||
|
||
**Blocked by:** `org.matrix.msc3892` = false on `matrix.lotusguild.org`
|
||
**What it would do:** Cleanly remove a reaction without redacting the parent message.
|
||
**Current behavior:** Full event redaction — acceptable fallback, no user-facing issue.
|
||
**Action when unblocked:** Find `onReactionToggle` redaction call site; swap in MSC3892 endpoint with fallback.
|
||
|
||
### [BLOCKED] · Room Preview Before Joining (MSC3266)
|
||
|
||
**Blocked by:** `GET /_matrix/client/v1/rooms/{roomId}/summary` returns `M_UNRECOGNIZED` 404 — endpoint not implemented in Synapse 1.155. Config flag `msc3266_enabled: true` is set but has no effect; Synapse appears not to have shipped a stable implementation at the v1 path. Verified 2026-06-18.
|
||
**What it would do:** Show room name, topic, avatar, member count before joining.
|
||
**Action when unblocked:** Re-test after each future Synapse upgrade.
|
||
|
||
### [BLOCKED] · Thread Subscriptions (MSC4306)
|
||
|
||
**Blocked by:** `org.matrix.msc4306` = false on `matrix.lotusguild.org`
|
||
**What it would do:** Follow a thread without posting; get notifications for replies.
|
||
**Action when unblocked:** Add "Follow thread" button in the thread panel header (depends on #P3-8 Thread Panel).
|
||
|
||
### [DONE] · Report User (MSC4260) ✅
|
||
|
||
**Previously blocked by:** Server spec v1.12, but `POST /_matrix/client/v3/users/{userId}/report` was confirmed **200** on 2026-06-18 (live since Synapse 1.133.0).
|
||
**What it does:** Reports a specific user to homeserver admins (separate from reporting a message).
|
||
**Note:** Report Message already exists in upstream Cinny. This adds Report User to the profile panel.
|
||
**Implemented 2026-06-18:** `ReportUserModal.tsx` added at `src/app/features/room/ReportUserModal.tsx`. Button wired into `UserRoomProfile.tsx` between UserModeration and UserDeviceSessions (hidden for own profile). Category dropdown + reason text, inline success/error feedback, auto-close 1500ms after success.
|
||
|
||
---
|
||
|
||
## Pending Audits
|
||
|
||
### [ ] Audit-3 · Profile banner image — Matrix protocol support
|
||
|
||
Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banner field. `uk.tcpip.msc4133.stable = true` on our server — check if a `banner_url` or similar field is defined. If no cross-client standard exists, do not implement.
|
||
|
||
---
|
||
|
||
## 📚 Implementation Reference
|
||
|
||
Exhaustive, low-level implementation details for backlog items. Follow these patterns to ensure code is "Lotus-perfect" (idiomatic, performant, and TDS-compliant).
|
||
|
||
### P3-8 · Thread Panel (Full Side Drawer)
|
||
|
||
**Architecture:** Mirror the `MembersDrawer` pattern but with a specialized timeline.
|
||
|
||
- **State (`src/app/state/room/thread.ts`):**
|
||
```typescript
|
||
export const activeThreadIdAtom = atom<string | null>(null);
|
||
```
|
||
- **Layout (`src/app/features/room/Room.tsx`):** Insert `ThreadPanel` conditionally alongside `RoomTimeline`:
|
||
```tsx
|
||
{
|
||
activeThreadId && (
|
||
<>
|
||
<Line variant="Background" direction="Vertical" size="300" />
|
||
<ThreadPanel roomId={roomId} threadId={activeThreadId} />
|
||
</>
|
||
);
|
||
}
|
||
```
|
||
- **Component (`src/app/features/room/thread/ThreadPanel.tsx`):** Use `room.getThread(threadId)` from the SDK. Render a `Header` with a "Close" button that sets `activeThreadIdAtom` to `null`. Reuse `RoomTimeline` but pass a filtered `EventTimelineSet`. Use `thread.timelineSet` directly for the most accurate thread view.
|
||
|
||
---
|
||
|
||
### P4-4 · Math / LaTeX Rendering
|
||
|
||
**Mechanism:** KaTeX injection into the HTML parser.
|
||
|
||
- **Sanitizer (`src/app/utils/sanitize.ts`):** Allow KaTeX-specific tags and classes (e.g., `span`, `annotation`, `math`). Use a specialized allowed list for math blocks.
|
||
> [Gemini_Found] `sanitize.ts` uses **`sanitize-html`** (not DOMPurify) with an explicit allowlist (`allowedTags`) and `disallowedTagsMode: 'discard'`. All MathML tags are currently absent from the allowlist and are silently stripped. Update `permittedHtmlTags` to include: `<math>`, `<mi>`, `<mo>`, `<mn>`, `<ms>`, `<mtext>`, `<mspace>`, `<mrow>`, `<mfrac>`, `<msqrt>`, `<mroot>`, `<mstyle>`, `<merror>`, `<mpadded>`, `<mphantom>`, `<mfenced>`, `<menclose>`, `<msub>`, `<msup>`, `<msubsup>`, `<munder>`, `<mover>`, `<munderover>`, `<mmultiscripts>`, `<mtable>`, `<mtr>`, `<mtd>`, `<maligngroup>`, `<malignmark>`, and `annotation`. Also add the required MathML attributes (e.g. `xmlns`, `display`, `mathvariant`) to `permittedTagToAttributes`.
|
||
- **Parser (`src/app/plugins/react-custom-html-parser.tsx`):** Detect `$ ... $` and `$$ ... $$` patterns in text nodes:
|
||
```tsx
|
||
if (node.type === 'text') {
|
||
const parts = node.data.split(/(\$\$.*?\$\$|\$.*?\$)/g);
|
||
return parts.map((p) => {
|
||
if (p.startsWith('$')) return <KaTeX math={p.replace(/\$/g, '')} />;
|
||
return p;
|
||
});
|
||
}
|
||
```
|
||
- **CSS (`src/app/styles/CustomHtml.css.ts`):** Import `katex/dist/katex.min.css` only when a math block is rendered to save initial bundle size.
|
||
|
||
---
|
||
|
||
### P4-6 · OIDC / SSO Next-Gen Auth (MSC3861)
|
||
|
||
**Mechanism:** Matrix Authentication Service (MAS) Integration.
|
||
|
||
- **Architecture:** Shift from password-based `/login` to OAuth2 `authorization_code` flow.
|
||
- **Key Files:** `src/app/pages/auth/Login.tsx` and `src/app/hooks/useAuth.ts`.
|
||
- **Implementation:** Use `oidc-client-ts` or a similar lightweight OIDC library. Check for `m.authentication` in `/.well-known/matrix/client`. Redirect to the MAS authorization endpoint. Handle the callback in a new `OidcCallback` route and store the OIDC `refresh_token`.
|
||
|
||
---
|
||
|
||
### P5-1 · Custom Accent Color Picker (Non-TDS only)
|
||
|
||
**Mechanism:** Dynamic CSS variable injection.
|
||
|
||
- **Setting (`src/app/state/settings.ts`):** Add `customAccentColor: string` (hex).
|
||
- **Manager (`src/app/pages/ThemeManager.tsx`):** Inside the `useEffect` that monitors theme changes:
|
||
```typescript
|
||
if (!lotusTerminal && customAccentColor) {
|
||
document.documentElement.style.setProperty('--lt-accent-orange', customAccentColor);
|
||
document.documentElement.style.setProperty('--lt-accent-orange-glow', `${customAccentColor}80`);
|
||
}
|
||
```
|
||
- **UI (`src/app/features/settings/general/General.tsx`):** Use `<Input type="color">`. Hide this section if `lotusTerminal` is `true`.
|
||
|
||
---
|
||
|
||
### P5-15 · In-Call Soundboard
|
||
|
||
**Mechanism:** Local-to-Global Audio Bridge via Web Audio API.
|
||
|
||
- Create an `AudioContext` and a `MediaStreamDestinationNode`.
|
||
- Create an `AudioBufferSourceNode` for each clip.
|
||
- Route the mic `MediaStream` and the clip source to the destination node.
|
||
- Pass the destination's `.stream` to the call bridge.
|
||
|
||
> ⚠️ **[Gemini_Found — CORRECTED]** Gemini originally suggested using LiveKit's `LocalAudioTrack.replaceTrack()` to mix audio into the call stream. This is **not possible** from Lotus Chat's realm: Element Call runs in a **cross-origin iframe** controlled via `matrix-widget-api` (postMessage). LiveKit's JS SDK and its `LocalAudioTrack` live inside EC's sandboxed context — inaccessible from our code. This directly contradicts the confirmed constraint already listed in the Server Capabilities table: _"Cindy CANNOT inject audio into EC call stream — In-call soundboard must be redesigned as local-only."_ The soundboard must be a local-playback-only feature (output through the user's speakers, not mixed into the call audio stream).
|
||
>
|
||
> 🔱 **[EC-FORK — RESOLVED]** Both the original claim and the earlier "practical blocker still holds" correction are now **outdated**. EC is same-origin **and** we own the source, so we no longer reach into EC's module scope from cinny — instead the fork **exposes the inject point itself**: the `io.lotus.inject_audio` widget action (`LotusWidgetActions.InjectAudio`) publishes a clip as a separate LiveKit track from inside EC. A **real** in-call soundboard (mixed into the call, not local-only) is therefore unblocked, and the cinny-side soundboard UI is now **built** (P5-15 above): uploadable clips played into the call via this action, stored in `io.lotus.soundboard` account data.
|
||
|
||
---
|
||
|
||
### P5-20 · Quick Reply from Browser Notification
|
||
|
||
**Mechanism:** Service Worker `notificationclick` Action.
|
||
|
||
> [Gemini_Found] Implementation detail: `serviceWorkerRegistration.showNotification()` should be used instead of `new Notification()` so that the service worker can listen to the `notificationclick` event. `new Notification()` creates notifications that are bound to the client page, not the SW.
|
||
|
||
```typescript
|
||
// src/sw.ts
|
||
self.addEventListener('notificationclick', (event) => {
|
||
if (event.action === 'reply' && event.reply) {
|
||
const { roomId, threadId } = event.notification.data;
|
||
const session = sessions.get(event.clientId);
|
||
fetch(`${session.baseUrl}/_matrix/client/v3/rooms/${roomId}/send/m.room.message`, {
|
||
method: 'POST',
|
||
headers: { Authorization: `Bearer ${session.accessToken}` },
|
||
body: JSON.stringify({
|
||
msgtype: 'm.text',
|
||
body: event.reply,
|
||
'm.relates_to': threadId ? { rel_type: 'm.thread', event_id: threadId } : undefined,
|
||
}),
|
||
});
|
||
}
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
### P5-30 · Advanced ML Noise Suppression — Model Roadmap
|
||
|
||
See shipped implementation in LOTUS_FEATURES.md → "Noise Suppression (Advanced Multi-Tier)".
|
||
|
||
**Models status:**
|
||
|
||
- **RNNoise** (sapphi, 48 kHz) — ✅ working, default fallback. Keep — runs on any hardware.
|
||
- **Speex** (sapphi, 48 kHz) — ✅ working, low value; candidate to drop.
|
||
- **DTLN** (@workadventure, 16 kHz) — 🟡 wired; sample-rate fix applied (was robotic at 48 kHz). **TODO: verify in a real call.** Narrowband (16 kHz) = slightly telephone-y even when correct.
|
||
|
||
**Constraints:** client-side AudioWorklet, fully self-hosted, no GPU, self-hosted SFU (no LiveKit Cloud).
|
||
|
||
**Roadmap:**
|
||
|
||
- [ ] Verify DTLN 16 kHz fix in a real call.
|
||
- [ ] **DeepFilterNet 3** — best self-hostable upgrade: Rust→WASM, CPU real-time, 48 kHz fullband. Self-host `df_bg.wasm` + DFN3 ONNX model; wire a 48 kHz worklet. Audio quality unverifiable without a real-call test.
|
||
- [ ] **Desktop-only / HW-gated:** FRCRN (Alibaba) or NVIDIA Maxine (RTX/Tensor only). Runs in Tauri Rust backend + bridges a virtual mic into the webview. Must detect capability; web + weak HW falls back to RNNoise/DTLN.
|
||
|
||
---
|
||
|
||
### P5-31 · Granular Voice & Screenshare Quality Controls
|
||
|
||
**Mechanism:** WebRTC Encoding Parameters + Backend Quality Guard.
|
||
|
||
- **State Event:** `io.lotus.room_quality` (state key `""`) containing:
|
||
```json
|
||
{ "audio_bitrate": 128000, "screen_max_res": "1080p", "screen_max_fps": 60 }
|
||
```
|
||
- **Screenshare:** In `src/app/plugins/call/CallControl.ts`, map the "Quality" setting to `getDisplayMedia` constraints.
|
||
- **Audio Bitrate:** After the call joins, find the `RTCRtpSender` for the audio track:
|
||
```typescript
|
||
const sender = peerConnection.getSenders().find((s) => s.track?.kind === 'audio');
|
||
const params = sender.getParameters();
|
||
params.encodings[0].maxBitrate = roomBitrate || 128000;
|
||
await sender.setParameters(params);
|
||
```
|
||
- **Backend Sidecar:** Extend `voice-limit-guard.py` (LXC 151) to fetch `io.lotus.room_quality` and inject limits into the LiveKit JWT or return them as an authorized config packet.
|
||
|
||
---
|
||
|
||
### P5-40 · Desktop — Proactive Update Notifications (Tauri)
|
||
|
||
**Key Files:** `src/app/hooks/useTauriUpdater.ts`, `src/app/pages/client/ClientNonUIFeatures.tsx`, `src/app/features/toast/LotusToastContainer.tsx`.
|
||
|
||
1. Create a `TauriUpdateFeature` component. Use `useTauriUpdater()` to get the `check` function and `status`.
|
||
2. In a `useEffect`, call `check()` on mount and then on a `setInterval` (every 12 hours).
|
||
3. When status transitions to `{ state: 'available', version: '...' }`, fire a Lotus Toast: "Lotus Chat v[version] is available!" with an "Update" button that calls `install()`.
|
||
4. Store `lastCheck` timestamp in `localStorage` to prevent redundant checks on refresh.
|
||
|
||
---
|
||
|
||
### Mobile Bookmarks Visibility Fix
|
||
|
||
**Issue:** `ClientLayout.tsx` explicitly restricts `BookmarksPanel` to `ScreenSize.Desktop` (lines 51-56).
|
||
|
||
```tsx
|
||
// ClientLayout.tsx
|
||
{
|
||
bookmarksOpen && (
|
||
<BookmarksPanel
|
||
onClose={() => setBookmarksOpen(false)}
|
||
isMobile={screenSize !== ScreenSize.Desktop}
|
||
/>
|
||
);
|
||
}
|
||
```
|
||
|
||
`BookmarksPanel.tsx` already supports the `isMobile` prop (line 127) to enable full-screen absolute positioning. No other changes required.
|
||
|
||
---
|
||
|
||
### Remind Me Later (Slack-style)
|
||
|
||
**Mechanism:** Account Data + Timer/Service Worker.
|
||
|
||
- **Storage (`src/app/hooks/useReminders.ts`):** Store in account data `io.lotus.reminders` as `Array<{ id: string, roomId: string, eventId: string, timestamp: number }>`.
|
||
- **Context Menu (`src/app/features/room/message/MessageContextMenu.tsx`):** Add "Remind me" option → opens date/time picker modal (reuse `JumpToTime.tsx` logic).
|
||
- **Trigger (foreground):** `setTimeout` in a hook inside `ReminderMonitor` in `ClientNonUIFeatures.tsx` → pushes to `toastQueueAtom` in `state/toast.ts` when due.
|
||
- **Trigger (background):** Use Service Worker — `setTimeout` in the main thread will not fire when the PWA is suspended.
|
||
|
||
---
|
||
|
||
### Mobile Usability Audit — Methodology
|
||
|
||
1. **Viewport & Touch:** All interactive elements must have at least `44px × 44px` touch targets. Audit for horizontal overflow (horizontal scrolling must be disabled).
|
||
2. **Modal Responsiveness:** All modals (Settings, Profile, etc.) MUST cover the full screen on mobile, not float as overlays.
|
||
3. **Sidebar / Panels:** On mobile, sidebar panels (Members, Bookmarks, Media) must become full-screen overlays (using a `Drawer` or `Modal` pattern) rather than side-by-side flexbox panels.
|
||
4. **Input & Composer:** Ensure the composer doesn't get obscured by the mobile keyboard. Test focus trap and blur behaviors.
|
||
|
||
---
|
||
|
||
## Implementation Notes
|
||
|
||
### ⚠️ TDS DESIGN LAW (repeated here for emphasis)
|
||
|
||
> Every TDS color, animation, glow, border, shadow, and font value MUST come from `/root/code/web_template/base.css`.
|
||
> Never hardcode hex values. Never invent CSS variable names.
|
||
> Key variables: `--lt-accent-orange` · `--lt-accent-cyan` · `--lt-accent-green` · `--lt-glow-*` · `--lt-box-glow-*` · `--lt-border-color` · `--lt-font-mono`
|
||
> Reference implementation: `/root/code/tinker_tickets/` (markdown.js, base.js, ticket.css)
|
||
> This applies without exception to every task marked `[IMPROVE]`, `[Build]`, or any UI change.
|
||
|
||
### Design Rules
|
||
|
||
- All new components must respect both TDS dark (`LotusTerminalTheme`) and TDS light (`LotusTerminalLightTheme`) modes
|
||
- Non-TDS theme work (custom accent color, theme presets) uses vanilla-extract theme files — match the pattern in `src/lotus-terminal.css.ts`
|
||
- Code syntax highlighting token classes: `.tok-kw .tok-str .tok-num .tok-cmt .tok-fn` (defined in `web_template/base.css`)
|
||
- `folds AvatarImage` does NOT accept children — wrap Avatar components externally for overlays/frames/borders
|
||
|
||
### CI/CD Pipeline
|
||
|
||
```
|
||
edit → commit → git push origin lotus
|
||
→ Gitea Actions: tsc --noEmit, eslint, prettier (~3 min)
|
||
→ Webhook: lotus_deploy.sh on LXC 106 polls CI, then npm ci && npm run build → rsync
|
||
→ Live at chat.lotusguild.org (~11 min total)
|
||
```
|
||
|
||
### Per-Feature Checklist (before marking complete)
|
||
|
||
- [ ] `npx tsc --noEmit` — zero TypeScript errors
|
||
- [ ] `npx eslint src/` — zero new errors (warnings OK if pre-existing)
|
||
- [ ] `npx prettier --check src/` — formatting passes
|
||
- [ ] `README.md` updated (Lotus-custom features only — not upstream Cinny features)
|
||
- [ ] `landing/index.html` updated if the feature appears in the comparison table
|
||
- [ ] Visually tested at `chat.lotusguild.org` after CI deploys
|
||
|
||
### Homeserver Access (for server audits)
|
||
|
||
- **Synapse (Matrix):** LXC 151 on `compute-storage-01` — `pct exec 151 -- bash`
|
||
- **Config:** `/etc/matrix-synapse/homeserver.yaml`
|
||
- **Version check:** `curl -s https://matrix.lotusguild.org/_matrix/client/versions`
|