From 39cfc23ebe138319f086dec5d9922b3d89435bc2 Mon Sep 17 00:00:00 2001 From: Jared Vititoe Date: Wed, 1 Jul 2026 21:19:02 -0400 Subject: [PATCH] =?UTF-8?q?docs:=20backlog=20housekeeping=20=E2=80=94=20st?= =?UTF-8?q?ale=20items=20closed,=20Thread=20Panel=20design=20captured?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TODO: P4-7 already-implemented [x]; P4-6 mozilla test enablement verified live; Audit-3 researched β†’ deferred tracking MSC4427 (banner_url proposal, unmerged); P3-8 Thread Panel now carries the complete SDK-evidence-backed build plan (threadSupport side effects, local-echo gap, receipt fix, 4-agent partition) β€” ready for its own session. BUGS: N127 removed, Big #5 (backgrounds/seasonal) done, CDN env-var closed (VITE_DECORATION_CDN exists), test count updated, KE section points at the new investigation kit. Co-Authored-By: Claude Opus 4.8 --- LOTUS_BUGS.md | 16 ++++++++------- LOTUS_TODO.md | 57 +++++++++++++++++++++++++++++---------------------- 2 files changed, 42 insertions(+), 31 deletions(-) diff --git a/LOTUS_BUGS.md b/LOTUS_BUGS.md index 603f15c02..9eeab1620 100644 --- a/LOTUS_BUGS.md +++ b/LOTUS_BUGS.md @@ -69,12 +69,14 @@ from testing: ## πŸ”΄ Open β€” Actionable -### Calls / Audio - -- ~~**N127 β€” ML denoise shim is never injected in `vite dev`.**~~ **RESOLVED (dissolved by the A7 denoise cutover).** `vite.config.js` no longer injects a getUserMedia shim at all β€” the forked Element Call runs ML denoise in-source as a LiveKit `TrackProcessor` (activated by `lotusDenoiseSource=1`), so there is no build-time injection that could be missing in dev. Nothing to fix. - ### 🧨 Encryption / E2EE β€” ⚠️ EXTREME COMPLEXITY Β· 🧠 PLANNING SESSION REQUIRED Β· πŸ‘€ SENIOR ENGINEER +> 🧰 **Investigation kit ready (2026-07):** [`LOTUS_E2EE_INVESTIGATION.md`](./LOTUS_E2EE_INVESTIGATION.md) +> has the per-KE capture runbook (console signatures, synapse-side queries, the +> KE-1β†’KE-2 causality decision tree, ranked remediations), and the client now +> ships a **Crypto Diagnostics** capture helper (Settings) β€” run it during the +> next affected call and download the report before starting any fix. + > **Observed live in prod 2026-06-30** on `chat.lotusguild.org` during a 2-person > **Element Call** (E2EE enabled). These span **client rust-crypto (via > `matrix-js-sdk@41.6.0-rc.0`) ↔ Synapse ↔ Element Call's MatrixRTC E2EE** and are @@ -144,10 +146,10 @@ retry … AbortError: Restart delayed event timed out before the HS responded`, ### Code Hygiene / DevEx -- **Automated test suite β€” 545 tests across 62 modules, a hard CI gate.** `npm test` runs Node's built-in runner via `tsx` (not vitest β€” Vite 8 is ahead of vitest's range) and **blocks the build job on failure**. Broad pure-logic coverage: utils (common, regex, sanitize/XSS, time, matrix, matrix-uia, mimeTypes, sort, accentColor, findAndReplace, AsyncSearch, ASCIILexicalTable, keyboard, room, matrix-crypto, featureCheck, syntaxHighlight, imageCompression, user-agent, callSounds), state (settings, sessions, recentSearches, upload, typingMembers, lists, room-list, toast, scheduledMessages, backupRestore, callEmbed/callPreferences, spaceRooms, …), plugins (matrix-to, call/utils, via-servers, bad-words, recent-emoji, custom-emoji, markdown block/inline/utils), OIDC (cs-api, useParsedLoginFlows, oidcState), lotus/avatarDecorations, message-search, search filters. Prevention work has caught + fixed **4 real bugs** (`findAndReplace` infinite-loop; `getSettings` crash-on-load when storage is blocked; `isMacOS` never matching modern Macs; `isMLDenoiseSupported` throwing `ReferenceError` instead of returning false on browsers lacking the `AudioWorkletNode` binding). **Next:** component/integration tests (the untestable-under-tsx DOM/React surface). +- **Automated test suite β€” 561+ tests across 65+ modules, a hard CI gate.** `npm test` runs Node's built-in runner via `tsx` (not vitest β€” Vite 8 is ahead of vitest's range) and **blocks the build job on failure**. Broad pure-logic coverage: utils (common, regex, sanitize/XSS, time, matrix, matrix-uia, mimeTypes, sort, accentColor, findAndReplace, AsyncSearch, ASCIILexicalTable, keyboard, room, matrix-crypto, featureCheck, syntaxHighlight, imageCompression, user-agent, callSounds), state (settings, sessions, recentSearches, upload, typingMembers, lists, room-list, toast, scheduledMessages, backupRestore, callEmbed/callPreferences, spaceRooms, …), plugins (matrix-to, call/utils, via-servers, bad-words, recent-emoji, custom-emoji, markdown block/inline/utils), OIDC (cs-api, useParsedLoginFlows, oidcState), lotus/avatarDecorations, message-search, search filters. Prevention work has caught + fixed **4 real bugs** (`findAndReplace` infinite-loop; `getSettings` crash-on-load when storage is blocked; `isMacOS` never matching modern Macs; `isMLDenoiseSupported` throwing `ReferenceError` instead of returning false on browsers lacking the `AudioWorkletNode` binding). **Next:** component/integration tests (the untestable-under-tsx DOM/React surface). - **Extensive `as any` casts** across `src/` β€” gradual typing cleanup. - **`types/matrix/` mirrors SDK types** instead of importing them β€” drift risk. -- **Hardcoded CDN URL** should move to an env var (the decoration CDN is now single-sourced in `avatarDecorations.ts`, but the literal is still in-repo). +- ~~**Hardcoded CDN URL** should move to an env var~~ β€” **done:** `avatarDecorations.ts` already honors a `VITE_DECORATION_CDN` env override (lines 14-16); the in-repo literal is only the default. Nothing left. - **`patch-folds.mjs` edits `node_modules` directly** β€” consider `patch-package`. - **Infra docs:** `contrib/nginx` lacks security headers (HSTS/CSP) + uses rewrites over `try_files`; `contrib/caddy` has a placeholder path. CI/CD (`prod-deploy.yml`): sequential deploy, aggressive 1-min Netlify timeout, `package-manager-cache: false`. - **README:** keep the fork-sync version + logo path current. (`CONTRIBUTING.md` is intentionally left as upstream Cinny's β€” not a Lotus concern.) @@ -156,4 +158,4 @@ retry … AbortError: Restart delayed event timed out before the HS responded`, ### Big Projects -- **#5 β€” Seasonal themes & chat-background redesign.** Current backgrounds are basic CSS; goal is high-fidelity, research-backed, GPU-accelerated designs (layered `oklch`, `backdrop-filter`, `contain:paint`) with WCAG-AA overlay contrast. Treat each as its own design sprint. +- ~~**#5 β€” Seasonal themes & chat-background redesign.**~~ **DONE (2026-06/07):** 11 seasonal/holiday overlays shipped and later toned down + given a settings preview grid; all 19 chat backgrounds redesigned (Carbon + Aurora kept per user preference), one design sprint each, GPU-friendly CSS with `prefers-reduced-motion` + pause toggle. Remaining polish rides normal bug flow, not a "big project." diff --git a/LOTUS_TODO.md b/LOTUS_TODO.md index f5d575094..00164f54c 100644 --- a/LOTUS_TODO.md +++ b/LOTUS_TODO.md @@ -164,7 +164,7 @@ Status: `[ ]` pending Β· `[~]` in progress Β· `[x]` completed ### [ ] P3-8 Β· Thread Panel (full side drawer) -**⚠️ LARGEST FEATURE β€” requires its own planning session before implementation.** +**⚠️ LARGEST FEATURE β€” 🟒 DESIGN COMPLETE (2026-07), READY FOR ITS OWN EXECUTION SESSION.** The full architecture (SDK-evidence-backed decisions, file inventory, 4-agent partition, risks, verification checklist) is in the Implementation Reference section below β€” no further planning needed, just a dedicated build session. **What:** A right-side drawer for threaded conversations. Currently "Reply in Thread" exists but there is no panel to read or write thread replies. Features: @@ -196,10 +196,10 @@ Features: ## Priority 4 β€” Specialized, high complexity, or low priority -### [ ] P4-7 Β· Virtualized Infinite Scroll for Search Results +### [x] P4-7 Β· Virtualized Infinite Scroll for Search Results β€” ALREADY IMPLEMENTED (found 2026-07) **What:** Replace the manual "load more" button with an automated, virtualized infinite scroll for search results. -**Approach:** Utilize `@tanstack/react-virtual` in `MessageSearch.tsx` to handle the `nextToken` automatically as the user scrolls. +**Status:** Done in a prior session β€” `MessageSearch.tsx` already uses `useVirtualizer` (~line 336) over the result groups AND auto-fetches the `nextToken` page when the last virtual item scrolls into view (~line 469) via `useInfiniteQuery`. Nothing left to build. ### [ ] P4-8 Β· Encrypted Message Search Indexing & Caching @@ -257,7 +257,7 @@ Features: - Account mgmt: `settings/account/OidcManageAccount.tsx`. - 13 unit tests (discovery/flow/session/cache/callback parsing). All gates green. **Awaiting verification (needs a real MSC3861 server β€” lotusguild is NOT one):** deploy + log into **mozilla.org** (requires adding mozilla to the deployed `config.json` homeserverList + its domains to the CSP `connect-src`/`img-src` β€” see below), OR run a local `matrix-authentication-service` + Synapse `msc3861` dev loop. - **To enable the mozilla.org test:** add to `matrix/cinny/config.json` homeserverList `"mozilla.org"`, and to the nginx CSP `connect-src`/`img-src`: `https://mozilla.org https://mozilla.modular.im https://chat.mozilla.org https://vector.im`. + **Mozilla.org test enablement: ALREADY DEPLOYED (verified 2026-07)** β€” `matrix/cinny/config.json` homeserverList includes `mozilla.org` and the nginx CSP `connect-src` includes the mozilla/modular/vector domains (`matrix/cinny/nginx.conf:42`). **Nothing blocks the test β€” just pick mozilla.org on the login screen and complete an OIDC login.** --- @@ -482,9 +482,9 @@ Check back after each Synapse upgrade β€” re-run `/matrix/client/versions` and ` ## Pending Audits -### [ ] Audit-3 Β· Profile banner image β€” Matrix protocol support +### [DEFERRED] Audit-3 Β· Profile banner image β€” Matrix protocol support β€” RESEARCHED (2026-07) -Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banner field. `uk.tcpip.msc4133.stable = true` on our server β€” check if a `banner_url` or similar field is defined. If no cross-client standard exists, do not implement. +**Finding:** [MSC4427 β€” Custom banners for user profiles](https://github.com/matrix-org/matrix-spec-proposals/pull/4427) defines a `banner_url` profile field on top of the MSC4133 extensible-profile system (which our server supports, `uk.tcpip.msc4133.stable = true`, and which became stable in Matrix v1.16). However MSC4427 is an **open proposal, not merged** β€” no cross-client standard yet, so per this item's own rule: do not implement. **Revisit when MSC4427 merges** (implementation would then be small: read/write the field via the MSC4133 profile API + render a banner in UserHero/profile popouts). --- @@ -492,26 +492,35 @@ Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banne Exhaustive, low-level implementation details for backlog items. Follow these patterns to ensure code is "Lotus-perfect" (idiomatic, performant, and TDS-compliant). -### P3-8 Β· Thread Panel (Full Side Drawer) +### P3-8 Β· Thread Panel (Full Side Drawer) β€” 🟒 FULL DESIGN (2026-07, ready to execute) -**Architecture:** Mirror the `MembersDrawer` pattern but with a specialized timeline. +**Decisions (each backed by SDK evidence in node_modules/matrix-js-sdk):** -- **State (`src/app/state/room/thread.ts`):** - ```typescript - export const activeThreadIdAtom = atom(null); - ``` -- **Layout (`src/app/features/room/Room.tsx`):** Insert `ThreadPanel` conditionally alongside `RoomTimeline`: - ```tsx - { - activeThreadId && ( - <> - - - - ); - } - ``` -- **Component (`src/app/features/room/thread/ThreadPanel.tsx`):** Use `room.getThread(threadId)` from the SDK. Render a `Header` with a "Close" button that sets `activeThreadIdAtom` to `null`. Reuse `RoomTimeline` but pass a filtered `EventTimelineSet`. Use `thread.timelineSet` directly for the most accurate thread view. +| Question | Decision | +|---|---| +| Thread rendering | **New lean `ThreadTimeline`** reusing `Message`, `useVirtualPaginator`, and RoomTimeline's exported timeline helpers (lines 156-227). Do NOT refactor 2214-line RoomTimeline (its ~35 hooks are hardwired to the room live timeline). | +| threadSupport | **Enable `threadSupport: true`** in `initMatrix.ts` (~line 39). ⚠️ Thread replies then LEAVE the main timeline (`room.js eventShouldLiveIn` β†’ `shouldLiveInRoom:false`), retroactively on reload β€” MUST ship the "N replies" summary chip in the same release. Roots stay in both timelines. | +| State | `roomIdToActiveThreadIdAtomFamily` (per-room, mirrors `roomIdToReplyDraftAtomFamily`) in new `state/room/thread.ts` + `getThreadDraftKey(roomId, threadRootId)` = `` `${roomId}::${threadRootId}` `` | +| Composer | **Reuse RoomInput**: add optional `threadRootId` prop; scope its 3 atom-family lookups by draftKey (isolates thread drafts from the main composer); pass `threadRootId ?? null` at all 7 `mx.sendMessage/sendEvent` call sites β€” the SDK's `addThreadRelationIfNeeded` then emits spec-correct `m.thread` relations incl. reply-in-thread. Separate `useEditor()` instance in the panel. Hide schedule + commands in thread mode v1. | +| Unreads | v1 = unread badge on the summary chip (`room.getThreadUnreadNotificationCount` β€” counts already synced independent of threadSupport) + `markThreadAsRead` threaded receipt when panel open at bottom. | +| Mobile | Pure CSS like `MembersDrawer.css.ts`: fixed width toRem(360) desktop, `position:fixed; inset:0` under 750px. | + +**Critical side-effect fixes (one-liners, land FIRST):** +1. `initMatrix.ts` β†’ `threadSupport: true`. +2. `utils/notifications.ts:24` β†’ `sendReadReceipt(latestEvent, type, /*unthreaded*/ true)` β€” otherwise markAsRead becomes `main`-scoped and room badges stick permanently unread (room unread total includes thread counts). + +**Known SDK traps (verified):** +- **Local echo gap:** chronological pending ordering means the thread timelineSet never receives pending events (`canContain` rejects; `room.getPendingEvents()` THROWS in this mode) β€” ThreadTimeline must render its own pending strip via `RoomEvent.LocalEchoUpdated` filtering on `threadRootId`, deduped against `thread.findEventById`. +- **Bootstrap:** `room.getThread(id) ?? room.createThread(id, room.findEventById(id), [], false)` β€” the SDK auto-fetches via `/relations` and inserts the root at top; gate rendering on `thread.initialEventsFetched`; decrypt with `decryptAllTimelineEvent` after init + each pagination. +- **Deep links:** `getEventTimeline(mainSet, threadEventId)` returns undefined for thread events β€” redirect jump-to-event to the panel (best-effort v1). +- **Summary chip** must render from the server-aggregated bundle (`unsigned['m.relations']['m.thread']`) so it works before any Thread object exists. +- Room-list "latest message" preview may show the root, not the newest reply β€” cosmetic, accept v1. + +**File inventory β€” new:** `state/room/thread.ts` (+test), `features/room/thread/{useThread.ts, threadSummary.ts(+test), ThreadTimeline.tsx(+css), ThreadPanel.tsx(+css), ThreadSummary.tsx, index.ts}`, `hooks/useThreadSummary.ts`. **Edited:** `initMatrix.ts` + `utils/notifications.ts` (coordinator, step 0), `RoomInput.tsx` (threadRootId prop), `RoomTimeline.tsx` (handleReplyClick startThread β†’ open panel; ThreadSummary chips at the two Message call sites; Reply onThreadClick; deep-link redirect), `components/message/Reply.tsx`, `Room.tsx` (render panel after MediaGallery block, gated `!callView && activeThreadId`, `key={roomId+threadId}`). + +**4-agent partition:** step 0 (coordinator one-liners) β†’ A: state+SDK glue (+tests) Β· B: ThreadTimeline (largest; copies the `useTimelinePagination` pattern rather than exporting it) Β· C: RoomInput changes Β· D: panel shell + RoomTimeline/Reply integration β€” all parallel against pinned interface contracts β†’ coordinator wires Room.tsx + gates. + +**Verification:** gates (tsc/eslint/build/tests) + post-merge manual QA: open thread via chip/menu/indicator; pendingβ†’confirmed echo; `is_falling_back:false` on reply-in-thread; main timeline shows root+chip only; badge clears; reload keeps partitioning; encrypted threads decrypt. **Release note required:** threaded replies no longer render inline in the main timeline. ---