Compare commits
7 Commits
258e3ec620
..
lotus
| Author | SHA1 | Date | |
|---|---|---|---|
 jared
|
39cfc23ebe | ||
|
jared
|
7a8cadc6ec | ||
|
jared
|
91bd360125 | ||
|
jared
|
7da960ac8c | ||
|
jared
|
ed51c39fe7 | ||
|
jared
|
c1efa7b94e | ||
|
jared
|
e31b84c08e |
+9
-7
@@ -69,12 +69,14 @@ from testing:
|
||||
|
||||
## 🔴 Open — Actionable
|
||||
|
||||
### Calls / Audio
|
||||
|
||||
- ~~**N127 — ML denoise shim is never injected in `vite dev`.**~~ **RESOLVED (dissolved by the A7 denoise cutover).** `vite.config.js` no longer injects a getUserMedia shim at all — the forked Element Call runs ML denoise in-source as a LiveKit `TrackProcessor` (activated by `lotusDenoiseSource=1`), so there is no build-time injection that could be missing in dev. Nothing to fix.
|
||||
|
||||
### 🧨 Encryption / E2EE — ⚠️ EXTREME COMPLEXITY · 🧠 PLANNING SESSION REQUIRED · 👤 SENIOR ENGINEER
|
||||
|
||||
> 🧰 **Investigation kit ready (2026-07):** [`LOTUS_E2EE_INVESTIGATION.md`](./LOTUS_E2EE_INVESTIGATION.md)
|
||||
> has the per-KE capture runbook (console signatures, synapse-side queries, the
|
||||
> KE-1→KE-2 causality decision tree, ranked remediations), and the client now
|
||||
> ships a **Crypto Diagnostics** capture helper (Settings) — run it during the
|
||||
> next affected call and download the report before starting any fix.
|
||||
|
||||
> **Observed live in prod 2026-06-30** on `chat.lotusguild.org` during a 2-person
|
||||
> **Element Call** (E2EE enabled). These span **client rust-crypto (via
|
||||
> `matrix-js-sdk@41.6.0-rc.0`) ↔ Synapse ↔ Element Call's MatrixRTC E2EE** and are
|
||||
@@ -144,10 +146,10 @@ retry … AbortError: Restart delayed event timed out before the HS responded`,
|
||||
|
||||
### Code Hygiene / DevEx
|
||||
|
||||
- **Automated test suite — 545 tests across 62 modules, a hard CI gate.** `npm test` runs Node's built-in runner via `tsx` (not vitest — Vite 8 is ahead of vitest's range) and **blocks the build job on failure**. Broad pure-logic coverage: utils (common, regex, sanitize/XSS, time, matrix, matrix-uia, mimeTypes, sort, accentColor, findAndReplace, AsyncSearch, ASCIILexicalTable, keyboard, room, matrix-crypto, featureCheck, syntaxHighlight, imageCompression, user-agent, callSounds), state (settings, sessions, recentSearches, upload, typingMembers, lists, room-list, toast, scheduledMessages, backupRestore, callEmbed/callPreferences, spaceRooms, …), plugins (matrix-to, call/utils, via-servers, bad-words, recent-emoji, custom-emoji, markdown block/inline/utils), OIDC (cs-api, useParsedLoginFlows, oidcState), lotus/avatarDecorations, message-search, search filters. Prevention work has caught + fixed **4 real bugs** (`findAndReplace` infinite-loop; `getSettings` crash-on-load when storage is blocked; `isMacOS` never matching modern Macs; `isMLDenoiseSupported` throwing `ReferenceError` instead of returning false on browsers lacking the `AudioWorkletNode` binding). **Next:** component/integration tests (the untestable-under-tsx DOM/React surface).
|
||||
- **Automated test suite — 561+ tests across 65+ modules, a hard CI gate.** `npm test` runs Node's built-in runner via `tsx` (not vitest — Vite 8 is ahead of vitest's range) and **blocks the build job on failure**. Broad pure-logic coverage: utils (common, regex, sanitize/XSS, time, matrix, matrix-uia, mimeTypes, sort, accentColor, findAndReplace, AsyncSearch, ASCIILexicalTable, keyboard, room, matrix-crypto, featureCheck, syntaxHighlight, imageCompression, user-agent, callSounds), state (settings, sessions, recentSearches, upload, typingMembers, lists, room-list, toast, scheduledMessages, backupRestore, callEmbed/callPreferences, spaceRooms, …), plugins (matrix-to, call/utils, via-servers, bad-words, recent-emoji, custom-emoji, markdown block/inline/utils), OIDC (cs-api, useParsedLoginFlows, oidcState), lotus/avatarDecorations, message-search, search filters. Prevention work has caught + fixed **4 real bugs** (`findAndReplace` infinite-loop; `getSettings` crash-on-load when storage is blocked; `isMacOS` never matching modern Macs; `isMLDenoiseSupported` throwing `ReferenceError` instead of returning false on browsers lacking the `AudioWorkletNode` binding). **Next:** component/integration tests (the untestable-under-tsx DOM/React surface).
|
||||
- **Extensive `as any` casts** across `src/` — gradual typing cleanup.
|
||||
- **`types/matrix/` mirrors SDK types** instead of importing them — drift risk.
|
||||
- **Hardcoded CDN URL** should move to an env var (the decoration CDN is now single-sourced in `avatarDecorations.ts`, but the literal is still in-repo).
|
||||
- ~~**Hardcoded CDN URL** should move to an env var~~ — **done:** `avatarDecorations.ts` already honors a `VITE_DECORATION_CDN` env override (lines 14-16); the in-repo literal is only the default. Nothing left.
|
||||
- **`patch-folds.mjs` edits `node_modules` directly** — consider `patch-package`.
|
||||
- **Infra docs:** `contrib/nginx` lacks security headers (HSTS/CSP) + uses rewrites over `try_files`; `contrib/caddy` has a placeholder path. CI/CD (`prod-deploy.yml`): sequential deploy, aggressive 1-min Netlify timeout, `package-manager-cache: false`.
|
||||
- **README:** keep the fork-sync version + logo path current. (`CONTRIBUTING.md` is intentionally left as upstream Cinny's — not a Lotus concern.)
|
||||
@@ -156,4 +158,4 @@ retry … AbortError: Restart delayed event timed out before the HS responded`,
|
||||
|
||||
### Big Projects
|
||||
|
||||
- **#5 — Seasonal themes & chat-background redesign.** Current backgrounds are basic CSS; goal is high-fidelity, research-backed, GPU-accelerated designs (layered `oklch`, `backdrop-filter`, `contain:paint`) with WCAG-AA overlay contrast. Treat each as its own design sprint.
|
||||
- ~~**#5 — Seasonal themes & chat-background redesign.**~~ **DONE (2026-06/07):** 11 seasonal/holiday overlays shipped and later toned down + given a settings preview grid; all 19 chat backgrounds redesigned (Carbon + Aurora kept per user preference), one design sprint each, GPU-friendly CSS with `prefers-reduced-motion` + pause toggle. Remaining polish rides normal bug flow, not a "big project."
|
||||
|
||||
@@ -0,0 +1,402 @@
|
||||
# Lotus Chat — E2EE Investigation Runbook (KE-1 → KE-4)
|
||||
|
||||
> **Scope:** evidence-gathering only. Do **not** apply fixes from this document
|
||||
> without a cross-system planning session (client rust-crypto ↔ Synapse ↔
|
||||
> Element Call MatrixRTC). Symptom source: `LOTUS_BUGS.md` §"Encryption / E2EE"
|
||||
> (KE-1..KE-4), observed live 2026-06-30 on `chat.lotusguild.org` during a
|
||||
> 2-person Element Call.
|
||||
>
|
||||
> **Client:** Lotus Cinny fork, `matrix-js-sdk@41.6.0-rc.0`, rust-crypto.
|
||||
> **Server:** Synapse `1.155.0` on **LXC 151** (`10.10.10.29`), PostgreSQL 17.9
|
||||
> on **LXC 109** (`10.10.10.44`). Facts below are copy-pasteable against that
|
||||
> deployment (paths/IPs from `/root/code/matrix/README.md`).
|
||||
|
||||
---
|
||||
|
||||
## 0. Deployment facts used by this runbook
|
||||
|
||||
From the matrix infra README (`/root/code/matrix/README.md`):
|
||||
|
||||
| Thing | Value |
|
||||
|-------|-------|
|
||||
| Synapse host | LXC **151**, `10.10.10.29` (Synapse 1.155.0) |
|
||||
| Synapse log | `/var/log/matrix-synapse/homeserver.log` |
|
||||
| Synapse config | `/etc/matrix-synapse/homeserver.yaml` (+ `conf.d/`) |
|
||||
| Synapse HTTP | `10.10.10.29:8008` |
|
||||
| PostgreSQL host | LXC **109**, `10.10.10.44` (PG 17.9), db `synapse` |
|
||||
| synapse-admin UI | `http://10.10.10.29:8080` |
|
||||
| LiveKit / lk-jwt / guard | LXC 151: LiveKit `:7880/:7881`, guard `:8070`, lk-jwt `:8071` |
|
||||
| SSH path to Synapse | `ssh root@10.10.10.4` then `pct enter 151` |
|
||||
| SSH path to PG | `ssh root@10.10.10.4` then `pct enter 109` |
|
||||
|
||||
**Getting a psql shell** (run on LXC 109, or from 151 over the network):
|
||||
|
||||
```bash
|
||||
# On LXC 109:
|
||||
sudo -u postgres psql synapse
|
||||
# From LXC 151 (pg_hba allows 10.10.10.29):
|
||||
psql "host=10.10.10.44 user=synapse dbname=synapse"
|
||||
```
|
||||
|
||||
**Tailing Synapse during a call** (on LXC 151):
|
||||
|
||||
```bash
|
||||
tail -F /var/log/matrix-synapse/homeserver.log | tee /tmp/lotus-call-$(date +%s).log
|
||||
```
|
||||
|
||||
Synapse E2EE/to-device logging is chatty at `INFO`; if a category is silent,
|
||||
temporarily raise it in `/etc/matrix-synapse/conf.d/log.yaml` (or the
|
||||
`log_config` file referenced by `homeserver.yaml`):
|
||||
|
||||
```yaml
|
||||
loggers:
|
||||
synapse.rest.client.keys: { level: DEBUG }
|
||||
synapse.handlers.e2e_keys: { level: DEBUG }
|
||||
synapse.storage.databases.main.end_to_end_keys: { level: DEBUG }
|
||||
synapse.handlers.devicemessage: { level: DEBUG } # to-device
|
||||
```
|
||||
|
||||
Then `systemctl reload matrix-synapse` (reload re-reads log config without a
|
||||
full restart). **Revert to `INFO` after the capture** — DEBUG is very verbose.
|
||||
|
||||
---
|
||||
|
||||
## 1. Per-KE evidence matrix
|
||||
|
||||
Client greps assume Chrome/Firefox DevTools console (filter box or, better,
|
||||
"Preserve log" + save-as). The **Crypto Diagnostics** card (Settings →
|
||||
Developer Tools) auto-captures every signature below into a downloadable JSON —
|
||||
use it as the primary client artifact and DevTools as the raw backup.
|
||||
|
||||
### KE-1 — OTK upload conflict storm (root-cause candidate)
|
||||
|
||||
- **Console signature (grep):**
|
||||
- `already exists`
|
||||
- full: `POST /_matrix/client/v3/keys/upload … 400 M_UNKNOWN: One time key signed_curve25519:<id> already exists. Old key: {…} new key: {…}`
|
||||
- **Capture client-side:**
|
||||
- Timestamp (first occurrence + rate — "N/sec"), **device id**, **user id**.
|
||||
- DevTools → **Network** → filter `keys/upload`: for a failing call save the
|
||||
**request body** (the `one_time_keys` map — note the exact `signed_curve25519:<id>`)
|
||||
and the **response body** (the `Old key` / `new key` JSON). This diff is the
|
||||
smoking gun: same key-id, different value ⇒ store vs server divergence.
|
||||
- Whether it self-heals or loops forever (KE-1 loops).
|
||||
- **Synapse log grep (LXC 151):**
|
||||
```bash
|
||||
grep -E "keys/upload|One time key .* already exists|OneTimeKey" \
|
||||
/var/log/matrix-synapse/homeserver.log | grep "<user_id>"
|
||||
```
|
||||
- **Synapse SQL (LXC 109) — what the server thinks it holds:**
|
||||
```sql
|
||||
-- Current OTK inventory for the device (compare key_id set against the
|
||||
-- request body the client keeps retrying).
|
||||
SELECT algorithm, key_id, ts_added_ms
|
||||
FROM e2e_one_time_keys_json
|
||||
WHERE user_id = '@user:matrix.lotusguild.org'
|
||||
AND device_id = '<DEVICE_ID>'
|
||||
ORDER BY algorithm, key_id;
|
||||
|
||||
-- Server's advertised counts (this is what /sync tells the client it has,
|
||||
-- and drives whether the client decides to upload more).
|
||||
SELECT algorithm, count(*) FROM e2e_one_time_keys_json
|
||||
WHERE user_id = '@user:matrix.lotusguild.org' AND device_id = '<DEVICE_ID>'
|
||||
GROUP BY algorithm;
|
||||
|
||||
-- Fallback key state (used when OTKs are exhausted).
|
||||
SELECT algorithm, key_id, used, ts_added_ms
|
||||
FROM e2e_fallback_keys_json
|
||||
WHERE user_id = '@user:matrix.lotusguild.org' AND device_id = '<DEVICE_ID>';
|
||||
```
|
||||
> Table names are Synapse 1.155 (`e2e_one_time_keys_json`,
|
||||
> `e2e_fallback_keys_json`). If a name is absent, list with `\dt e2e*` in psql.
|
||||
- **Confirms:** if the offending `key_id` (from the 400) is **present** in
|
||||
`e2e_one_time_keys_json` with a **different** stored value than the client's
|
||||
request body → OTK state has diverged (rust-crypto store vs Synapse). That is
|
||||
the KE-1 root condition.
|
||||
|
||||
### KE-2 — EC media keys not arriving/decrypting (audio/video cutouts)
|
||||
|
||||
- **Console signature (grep):**
|
||||
- `MissingKey`
|
||||
- `missing key at index` (e.g. `MissingKey: missing key at index N for participant @user`)
|
||||
- `key set not found`
|
||||
- `io.element.call.encryption_keys` (rust-crypto: `WARN … Received an unexpected encrypted to-device event … event_type="io.element.call.encryption_keys"`)
|
||||
- **Capture client-side:**
|
||||
- Timestamp windows where a participant's audio/video cut out, and the
|
||||
`@participant` + `index N` from the message.
|
||||
- The `io.element.call.encryption_keys` warnings (these are the media-key
|
||||
to-device events failing to decrypt) with their timestamps.
|
||||
- Own device id + user id (to correlate with the sender's Olm session).
|
||||
- **Synapse log grep (LXC 151) — to-device delivery of the media keys:**
|
||||
```bash
|
||||
grep -E "io.element.call.encryption_keys|m.room.encrypted|/sendToDevice|to_device" \
|
||||
/var/log/matrix-synapse/homeserver.log | grep -E "<user_id>|<participant_id>"
|
||||
```
|
||||
- **Synapse SQL (LXC 109) — undelivered / queued to-device events:**
|
||||
```sql
|
||||
-- Backlog of to-device messages queued for the affected device. A growing
|
||||
-- count here = the HS has the media-key events but the device isn't draining
|
||||
-- them via /sync (or they were sent to a stale device id).
|
||||
SELECT user_id, device_id, count(*) AS pending
|
||||
FROM device_inbox
|
||||
WHERE user_id = '@user:matrix.lotusguild.org'
|
||||
GROUP BY user_id, device_id;
|
||||
|
||||
-- Cross-check the device id the sender is targeting actually exists / is current.
|
||||
SELECT device_id, display_name, last_seen, ts
|
||||
FROM devices WHERE user_id = '@user:matrix.lotusguild.org';
|
||||
```
|
||||
- **Confirms:** to-device events present but undecryptable (client shows the
|
||||
`io.element.call.encryption_keys` "unexpected encrypted" warning) ⇒ there is
|
||||
**no valid Olm session** to decrypt them — the expected downstream of KE-1.
|
||||
|
||||
### KE-3 — Timeline decryption error: missing `algorithm` field
|
||||
|
||||
- **Console signature (grep):**
|
||||
- `DecryptionError`
|
||||
- full: `Error decrypting event (… type=m.room.encrypted …): DecryptionError[msg: missing field 'algorithm' at line 1 column 138 …]`
|
||||
- **Capture client-side:**
|
||||
- The **event id** (`$SASBBzoqj…` was one) and the **room id**.
|
||||
- Pull the raw event JSON via DevTools or the Developer Tools account-data/event
|
||||
viewer, or directly:
|
||||
```
|
||||
GET https://matrix.lotusguild.org/_matrix/client/v3/rooms/<roomId>/event/<eventId>
|
||||
```
|
||||
Inspect `content` — confirm whether `algorithm` (should be
|
||||
`m.megolm.v1.aes-sha2`) is truly absent vs a serialization mismatch.
|
||||
- **Synapse log grep (LXC 151):**
|
||||
```bash
|
||||
grep -E "<eventId>" /var/log/matrix-synapse/homeserver.log
|
||||
```
|
||||
- **Synapse SQL (LXC 109) — the stored event content as the HS holds it:**
|
||||
```sql
|
||||
SELECT ej.event_id, e.type, e.sender, e.origin_server_ts,
|
||||
(ej.json::json -> 'content' -> 'algorithm') AS algorithm
|
||||
FROM event_json ej
|
||||
JOIN events e USING (event_id)
|
||||
WHERE ej.event_id = '$SASBBzoqj...';
|
||||
```
|
||||
- **Confirms:** if the stored `content.algorithm` is **NULL/absent** on the HS →
|
||||
a malformed/legacy event was persisted (sender-side or federation). If it is
|
||||
**present** on the HS but the client throws → an RC-SDK deserialization bug.
|
||||
This distinction decides whether KE-3 is a data problem or a client problem.
|
||||
|
||||
### KE-4 — MatrixRTC delayed-event / membership timeouts
|
||||
|
||||
- **Console signature (grep):**
|
||||
- `update_delayed_event` (`org.matrix.msc4157.update_delayed_event`)
|
||||
- `delayed event` / `Restart delayed event timed out`
|
||||
- full: `[MembershipManager] Network local timeout error while sending event, immediate retry … AbortError: Restart delayed event timed out before the HS responded`
|
||||
- **Capture client-side:**
|
||||
- Timestamps of each timeout; whether they correlate with call join/leave or
|
||||
with general sync slowness.
|
||||
- DevTools → Network: the `…/delayed_events…` / `update_delayed_event`
|
||||
requests — their **HTTP status and latency** (timed-out vs slow-200).
|
||||
- **Synapse log grep (LXC 151):**
|
||||
```bash
|
||||
grep -E "delayed_event|msc4140|msc4157|update_delayed" \
|
||||
/var/log/matrix-synapse/homeserver.log | grep "<user_id>"
|
||||
# HS responsiveness in the same window (KE-4 may be pure latency):
|
||||
grep -E "Processed request|/sync" /var/log/matrix-synapse/homeserver.log | tail -50
|
||||
```
|
||||
- **Server-side corroboration (Grafana, `dashboard.lotusguild.org`):** Synapse
|
||||
p99 response time (excl. `/sync`), event-processing lag, DB query latency for
|
||||
the call window. High latency here ⇒ KE-4 is (partly) homeserver
|
||||
responsiveness, not a client bug.
|
||||
- **Confirms:** timeouts that line up with HS latency spikes → reliability/load;
|
||||
timeouts with a healthy HS → client MembershipManager retry logic.
|
||||
|
||||
---
|
||||
|
||||
## 2. Causality hypothesis
|
||||
|
||||
```
|
||||
KE-1 OTK upload conflict storm
|
||||
(rust-crypto store ↔ Synapse OTK state DIVERGED; server rejects re-uploads)
|
||||
│ no fresh OTKs can be published/claimed
|
||||
▼
|
||||
No new Olm (1:1) sessions can be established with this device
|
||||
│
|
||||
▼
|
||||
KE-2 EC media-key to-device events (io.element.call.encryption_keys)
|
||||
arrive but cannot be decrypted ⇒ MissingKey at index N
|
||||
⇒ friend's audio/video cuts out
|
||||
```
|
||||
|
||||
KE-3 (missing `algorithm`) and KE-4 (delayed-event timeouts) are **likely
|
||||
independent** of the KE-1→KE-2 chain: KE-3 is a decode/serialization path,
|
||||
KE-4 is a MatrixRTC-vs-HS reliability path. Confirm/refute independence with the
|
||||
decision tree below.
|
||||
|
||||
### Decision tree — which capture confirms/refutes each link
|
||||
|
||||
```
|
||||
Q1. Does the KE-1 offending key_id from the 400 response exist in
|
||||
e2e_one_time_keys_json with a DIFFERENT value than the client request body?
|
||||
├─ YES → OTK divergence CONFIRMED (KE-1 root). Go to Q2.
|
||||
└─ NO → Not divergence. Check: are OTK counts at 0 with fallback key `used=true`?
|
||||
├─ YES → OTK exhaustion, not divergence — different remediation.
|
||||
└─ NO → Suspect RC-SDK 41.6.0-rc.0 upload-loop regression (see §3).
|
||||
|
||||
Q2. During the same call, are io.element.call.encryption_keys to-device events
|
||||
present in device_inbox / Synapse to-device logs for our device id?
|
||||
├─ YES + client shows "unexpected encrypted"/MissingKey
|
||||
│ → KE-1 ⇒ KE-2 LINK CONFIRMED (events delivered, no Olm session to open them).
|
||||
├─ YES + client decrypts fine, but LiveKit still silent
|
||||
│ → KE-2 is downstream of LiveKit/SFU, NOT KE-1. Decouple from crypto.
|
||||
└─ NO (nothing queued/targeted our device)
|
||||
→ media keys never sent to us: stale device id / membership (see KE-4)
|
||||
→ KE-2 is a device-targeting problem, weakly linked to KE-1.
|
||||
|
||||
Q3. KE-3: is content.algorithm NULL in event_json on the HS?
|
||||
├─ YES → malformed persisted event (sender/federation). Independent of KE-1.
|
||||
└─ NO → client-side RC-SDK deserialization bug. Independent of KE-1.
|
||||
|
||||
Q4. KE-4: do delayed-event timeouts coincide with Synapse p99 latency spikes
|
||||
(Grafana) in the same minute?
|
||||
├─ YES → homeserver responsiveness/load. Independent of KE-1..KE-3.
|
||||
└─ NO → client MembershipManager retry behavior. Independent.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Ranked remediation options (with blast radius)
|
||||
|
||||
> Ordered least-destructive → most-destructive. **Do not run any of these as a
|
||||
> "fix" before the planning session** — they are listed so evidence collection
|
||||
> can be paired with a recovery plan. Confirm the root condition (Q1/Q2) first.
|
||||
|
||||
1. **Per-device logout + re-login of the affected device** *(lowest blast radius)*
|
||||
- **What:** log the one glitching device out and back in. Forces a fresh
|
||||
device id, fresh device keys, and a clean OTK batch — sidesteps a diverged
|
||||
OTK store without touching other sessions.
|
||||
- **Blast radius:** that device only. Other sessions/devices untouched.
|
||||
- **Cost:** the new device must be re-verified (cross-signing) and will need
|
||||
to restore room keys from **key backup** to read old encrypted history.
|
||||
- **Confirms/uses:** if KE-1 stops after this, OTK-store divergence (Q1) was
|
||||
the cause.
|
||||
|
||||
2. **Client crypto-store reset (`clearLoginData` path)** *(medium)*
|
||||
- **What:** `clearLoginData()` in `src/client/initMatrix.ts` (coordinator's
|
||||
file — do not edit) **deletes ALL IndexedDB databases** (incl.
|
||||
`web-sync-store` and the rust-crypto store `crypto-store`), **unregisters
|
||||
service workers**, **clears all Cache Storage**, and **`localStorage.clear()`**,
|
||||
then reloads. `clearCacheAndReload()` is lighter — it only calls
|
||||
`mx.store.deleteAllData()` (sync cache) and does **not** wipe crypto.
|
||||
- **Blast radius:** this browser profile only, but total: you are logged out,
|
||||
lose all cached sync state, drafts, settings, and **the local
|
||||
megolm/room-key store**.
|
||||
- **⚠️ Message-history / backup implication:** wiping `crypto-store` destroys
|
||||
locally-held **room keys (megolm inbound sessions)**. Any history **not
|
||||
backed up to server-side Key Backup** becomes **permanently undecryptable
|
||||
on this device**. Before doing this: verify Key Backup is enabled and the
|
||||
recovery key / passphrase is available (Settings → Security), or the user
|
||||
loses readable history. Cross-signing must be re-established too.
|
||||
- **Use when:** the rust-crypto store itself is corrupt/diverged and option 1
|
||||
didn't clear it.
|
||||
|
||||
3. **SDK pin change off the RC** *(medium — codebase change, needs rebuild)*
|
||||
- **Current pin:** `package.json` → `"matrix-js-sdk": "41.6.0-rc.0"` (a
|
||||
release candidate).
|
||||
- **Finding (npm / GitHub changelog, checked 2026-07):** stable **`41.6.0`**
|
||||
was released **2026-05-26**. Its only changelog line is *"Throw sane error
|
||||
on completeLoginOnNewDevice IdP rejection"* — **no OTK / keys-upload / Olm /
|
||||
to-device fix** relative to the RC. Later stable lines exist
|
||||
(`41.7.0`, `41.8.0`; `41.7.0-rc.3` / `41.9.0-rc.0` seen as pre-releases).
|
||||
Nearby crypto-relevant entries: `41.5.0` *"Enable encrypted history sharing
|
||||
by default"*; `41.4.0` key-backup handling. **No changelog entry directly
|
||||
addresses the KE-1 OTK-conflict symptom** in the immediate range — so
|
||||
moving RC→`41.6.0` stable is a low-risk hygiene step but is **not expected
|
||||
to fix KE-1 by itself**. Before pinning, re-read the CHANGELOG for any
|
||||
`41.7.x`/`41.8.x` OTK/one-time-key/olm entry that post-dates this note.
|
||||
- **Blast radius:** all users after the next `cinny-build.sh` deploy. Test the
|
||||
rust-crypto IndexedDB schema — a downgrade triggers the `IDB_VERSION_CONFLICT`
|
||||
path in `initMatrix.ts`.
|
||||
|
||||
4. **Synapse-side OTK row surgery** *(LAST RESORT — highest danger)*
|
||||
- **What:** deleting/rewriting rows in `e2e_one_time_keys_json` (and/or
|
||||
`e2e_fallback_keys_json`, `device_inbox`) for the affected device to force
|
||||
the client to re-upload a clean batch.
|
||||
- **⚠️ Danger:** direct writes to Synapse crypto tables can **desync every
|
||||
device of that user**, break Olm sessions **for everyone who has claimed one
|
||||
of those keys**, and are easy to get wrong (wrong `key_id`, cache not
|
||||
invalidated). Synapse caches OTK counts — a raw DELETE without a restart can
|
||||
leave the advertised count wrong, **worsening** the KE-1 loop.
|
||||
- **Guardrails if ever done (planning session + HS owner only):** full
|
||||
`pg_dump` of `synapse` first; do it during **zero active calls**; delete only
|
||||
the exact diverged `key_id` for the exact `device_id`; `systemctl restart
|
||||
matrix-synapse` to flush caches; then log the device out/in (option 1) so it
|
||||
republishes. **Never** run this speculatively.
|
||||
|
||||
---
|
||||
|
||||
## 4. "Capture session" checklist (run during the next call)
|
||||
|
||||
Do these **in order**. Aim to have client + server capturing the **same call**.
|
||||
|
||||
1. **Prep server tail (LXC 151):** SSH in, start
|
||||
`tail -F /var/log/matrix-synapse/homeserver.log | tee /tmp/lotus-call-$(date +%s).log`.
|
||||
(Optionally raise the `synapse.rest.client.keys` / `handlers.e2e_keys` /
|
||||
`handlers.devicemessage` loggers to DEBUG per §0 and `systemctl reload
|
||||
matrix-synapse` — remember to revert after.)
|
||||
2. **Prep client:** open Lotus Chat → Settings → Developer Tools → **enable
|
||||
Developer Tools** so the **Crypto Diagnostics** card is visible; note its
|
||||
entry count starts at (or reset by reload to) 0.
|
||||
3. **Open DevTools** (F12) → Console: enable **Preserve log**; Network tab:
|
||||
enable **Preserve log** + **Record**. Note your **device id** and **user id**
|
||||
(Settings → Devices / Developer Tools → Copy access token page shows ids).
|
||||
4. **Note wall-clock start time** (ISO/UTC) on both machines so logs align.
|
||||
5. **Join the Element Call** with the second participant; reproduce the fault
|
||||
(wait for the audio/video cutouts and let KE-1 storm run ~30–60s).
|
||||
6. **When a fault occurs, note the wall-clock timestamp** and which symptom
|
||||
(audio cut / video freeze / etc.) — this bounds the log window.
|
||||
7. **Client artifacts:** in the Crypto Diagnostics card click **Download report**
|
||||
(`lotus-crypto-diag-<ts>.json`); in DevTools Network, save the failing
|
||||
`keys/upload` request+response (right-click → Save/Copy), and the raw HAR
|
||||
(Network → Save all as HAR) for the call window.
|
||||
8. **Grab KE-3 event id / KE-2 participant+index** from the console (or the
|
||||
diag JSON `entries[]`) for the SQL lookups.
|
||||
9. **Server artifacts:** stop the tail; run the per-KE greps and SQL from §1
|
||||
against the noted device id / user id / event id, saving output alongside the
|
||||
client JSON. Screenshot the Grafana Synapse latency panels for the window
|
||||
(for KE-4).
|
||||
10. **Bundle & label:** put client JSON + HAR + server log slice + SQL output in
|
||||
one folder named with the call's UTC start time. Revert any DEBUG log config
|
||||
(`systemctl reload matrix-synapse`). Hand off to the planning session — **do
|
||||
not apply §3 remediations yet.**
|
||||
|
||||
---
|
||||
|
||||
## 5. Client diagnostics helper (this kit)
|
||||
|
||||
- **`src/app/utils/cryptoDiagLog.ts`** — capture-only console instrumentation.
|
||||
- `installCryptoDiagLog()` — idempotent; wraps `console.warn`/`console.error`
|
||||
with pass-through wrappers (originals always called) that ring-buffer (max
|
||||
**200**) any line matching the KE signatures. No network, no timers.
|
||||
- `getCryptoDiagEntries()` — snapshot copy of the buffer (`{ ts, level, ke,
|
||||
signature, message }`, most-recent-last).
|
||||
- `buildCryptoDiagReport(mx)` — JSON string: SDK version, device id, user id,
|
||||
sync state, `cryptoReady` (`mx.getCrypto()` presence), per-KE counts, and the
|
||||
entry buffer. No tokens/PII beyond those ids; captured log lines are retained
|
||||
verbatim as evidence.
|
||||
- **Signatures → KE mapping:** `already exists`→KE-1; `missing key at index` /
|
||||
`io.element.call.encryption_keys` / `MissingKey`→KE-2; `DecryptionError`→KE-3;
|
||||
`update_delayed_event` / `delayed event`→KE-4.
|
||||
- **`src/app/features/settings/developer/CryptoDiagnostics.tsx`** — a folds
|
||||
`SequenceCard`/`SettingTile` card (mirrors `developer-tools/DevelopTools.tsx`)
|
||||
showing the live matched-entry count (Badge) and a **Download report** button
|
||||
(Blob → `lotus-crypto-diag-<ts>.json`, same download idiom as
|
||||
`room-settings/ExportRoomHistory.tsx`).
|
||||
|
||||
### Recommended mount points (coordinator)
|
||||
|
||||
- **Install call:** call `installCryptoDiagLog()` **as early as possible during
|
||||
boot** so it captures crypto errors from first sync — ideally at the top of
|
||||
the client entry module or inside `ClientRoot` before/around `initClient`
|
||||
(e.g. `src/app/pages/client/ClientRoot.tsx`). It is idempotent, side-effect
|
||||
only, and needs no `mx`, so a module-scope call at app entry is safe. (Do
|
||||
**not** put it in `initMatrix.ts` — that file is off-limits.)
|
||||
- **Settings card:** render `<CryptoDiagnostics />` inside the Developer Tools
|
||||
page — in `src/app/features/settings/developer-tools/DevelopTools.tsx`, add it
|
||||
to the `Box direction="Column" gap="700"` list (guarded by the existing
|
||||
`developerTools` flag), right after the "Access Token" card. It pulls `mx`
|
||||
from `useMatrixClient()` itself, so it just needs to be placed in the tree.
|
||||
+33
-24
@@ -164,7 +164,7 @@ Status: `[ ]` pending · `[~]` in progress · `[x]` completed
|
||||
|
||||
### [ ] P3-8 · Thread Panel (full side drawer)
|
||||
|
||||
**⚠️ LARGEST FEATURE — requires its own planning session before implementation.**
|
||||
**⚠️ LARGEST FEATURE — 🟢 DESIGN COMPLETE (2026-07), READY FOR ITS OWN EXECUTION SESSION.** The full architecture (SDK-evidence-backed decisions, file inventory, 4-agent partition, risks, verification checklist) is in the Implementation Reference section below — no further planning needed, just a dedicated build session.
|
||||
**What:** A right-side drawer for threaded conversations. Currently "Reply in Thread" exists but there is no panel to read or write thread replies.
|
||||
|
||||
Features:
|
||||
@@ -196,10 +196,10 @@ Features:
|
||||
|
||||
## Priority 4 — Specialized, high complexity, or low priority
|
||||
|
||||
### [ ] P4-7 · Virtualized Infinite Scroll for Search Results
|
||||
### [x] P4-7 · Virtualized Infinite Scroll for Search Results — ALREADY IMPLEMENTED (found 2026-07)
|
||||
|
||||
**What:** Replace the manual "load more" button with an automated, virtualized infinite scroll for search results.
|
||||
**Approach:** Utilize `@tanstack/react-virtual` in `MessageSearch.tsx` to handle the `nextToken` automatically as the user scrolls.
|
||||
**Status:** Done in a prior session — `MessageSearch.tsx` already uses `useVirtualizer` (~line 336) over the result groups AND auto-fetches the `nextToken` page when the last virtual item scrolls into view (~line 469) via `useInfiniteQuery`. Nothing left to build.
|
||||
|
||||
### [ ] P4-8 · Encrypted Message Search Indexing & Caching
|
||||
|
||||
@@ -257,7 +257,7 @@ Features:
|
||||
- Account mgmt: `settings/account/OidcManageAccount.tsx`.
|
||||
- 13 unit tests (discovery/flow/session/cache/callback parsing). All gates green.
|
||||
**Awaiting verification (needs a real MSC3861 server — lotusguild is NOT one):** deploy + log into **mozilla.org** (requires adding mozilla to the deployed `config.json` homeserverList + its domains to the CSP `connect-src`/`img-src` — see below), OR run a local `matrix-authentication-service` + Synapse `msc3861` dev loop.
|
||||
**To enable the mozilla.org test:** add to `matrix/cinny/config.json` homeserverList `"mozilla.org"`, and to the nginx CSP `connect-src`/`img-src`: `https://mozilla.org https://mozilla.modular.im https://chat.mozilla.org https://vector.im`.
|
||||
**Mozilla.org test enablement: ALREADY DEPLOYED (verified 2026-07)** — `matrix/cinny/config.json` homeserverList includes `mozilla.org` and the nginx CSP `connect-src` includes the mozilla/modular/vector domains (`matrix/cinny/nginx.conf:42`). **Nothing blocks the test — just pick mozilla.org on the login screen and complete an OIDC login.**
|
||||
|
||||
---
|
||||
|
||||
@@ -482,9 +482,9 @@ Check back after each Synapse upgrade — re-run `/matrix/client/versions` and `
|
||||
|
||||
## Pending Audits
|
||||
|
||||
### [ ] Audit-3 · Profile banner image — Matrix protocol support
|
||||
### [DEFERRED] Audit-3 · Profile banner image — Matrix protocol support — RESEARCHED (2026-07)
|
||||
|
||||
Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banner field. `uk.tcpip.msc4133.stable = true` on our server — check if a `banner_url` or similar field is defined. If no cross-client standard exists, do not implement.
|
||||
**Finding:** [MSC4427 — Custom banners for user profiles](https://github.com/matrix-org/matrix-spec-proposals/pull/4427) defines a `banner_url` profile field on top of the MSC4133 extensible-profile system (which our server supports, `uk.tcpip.msc4133.stable = true`, and which became stable in Matrix v1.16). However MSC4427 is an **open proposal, not merged** — no cross-client standard yet, so per this item's own rule: do not implement. **Revisit when MSC4427 merges** (implementation would then be small: read/write the field via the MSC4133 profile API + render a banner in UserHero/profile popouts).
|
||||
|
||||
---
|
||||
|
||||
@@ -492,26 +492,35 @@ Research whether Matrix spec or MSC4133 (v1.16) defines a standard profile banne
|
||||
|
||||
Exhaustive, low-level implementation details for backlog items. Follow these patterns to ensure code is "Lotus-perfect" (idiomatic, performant, and TDS-compliant).
|
||||
|
||||
### P3-8 · Thread Panel (Full Side Drawer)
|
||||
### P3-8 · Thread Panel (Full Side Drawer) — 🟢 FULL DESIGN (2026-07, ready to execute)
|
||||
|
||||
**Architecture:** Mirror the `MembersDrawer` pattern but with a specialized timeline.
|
||||
**Decisions (each backed by SDK evidence in node_modules/matrix-js-sdk):**
|
||||
|
||||
- **State (`src/app/state/room/thread.ts`):**
|
||||
```typescript
|
||||
export const activeThreadIdAtom = atom<string | null>(null);
|
||||
```
|
||||
- **Layout (`src/app/features/room/Room.tsx`):** Insert `ThreadPanel` conditionally alongside `RoomTimeline`:
|
||||
```tsx
|
||||
{
|
||||
activeThreadId && (
|
||||
<>
|
||||
<Line variant="Background" direction="Vertical" size="300" />
|
||||
<ThreadPanel roomId={roomId} threadId={activeThreadId} />
|
||||
</>
|
||||
);
|
||||
}
|
||||
```
|
||||
- **Component (`src/app/features/room/thread/ThreadPanel.tsx`):** Use `room.getThread(threadId)` from the SDK. Render a `Header` with a "Close" button that sets `activeThreadIdAtom` to `null`. Reuse `RoomTimeline` but pass a filtered `EventTimelineSet`. Use `thread.timelineSet` directly for the most accurate thread view.
|
||||
| Question | Decision |
|
||||
|---|---|
|
||||
| Thread rendering | **New lean `ThreadTimeline`** reusing `Message`, `useVirtualPaginator`, and RoomTimeline's exported timeline helpers (lines 156-227). Do NOT refactor 2214-line RoomTimeline (its ~35 hooks are hardwired to the room live timeline). |
|
||||
| threadSupport | **Enable `threadSupport: true`** in `initMatrix.ts` (~line 39). ⚠️ Thread replies then LEAVE the main timeline (`room.js eventShouldLiveIn` → `shouldLiveInRoom:false`), retroactively on reload — MUST ship the "N replies" summary chip in the same release. Roots stay in both timelines. |
|
||||
| State | `roomIdToActiveThreadIdAtomFamily` (per-room, mirrors `roomIdToReplyDraftAtomFamily`) in new `state/room/thread.ts` + `getThreadDraftKey(roomId, threadRootId)` = `` `${roomId}::${threadRootId}` `` |
|
||||
| Composer | **Reuse RoomInput**: add optional `threadRootId` prop; scope its 3 atom-family lookups by draftKey (isolates thread drafts from the main composer); pass `threadRootId ?? null` at all 7 `mx.sendMessage/sendEvent` call sites — the SDK's `addThreadRelationIfNeeded` then emits spec-correct `m.thread` relations incl. reply-in-thread. Separate `useEditor()` instance in the panel. Hide schedule + commands in thread mode v1. |
|
||||
| Unreads | v1 = unread badge on the summary chip (`room.getThreadUnreadNotificationCount` — counts already synced independent of threadSupport) + `markThreadAsRead` threaded receipt when panel open at bottom. |
|
||||
| Mobile | Pure CSS like `MembersDrawer.css.ts`: fixed width toRem(360) desktop, `position:fixed; inset:0` under 750px. |
|
||||
|
||||
**Critical side-effect fixes (one-liners, land FIRST):**
|
||||
1. `initMatrix.ts` → `threadSupport: true`.
|
||||
2. `utils/notifications.ts:24` → `sendReadReceipt(latestEvent, type, /*unthreaded*/ true)` — otherwise markAsRead becomes `main`-scoped and room badges stick permanently unread (room unread total includes thread counts).
|
||||
|
||||
**Known SDK traps (verified):**
|
||||
- **Local echo gap:** chronological pending ordering means the thread timelineSet never receives pending events (`canContain` rejects; `room.getPendingEvents()` THROWS in this mode) — ThreadTimeline must render its own pending strip via `RoomEvent.LocalEchoUpdated` filtering on `threadRootId`, deduped against `thread.findEventById`.
|
||||
- **Bootstrap:** `room.getThread(id) ?? room.createThread(id, room.findEventById(id), [], false)` — the SDK auto-fetches via `/relations` and inserts the root at top; gate rendering on `thread.initialEventsFetched`; decrypt with `decryptAllTimelineEvent` after init + each pagination.
|
||||
- **Deep links:** `getEventTimeline(mainSet, threadEventId)` returns undefined for thread events — redirect jump-to-event to the panel (best-effort v1).
|
||||
- **Summary chip** must render from the server-aggregated bundle (`unsigned['m.relations']['m.thread']`) so it works before any Thread object exists.
|
||||
- Room-list "latest message" preview may show the root, not the newest reply — cosmetic, accept v1.
|
||||
|
||||
**File inventory — new:** `state/room/thread.ts` (+test), `features/room/thread/{useThread.ts, threadSummary.ts(+test), ThreadTimeline.tsx(+css), ThreadPanel.tsx(+css), ThreadSummary.tsx, index.ts}`, `hooks/useThreadSummary.ts`. **Edited:** `initMatrix.ts` + `utils/notifications.ts` (coordinator, step 0), `RoomInput.tsx` (threadRootId prop), `RoomTimeline.tsx` (handleReplyClick startThread → open panel; ThreadSummary chips at the two Message call sites; Reply onThreadClick; deep-link redirect), `components/message/Reply.tsx`, `Room.tsx` (render panel after MediaGallery block, gated `!callView && activeThreadId`, `key={roomId+threadId}`).
|
||||
|
||||
**4-agent partition:** step 0 (coordinator one-liners) → A: state+SDK glue (+tests) · B: ThreadTimeline (largest; copies the `useTimelinePagination` pattern rather than exporting it) · C: RoomInput changes · D: panel shell + RoomTimeline/Reply integration — all parallel against pinned interface contracts → coordinator wires Room.tsx + gates.
|
||||
|
||||
**Verification:** gates (tsc/eslint/build/tests) + post-merge manual QA: open thread via chip/menu/indicator; pending→confirmed echo; `is_falling_back:false` on reply-in-thread; main timeline shows root+chip only; badge clears; reload keeps partitioning; encrypted threads decrypt. **Release note required:** threaded replies no longer render inline in the main timeline.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Generated
+34
@@ -51,6 +51,7 @@
|
||||
"immer": "11.1.8",
|
||||
"is-hotkey": "0.2.0",
|
||||
"jotai": "2.20.0",
|
||||
"katex": "0.16.11",
|
||||
"linkify-react": "4.3.3",
|
||||
"linkifyjs": "4.3.3",
|
||||
"matrix-js-sdk": "41.6.0-rc.0",
|
||||
@@ -83,6 +84,7 @@
|
||||
"@types/chroma-js": "3.1.2",
|
||||
"@types/file-saver": "2.0.7",
|
||||
"@types/is-hotkey": "0.1.10",
|
||||
"@types/katex": "0.16.8",
|
||||
"@types/node": "25.9.1",
|
||||
"@types/prismjs": "1.26.6",
|
||||
"@types/react": "19.2.15",
|
||||
@@ -3974,6 +3976,13 @@
|
||||
"integrity": "sha512-dRLjCWHYg4oaA77cxO64oO+7JwCwnIzkZPdrrC71jQmQtlhM556pwKo5bUzqvZndkVbeFLIIi+9TC40JNF5hNQ==",
|
||||
"dev": true
|
||||
},
|
||||
"node_modules/@types/katex": {
|
||||
"version": "0.16.8",
|
||||
"resolved": "https://registry.npmjs.org/@types/katex/-/katex-0.16.8.tgz",
|
||||
"integrity": "sha512-trgaNyfU+Xh2Tc+ABIb44a5AYUpicB3uwirOioeOkNPPbmgRNtcWyDeeFRzjPZENO9Vq8gvVqfhaaXWLlevVwg==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/node": {
|
||||
"version": "25.9.1",
|
||||
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.9.1.tgz",
|
||||
@@ -9087,6 +9096,31 @@
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/katex": {
|
||||
"version": "0.16.11",
|
||||
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.11.tgz",
|
||||
"integrity": "sha512-RQrI8rlHY92OLf3rho/Ts8i/XvjgguEjOkO1BEXcU3N8BqPpSzBNwV/G0Ukr+P/l3ivvJUE/Fa/CwbS6HesGNQ==",
|
||||
"funding": [
|
||||
"https://opencollective.com/katex",
|
||||
"https://github.com/sponsors/katex"
|
||||
],
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"commander": "^8.3.0"
|
||||
},
|
||||
"bin": {
|
||||
"katex": "cli.js"
|
||||
}
|
||||
},
|
||||
"node_modules/katex/node_modules/commander": {
|
||||
"version": "8.3.0",
|
||||
"resolved": "https://registry.npmjs.org/commander/-/commander-8.3.0.tgz",
|
||||
"integrity": "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">= 12"
|
||||
}
|
||||
},
|
||||
"node_modules/keyv": {
|
||||
"version": "4.5.4",
|
||||
"resolved": "https://registry.npmjs.org/keyv/-/keyv-4.5.4.tgz",
|
||||
|
||||
@@ -76,6 +76,7 @@
|
||||
"immer": "11.1.8",
|
||||
"is-hotkey": "0.2.0",
|
||||
"jotai": "2.20.0",
|
||||
"katex": "0.16.11",
|
||||
"linkify-react": "4.3.3",
|
||||
"linkifyjs": "4.3.3",
|
||||
"matrix-js-sdk": "41.6.0-rc.0",
|
||||
@@ -108,6 +109,7 @@
|
||||
"@types/chroma-js": "3.1.2",
|
||||
"@types/file-saver": "2.0.7",
|
||||
"@types/is-hotkey": "0.1.10",
|
||||
"@types/katex": "0.16.8",
|
||||
"@types/node": "25.9.1",
|
||||
"@types/prismjs": "1.26.6",
|
||||
"@types/react": "19.2.15",
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
import React from 'react';
|
||||
import katex from 'katex';
|
||||
import 'katex/dist/katex.min.css';
|
||||
|
||||
type KaTeXProps = {
|
||||
/** Raw LaTeX source (without `$`/`$$` delimiters). */
|
||||
latex: string;
|
||||
/** Render as block (display) math when true, inline otherwise. */
|
||||
displayMode?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* Lazily-loaded KaTeX renderer.
|
||||
*
|
||||
* This module statically imports `katex` and its stylesheet, so both only enter
|
||||
* the bundle via the dynamic `import()` of this file (see the `lazy()` wrapper
|
||||
* in `react-custom-html-parser.tsx`). They are therefore NOT part of the eager
|
||||
* import graph.
|
||||
*
|
||||
* We render with `throwOnError: false`, so KaTeX itself renders a parse error
|
||||
* inline (in its error colour) rather than throwing. The HTML returned by
|
||||
* `renderToString` is produced by our own trusted call from a fixed options
|
||||
* object — it is safe to inject via `dangerouslySetInnerHTML`.
|
||||
*/
|
||||
export default function KaTeX({ latex, displayMode = false }: KaTeXProps) {
|
||||
const html = katex.renderToString(latex, {
|
||||
displayMode,
|
||||
throwOnError: false,
|
||||
output: 'htmlAndMathml',
|
||||
});
|
||||
|
||||
const Wrapper = displayMode ? 'div' : 'span';
|
||||
|
||||
return (
|
||||
<Wrapper
|
||||
// KaTeX output is generated by our own render call (trusted-safe).
|
||||
// eslint-disable-next-line react/no-danger
|
||||
dangerouslySetInnerHTML={{ __html: html }}
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -66,8 +66,9 @@ function ControlButton({ label, glyph, onClick, close }: ControlButtonProps) {
|
||||
*
|
||||
* Renders `null` unless we're inside Tauri **and** the user opted into custom
|
||||
* window chrome. Otherwise it draws a thin (~32px) folds/TDS-styled titlebar: a
|
||||
* draggable region (`data-tauri-drag-region`) with the app brand, plus
|
||||
* minimize / maximize / close controls that call the native window commands.
|
||||
* draggable region (explicit `window_start_drag` on mousedown, double-press to
|
||||
* maximize) with the app brand, plus minimize / maximize / close controls that
|
||||
* call the native window commands.
|
||||
*
|
||||
* OS-aware: Windows/Linux put the controls on the right; macOS mirrors them to
|
||||
* the left (the native traffic-light position) since decorations — and thus the
|
||||
@@ -80,10 +81,18 @@ export function TitleBar() {
|
||||
|
||||
const mac = isMacOS();
|
||||
|
||||
const handleDoubleClick = (evt: MouseEvent<HTMLDivElement>): void => {
|
||||
// Only the drag surface itself toggles maximize, not the brand/children.
|
||||
if (evt.target !== evt.currentTarget) return;
|
||||
invokeTauri('window_toggle_maximize');
|
||||
// Official Tauri custom-titlebar recipe: primary-button mousedown starts an
|
||||
// OS window drag; a double press (detail === 2) toggles maximize instead. An
|
||||
// explicit `window_start_drag` invoke is used rather than
|
||||
// `data-tauri-drag-region` because the attribute only fires when the exact
|
||||
// element is the event target (children like the brand text wouldn't drag).
|
||||
const handleDragMouseDown = (evt: MouseEvent<HTMLDivElement>): void => {
|
||||
if (evt.button !== 0) return;
|
||||
if (evt.detail === 2) {
|
||||
invokeTauri('window_toggle_maximize');
|
||||
} else {
|
||||
invokeTauri('window_start_drag');
|
||||
}
|
||||
};
|
||||
|
||||
const controls = (
|
||||
@@ -108,7 +117,7 @@ export function TitleBar() {
|
||||
);
|
||||
|
||||
const dragRegion = (
|
||||
<div className={css.DragRegion} data-tauri-drag-region onDoubleClick={handleDoubleClick}>
|
||||
<div className={css.DragRegion} onMouseDown={handleDragMouseDown}>
|
||||
<span className={css.Brand}>
|
||||
<Text as="span" size="T200" truncate>
|
||||
Lotus Chat
|
||||
|
||||
@@ -11,6 +11,8 @@ import {
|
||||
Line,
|
||||
toRem,
|
||||
Button,
|
||||
Switch,
|
||||
Chip,
|
||||
} from 'folds';
|
||||
import { useAtom, useAtomValue } from 'jotai';
|
||||
import { useVirtualizer } from '@tanstack/react-virtual';
|
||||
@@ -41,7 +43,9 @@ import {
|
||||
ResultGroup,
|
||||
useMessageSearch,
|
||||
} from './useMessageSearch';
|
||||
import { useLocalMessageSearch } from './useLocalMessageSearch';
|
||||
import { LocalSearchResult, useLocalMessageSearch } from './useLocalMessageSearch';
|
||||
import { searchCacheEnabledAtom } from '../../state/searchCacheEnabled';
|
||||
import { clearAll as clearSearchCache } from '../../utils/searchCache';
|
||||
import { addRecentSearch, recentSearchesAtom } from '../../state/recentSearches';
|
||||
import { SearchResultGroup } from './SearchResultGroup';
|
||||
import { SearchInput } from './SearchInput';
|
||||
@@ -240,6 +244,10 @@ export function MessageSearch({
|
||||
// Bump this whenever more messages are loaded so localResult re-computes
|
||||
const [cacheVersion, setCacheVersion] = useState(0);
|
||||
const handleCacheLoaded = useCallback(() => setCacheVersion((v) => v + 1), []);
|
||||
// Explicit wipe of the persistent on-disk index, then re-run the merge.
|
||||
const handleClearSearchCache = useCallback(() => {
|
||||
clearSearchCache().then(() => setCacheVersion((v) => v + 1));
|
||||
}, []);
|
||||
|
||||
// The rooms actually in scope for this search (mirrors server-side logic)
|
||||
const localSearchRooms = useMemo(
|
||||
@@ -253,24 +261,43 @@ export function MessageSearch({
|
||||
const hasActiveSearch = msgSearchParams.term !== undefined || !!msgSearchParams.senders?.length;
|
||||
const senderOnlyMode = !msgSearchParams.term && !!msgSearchParams.senders?.length;
|
||||
|
||||
// Run synchronous client-side search immediately.
|
||||
// Run the client-side search whenever inputs change.
|
||||
// In text-search mode: covers encrypted rooms only (server handles plaintext).
|
||||
// In sender-only mode: covers all rooms (server has no sender-only search).
|
||||
// cacheVersion in deps so it re-runs after "Load more" paginates new events.
|
||||
const localResult = useMemo(() => {
|
||||
if (!hasActiveSearch) return null;
|
||||
return searchLocalMessages({
|
||||
// The scan is async because — when the persistent cache is enabled — it also
|
||||
// reads cached rows from IndexedDB and merges them with the in-memory hits.
|
||||
// cacheVersion in deps so it re-runs after "Load more" paginates new events;
|
||||
// searchCacheEnabled so toggling the cache re-runs the merge.
|
||||
const [searchCacheEnabled, setSearchCacheEnabled] = useAtom(searchCacheEnabledAtom);
|
||||
const [localResult, setLocalResult] = useState<LocalSearchResult | null>(null);
|
||||
useEffect(() => {
|
||||
if (!hasActiveSearch) {
|
||||
setLocalResult(null);
|
||||
return undefined;
|
||||
}
|
||||
let cancelled = false;
|
||||
searchLocalMessages({
|
||||
term: msgSearchParams.term ?? '',
|
||||
roomIds: localSearchRooms,
|
||||
senders: msgSearchParams.senders,
|
||||
fromTs: msgSearchParams.fromTs,
|
||||
toTs: msgSearchParams.toTs,
|
||||
}).then((result) => {
|
||||
if (!cancelled) setLocalResult(result);
|
||||
});
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [
|
||||
searchLocalMessages,
|
||||
localSearchRooms,
|
||||
msgSearchParams.term,
|
||||
msgSearchParams.senders,
|
||||
msgSearchParams.fromTs,
|
||||
msgSearchParams.toTs,
|
||||
hasActiveSearch,
|
||||
cacheVersion,
|
||||
searchCacheEnabled,
|
||||
]);
|
||||
|
||||
const { status, data, error, fetchNextPage, hasNextPage, isFetchingNextPage } = useInfiniteQuery({
|
||||
@@ -668,6 +695,37 @@ export function MessageSearch({
|
||||
? `Showing locally cached messages from ${localResult.searchedRoomsCount} encrypted room${localResult.searchedRoomsCount !== 1 ? 's' : ''}. Load more history below to extend coverage.`
|
||||
: `No matches in your local cache. Load messages below to search further back.`}
|
||||
</Text>
|
||||
<Box
|
||||
alignItems="Center"
|
||||
gap="200"
|
||||
style={{
|
||||
padding: config.space.S200,
|
||||
background: color.SurfaceVariant.Container,
|
||||
borderRadius: config.radii.R300,
|
||||
}}
|
||||
>
|
||||
<Switch
|
||||
variant="Primary"
|
||||
value={searchCacheEnabled}
|
||||
onChange={setSearchCacheEnabled}
|
||||
/>
|
||||
<Box grow="Yes" direction="Column" style={{ minWidth: 0 }}>
|
||||
<Text size="T300">Persist search index on this device</Text>
|
||||
<Text size="T200" priority="300">
|
||||
Stores decrypted text on this device
|
||||
</Text>
|
||||
</Box>
|
||||
{searchCacheEnabled && (
|
||||
<Chip
|
||||
variant="Secondary"
|
||||
radii="Pill"
|
||||
onClick={handleClearSearchCache}
|
||||
before={<Icon size="100" src={Icons.Delete} />}
|
||||
>
|
||||
<Text size="T200">Clear cached index</Text>
|
||||
</Chip>
|
||||
)}
|
||||
</Box>
|
||||
<Line size="300" variant="Surface" />
|
||||
</Box>
|
||||
{localGroups.length > 0 && (
|
||||
|
||||
@@ -1,12 +1,23 @@
|
||||
import { EventType } from 'matrix-js-sdk';
|
||||
import { EventType, MatrixEvent } from 'matrix-js-sdk';
|
||||
import { useCallback } from 'react';
|
||||
import { useAtomValue } from 'jotai';
|
||||
import { useMatrixClient } from '../../hooks/useMatrixClient';
|
||||
import { ResultGroup, ResultItem } from './useMessageSearch';
|
||||
import { searchCacheEnabledAtom } from '../../state/searchCacheEnabled';
|
||||
import {
|
||||
mergeSearchResults,
|
||||
queryRoom,
|
||||
saveRoomIndex,
|
||||
SearchCacheRow,
|
||||
} from '../../utils/searchCache';
|
||||
|
||||
export type LocalSearchParams = {
|
||||
term: string;
|
||||
roomIds: string[];
|
||||
senders?: string[];
|
||||
/** Optional date-range filter (ms). Applied to both memory and cached rows. */
|
||||
fromTs?: number;
|
||||
toTs?: number;
|
||||
};
|
||||
|
||||
export type LocalSearchResult = {
|
||||
@@ -17,19 +28,110 @@ export type LocalSearchResult = {
|
||||
searchedRoomsCount: number;
|
||||
};
|
||||
|
||||
/** Extracted, searchable plaintext for a single message event. */
|
||||
type ExtractedText = {
|
||||
body: string;
|
||||
formattedBody: string;
|
||||
pollText: string;
|
||||
};
|
||||
|
||||
const POLL_START_TYPES = ['m.poll.start', 'org.matrix.msc3381.poll.start'];
|
||||
|
||||
/**
|
||||
* Pull the text we index/search from a decrypted event's content. Returns
|
||||
* `null` for events that carry no searchable text (e.g. stickers).
|
||||
*/
|
||||
const extractText = (event: MatrixEvent): ExtractedText | null => {
|
||||
const evType = event.getType();
|
||||
const content = event.getContent();
|
||||
|
||||
if (POLL_START_TYPES.includes(evType)) {
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const poll = (content['m.poll'] ?? content['org.matrix.msc3381.poll.start']) as any;
|
||||
if (!poll) return null;
|
||||
const qBody =
|
||||
(poll.question?.['m.text'] as Array<{ body: string }> | undefined)?.[0]?.body ??
|
||||
(poll.question?.body as string | undefined) ??
|
||||
'';
|
||||
const answerBodies = ((poll.answers ?? []) as Array<Record<string, unknown>>)
|
||||
.map(
|
||||
(a) =>
|
||||
((a['m.text'] as Array<{ body: string }> | undefined)?.[0]?.body ??
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
(a['org.matrix.msc3381.poll.answer'] as any)?.body ??
|
||||
'') as string,
|
||||
)
|
||||
.join(' ');
|
||||
const pollText = `${qBody} ${answerBodies}`.trim();
|
||||
return pollText ? { body: '', formattedBody: '', pollText } : null;
|
||||
}
|
||||
|
||||
if (evType !== EventType.RoomMessage) return null;
|
||||
|
||||
const body = (content.body as string | undefined) ?? '';
|
||||
const formattedBody = (content.formatted_body as string | undefined) ?? '';
|
||||
if (!body && !formattedBody) return null;
|
||||
return { body, formattedBody, pollText: '' };
|
||||
};
|
||||
|
||||
/** Does the extracted text contain the (already-lowercased) term? */
|
||||
const matchesTerm = (text: ExtractedText, termLower: string): boolean =>
|
||||
text.body.toLowerCase().includes(termLower) ||
|
||||
text.formattedBody.toLowerCase().includes(termLower) ||
|
||||
text.pollText.toLowerCase().includes(termLower);
|
||||
|
||||
const rowMatchesTerm = (row: SearchCacheRow, termLower: string): boolean =>
|
||||
row.body.toLowerCase().includes(termLower) ||
|
||||
(row.formattedBody ?? '').toLowerCase().includes(termLower) ||
|
||||
(row.pollText ?? '').toLowerCase().includes(termLower);
|
||||
|
||||
/** Build the synthetic result item a cached row renders as (text message). */
|
||||
const rowToResultItem = (row: SearchCacheRow): ResultItem => {
|
||||
const bodyText = row.body || row.pollText || '';
|
||||
const content: Record<string, unknown> = { msgtype: 'm.text', body: bodyText };
|
||||
if (row.formattedBody) {
|
||||
content.format = 'org.matrix.custom.html';
|
||||
content.formatted_body = row.formattedBody;
|
||||
}
|
||||
const syntheticEvent = {
|
||||
room_id: row.roomId,
|
||||
event_id: row.eventId,
|
||||
type: EventType.RoomMessage,
|
||||
sender: row.sender,
|
||||
origin_server_ts: row.ts,
|
||||
content,
|
||||
unsigned: {},
|
||||
};
|
||||
return {
|
||||
rank: 0,
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
event: syntheticEvent as any,
|
||||
context: { events_before: [], events_after: [], profile_info: {} },
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Client-side full-text search over locally cached events in encrypted rooms.
|
||||
* The homeserver cannot search E2EE message content, so we scan whatever the
|
||||
* client has already received and decrypted in memory.
|
||||
*
|
||||
* Limitation: only messages present in the live timeline window are covered.
|
||||
* Rooms that haven't been opened yet will return no results.
|
||||
* When the persistent search cache is enabled (opt-in), the in-memory scan is
|
||||
* also persisted to IndexedDB (fire-and-forget) and merged with prior cached
|
||||
* coverage so results survive reloads. When disabled, zero cache reads/writes
|
||||
* occur.
|
||||
*/
|
||||
export const useLocalMessageSearch = () => {
|
||||
const mx = useMatrixClient();
|
||||
const cacheEnabled = useAtomValue(searchCacheEnabledAtom);
|
||||
|
||||
const search = useCallback(
|
||||
({ term, roomIds, senders }: LocalSearchParams): LocalSearchResult => {
|
||||
async ({
|
||||
term,
|
||||
roomIds,
|
||||
senders,
|
||||
fromTs,
|
||||
toTs,
|
||||
}: LocalSearchParams): Promise<LocalSearchResult> => {
|
||||
const trimmedTerm = term.trim();
|
||||
const senderSet = senders && senders.length > 0 ? new Set(senders) : null;
|
||||
|
||||
@@ -41,6 +143,9 @@ export const useLocalMessageSearch = () => {
|
||||
}
|
||||
|
||||
const termLower = trimmedTerm.toLowerCase();
|
||||
const inRange = (ts: number): boolean =>
|
||||
(fromTs === undefined || ts >= fromTs) && (toTs === undefined || ts <= toTs);
|
||||
|
||||
const groups: ResultGroup[] = [];
|
||||
let encryptedRoomsCount = 0;
|
||||
let searchedRoomsCount = 0;
|
||||
@@ -61,106 +166,99 @@ export const useLocalMessageSearch = () => {
|
||||
.getUnfilteredTimelineSet()
|
||||
.getTimelines()
|
||||
.flatMap((tl) => tl.getEvents());
|
||||
if (events.length === 0) continue;
|
||||
|
||||
// eslint-disable-next-line no-await-in-loop
|
||||
const cachedRows = cacheEnabled ? await queryRoom(roomId) : [];
|
||||
|
||||
if (events.length === 0 && cachedRows.length === 0) continue;
|
||||
|
||||
searchedRoomsCount += 1;
|
||||
|
||||
const items: ResultItem[] = [];
|
||||
const memoryItems: ResultItem[] = [];
|
||||
const rowsToPersist: SearchCacheRow[] = [];
|
||||
|
||||
for (let i = 0; i < events.length; i += 1) {
|
||||
const event = events[i];
|
||||
|
||||
// In sender-only mode: include all message types; skip non-message events
|
||||
if (event.getType() !== EventType.RoomMessage) {
|
||||
if (senderOnlyMode) continue;
|
||||
const evType = event.getType();
|
||||
const isSticker = evType === 'm.sticker';
|
||||
const isPoll = evType === 'm.poll.start' || evType === 'org.matrix.msc3381.poll.start';
|
||||
if (!isSticker && !isPoll) continue;
|
||||
}
|
||||
|
||||
if (event.isDecryptionFailure()) continue;
|
||||
if (event.isRedacted()) continue;
|
||||
if (senderSet && !senderSet.has(event.getSender() ?? '')) continue;
|
||||
|
||||
// getContent() returns decrypted plaintext regardless of encryption
|
||||
const content = event.getContent();
|
||||
const evType = event.getType();
|
||||
const isSticker = evType === 'm.sticker';
|
||||
const isMessageLike =
|
||||
evType === EventType.RoomMessage || POLL_START_TYPES.includes(evType);
|
||||
|
||||
// Sender-only mode: no text filter needed
|
||||
if (!senderOnlyMode) {
|
||||
const evType = event.getType();
|
||||
const isPoll = evType === 'm.poll.start' || evType === 'org.matrix.msc3381.poll.start';
|
||||
// Sender-only mode indexes/returns all message types; text mode needs text.
|
||||
if (!senderOnlyMode && !isMessageLike && !isSticker) continue;
|
||||
|
||||
let body = '';
|
||||
let formattedBody = '';
|
||||
if (!isPoll) {
|
||||
body = (content.body as string | undefined) ?? '';
|
||||
formattedBody = (content.formatted_body as string | undefined) ?? '';
|
||||
} else {
|
||||
// Poll — index question text and all answer options
|
||||
const poll = (content['m.poll'] ??
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
content['org.matrix.msc3381.poll.start']) as any;
|
||||
if (poll) {
|
||||
const qBody =
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
(poll.question?.['m.text'] as Array<{ body: string }> | undefined)?.[0]?.body ??
|
||||
(poll.question?.body as string | undefined) ??
|
||||
'';
|
||||
const answerBodies = ((poll.answers ?? []) as Array<Record<string, unknown>>)
|
||||
.map(
|
||||
(a) =>
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
((a['m.text'] as Array<{ body: string }> | undefined)?.[0]?.body ??
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
(a['org.matrix.msc3381.poll.answer'] as any)?.body ??
|
||||
'') as string,
|
||||
)
|
||||
.join(' ');
|
||||
body = `${qBody} ${answerBodies}`.trim();
|
||||
}
|
||||
}
|
||||
const sender = event.getSender() ?? '';
|
||||
const ts = event.getTs();
|
||||
const text = extractText(event);
|
||||
|
||||
if (
|
||||
!body.toLowerCase().includes(termLower) &&
|
||||
!formattedBody.toLowerCase().includes(termLower)
|
||||
)
|
||||
continue;
|
||||
// Persist every indexable (text-bearing) event we scanned, regardless
|
||||
// of whether it matches the current term — future searches benefit.
|
||||
if (cacheEnabled && text && event.getId()) {
|
||||
rowsToPersist.push({
|
||||
roomId,
|
||||
eventId: event.getId() as string,
|
||||
ts,
|
||||
sender,
|
||||
body: text.body,
|
||||
...(text.formattedBody ? { formattedBody: text.formattedBody } : {}),
|
||||
...(text.pollText ? { pollText: text.pollText } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
// Build a synthetic IEventWithRoomId using decrypted content so the
|
||||
// existing SearchResultGroup renderer works without modification.
|
||||
if (senderSet && !senderSet.has(sender)) continue;
|
||||
if (!inRange(ts)) continue;
|
||||
|
||||
if (!senderOnlyMode) {
|
||||
if (!text || !matchesTerm(text, termLower)) continue;
|
||||
}
|
||||
|
||||
const content = event.getContent();
|
||||
const syntheticEvent = {
|
||||
room_id: roomId,
|
||||
event_id: event.getId() ?? '',
|
||||
type: event.getType(),
|
||||
sender: event.getSender() ?? '',
|
||||
origin_server_ts: event.getTs(),
|
||||
type: evType,
|
||||
sender,
|
||||
origin_server_ts: ts,
|
||||
content,
|
||||
unsigned: event.getUnsigned(),
|
||||
};
|
||||
|
||||
items.push({
|
||||
memoryItems.push({
|
||||
rank: 0,
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
event: syntheticEvent as any,
|
||||
context: {
|
||||
events_before: [],
|
||||
events_after: [],
|
||||
profile_info: {},
|
||||
},
|
||||
context: { events_before: [], events_after: [], profile_info: {} },
|
||||
});
|
||||
}
|
||||
|
||||
// Match cached rows (skip ids already present in memory happens in merge).
|
||||
const cachedItems: ResultItem[] = [];
|
||||
cachedRows.forEach((row) => {
|
||||
if (senderSet && !senderSet.has(row.sender)) return;
|
||||
if (!inRange(row.ts)) return;
|
||||
if (!senderOnlyMode && !rowMatchesTerm(row, termLower)) return;
|
||||
cachedItems.push(rowToResultItem(row));
|
||||
});
|
||||
|
||||
const items = mergeSearchResults(memoryItems, cachedItems);
|
||||
|
||||
if (items.length > 0) {
|
||||
items.sort((a, b) => (b.event.origin_server_ts ?? 0) - (a.event.origin_server_ts ?? 0));
|
||||
groups.push({ roomId, items });
|
||||
}
|
||||
|
||||
// Fire-and-forget persist of freshly scanned rows + coverage.
|
||||
// saveRoomIndex swallows all errors internally, so a floating promise
|
||||
// here can never reject.
|
||||
if (cacheEnabled && rowsToPersist.length > 0) {
|
||||
saveRoomIndex(roomId, rowsToPersist);
|
||||
}
|
||||
}
|
||||
|
||||
return { groups, encryptedRoomsCount, searchedRoomsCount };
|
||||
},
|
||||
[mx],
|
||||
[mx, cacheEnabled],
|
||||
);
|
||||
|
||||
return search;
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
} from '../../../components/AccountDataEditor';
|
||||
import { copyToClipboard } from '../../../utils/dom';
|
||||
import { AccountData } from './AccountData';
|
||||
import { CryptoDiagnostics } from '../developer/CryptoDiagnostics';
|
||||
|
||||
type DeveloperToolsProps = {
|
||||
requestClose: () => void;
|
||||
@@ -109,6 +110,7 @@ export function DeveloperTools({ requestClose }: DeveloperToolsProps) {
|
||||
/>
|
||||
</SequenceCard>
|
||||
)}
|
||||
{developerTools && <CryptoDiagnostics />}
|
||||
</Box>
|
||||
{developerTools && (
|
||||
<AccountData
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
import React, { useCallback } from 'react';
|
||||
import { Badge, Box, Button, Text } from 'folds';
|
||||
import { SequenceCard } from '../../../components/sequence-card';
|
||||
import { SequenceCardStyle } from '../styles.css';
|
||||
import { SettingTile } from '../../../components/setting-tile';
|
||||
import { useMatrixClient } from '../../../hooks/useMatrixClient';
|
||||
import { useForceUpdate } from '../../../hooks/useForceUpdate';
|
||||
import { useInterval } from '../../../hooks/useInterval';
|
||||
import { buildCryptoDiagReport, getCryptoDiagEntries } from '../../../utils/cryptoDiagLog';
|
||||
|
||||
// Lotus E2EE investigation kit — Crypto Diagnostics settings card.
|
||||
// Mirrors the surrounding Developer Tools cards (see DevelopTools.tsx).
|
||||
|
||||
const REFRESH_MS = 1000;
|
||||
|
||||
export function CryptoDiagnostics() {
|
||||
const mx = useMatrixClient();
|
||||
// Re-render on a light interval so the live matched-entry count stays fresh
|
||||
// while the settings pane is open.
|
||||
const [, forceUpdate] = useForceUpdate();
|
||||
useInterval(forceUpdate, REFRESH_MS);
|
||||
|
||||
const count = getCryptoDiagEntries().length;
|
||||
|
||||
const handleDownload = useCallback(() => {
|
||||
const report = buildCryptoDiagReport(mx);
|
||||
const blob = new Blob([report], { type: 'application/json' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `lotus-crypto-diag-${new Date().toISOString().replace(/[:.]/g, '-')}.json`;
|
||||
a.click();
|
||||
URL.revokeObjectURL(url);
|
||||
}, [mx]);
|
||||
|
||||
return (
|
||||
<Box direction="Column" gap="100">
|
||||
<Text size="L400">Crypto Diagnostics</Text>
|
||||
<SequenceCard
|
||||
className={SequenceCardStyle}
|
||||
variant="SurfaceVariant"
|
||||
direction="Column"
|
||||
gap="400"
|
||||
>
|
||||
<SettingTile
|
||||
title="Crypto Diagnostics — captures E2EE error signatures this session"
|
||||
description="Ring-buffers up to 200 matched console warnings/errors for the KE-1..KE-4 bug cluster. Local only — no network calls. The downloaded report includes the matched log lines as evidence."
|
||||
after={
|
||||
<Box alignItems="Center" gap="200" shrink="No">
|
||||
<Badge variant={count > 0 ? 'Critical' : 'Secondary'} fill="Solid" radii="Pill">
|
||||
<Text as="span" size="L400">
|
||||
{count}
|
||||
</Text>
|
||||
</Badge>
|
||||
<Button
|
||||
onClick={handleDownload}
|
||||
variant="Secondary"
|
||||
fill="Soft"
|
||||
size="300"
|
||||
radii="300"
|
||||
outlined
|
||||
>
|
||||
<Text size="B300">Download report</Text>
|
||||
</Button>
|
||||
</Box>
|
||||
}
|
||||
/>
|
||||
</SequenceCard>
|
||||
</Box>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import { useEffect } from 'react';
|
||||
import { getFallbackSession, subscribeSessionChanges } from '../state/sessions';
|
||||
|
||||
/**
|
||||
* Keep this tab in sync with session changes performed in other tabs/windows.
|
||||
*
|
||||
* The coordinator mounts this once inside the authenticated client shell.
|
||||
* `storage` events fire only in tabs that did NOT perform the write, so the
|
||||
* callback here always represents an out-of-tab change.
|
||||
*
|
||||
* Default action is the safest one for auth-critical state — a full reload:
|
||||
* - session REMOVED elsewhere (logout / localStorage.clear()) → the access
|
||||
* token disappears, so we reload; the router bounces to auth on next boot.
|
||||
* - session APPEARED or its access token CHANGED elsewhere (a fresh login or
|
||||
* a token rotation) → we reload so the client re-initialises with the new
|
||||
* credentials rather than running on a stale/revoked token.
|
||||
*
|
||||
* A change that does not alter the access token (e.g. an OIDC metadata-only
|
||||
* rewrite) is ignored, which also collapses the several storage events emitted
|
||||
* by a single dual-write into at most one reload.
|
||||
*/
|
||||
export const useSessionSync = (): void => {
|
||||
useEffect(() => {
|
||||
// Snapshot the credential this tab booted with; compare against it so we
|
||||
// only reload on a genuine credential change.
|
||||
const initialAccessToken = getFallbackSession()?.accessToken ?? null;
|
||||
|
||||
const unsubscribe = subscribeSessionChanges((session) => {
|
||||
const nextAccessToken = session?.accessToken ?? null;
|
||||
if (nextAccessToken === initialAccessToken) return;
|
||||
window.location.reload();
|
||||
});
|
||||
|
||||
return unsubscribe;
|
||||
}, []);
|
||||
};
|
||||
@@ -43,8 +43,15 @@ import { stopPropagation } from '../../utils/keyboard';
|
||||
import { SyncStatus } from './SyncStatus';
|
||||
import { AuthMetadataProvider } from '../../hooks/useAuthMetadata';
|
||||
import { getFallbackSession, removeFallbackSession } from '../../state/sessions';
|
||||
import { useSessionSync } from '../../hooks/useSessionSync';
|
||||
import { installCryptoDiagLog } from '../../utils/cryptoDiagLog';
|
||||
import { AutoDiscovery } from './AutoDiscovery';
|
||||
|
||||
// Capture-only E2EE diagnostics ring buffer (KE-1→4 signatures) — installed at
|
||||
// module load so it sees crypto warnings from the very first sync. Idempotent;
|
||||
// report download lives in Settings → Developer Tools → Crypto Diagnostics.
|
||||
installCryptoDiagLog();
|
||||
|
||||
function ClientRootLoading() {
|
||||
return (
|
||||
<SplashScreen>
|
||||
@@ -178,6 +185,9 @@ export function ClientRoot({ children }: ClientRootProps) {
|
||||
);
|
||||
|
||||
useLogoutListener(mx);
|
||||
// Cross-tab session sync: another tab logging out / in (access token changed
|
||||
// in localStorage) reloads this tab so it never runs with stale credentials.
|
||||
useSessionSync();
|
||||
|
||||
useEffect(() => {
|
||||
if (loadState.status === AsyncStatus.Idle) {
|
||||
|
||||
@@ -43,9 +43,14 @@ import { onEnterOrSpace } from '../utils/keyboard';
|
||||
import { copyToClipboard, tryDecodeURIComponent } from '../utils/dom';
|
||||
import { useTimeoutToggle } from '../hooks/useTimeoutToggle';
|
||||
import { tokenize, tokenStyle } from '../utils/syntaxHighlight';
|
||||
import { splitMathSegments } from '../utils/mathParse';
|
||||
|
||||
const ReactPrism = lazy(() => import('./react-prism/ReactPrism'));
|
||||
|
||||
// KaTeX (and its CSS) is heavy, so it is code-split behind this dynamic import
|
||||
// and is NOT part of the eager import graph — see src/app/components/math/KaTeX.tsx.
|
||||
const KaTeXMath = lazy(() => import('../components/math/KaTeX'));
|
||||
|
||||
/** Languages handled by the custom TDS tokenizer. */
|
||||
const TDS_TOKENIZER_LANGS = new Set([
|
||||
'js',
|
||||
@@ -78,6 +83,27 @@ function renderTokenizedCode(code: string, lang: string): React.ReactNode {
|
||||
));
|
||||
}
|
||||
|
||||
/**
|
||||
* Renders LaTeX via the lazily-loaded KaTeX component.
|
||||
*
|
||||
* `suspenseFallback` is shown while the KaTeX chunk loads (the raw LaTeX text).
|
||||
* `errorFallback` is shown if rendering fails outright — for the spec
|
||||
* `data-mx-maths` path this is the element's original children (the spec
|
||||
* fallback content); for the plain-text `$…$` path it is the raw source.
|
||||
*/
|
||||
const renderMath = (
|
||||
latex: string,
|
||||
displayMode: boolean,
|
||||
suspenseFallback: React.ReactNode,
|
||||
errorFallback: React.ReactNode,
|
||||
): JSX.Element => (
|
||||
<ErrorBoundary fallback={<>{errorFallback}</>}>
|
||||
<Suspense fallback={<>{suspenseFallback}</>}>
|
||||
<KaTeXMath latex={latex} displayMode={displayMode} />
|
||||
</Suspense>
|
||||
</ErrorBoundary>
|
||||
);
|
||||
|
||||
const EMOJI_REG_G = new RegExp(`${URL_NEG_LB}(${EMOJI_PATTERN})`, 'g');
|
||||
|
||||
export const LINKIFY_OPTS: LinkifyOpts = {
|
||||
@@ -503,6 +529,21 @@ export const getReactCustomHtmlParser = (
|
||||
if (mention) return mention;
|
||||
}
|
||||
|
||||
if ((name === 'span' || name === 'div') && 'data-mx-maths' in props) {
|
||||
// Spec (CS-API §11.5): render the `data-mx-maths` LaTeX with KaTeX
|
||||
// (block for <div>, inline for <span>). On failure fall back to the
|
||||
// element's existing children, which the spec defines as the fallback
|
||||
// representation.
|
||||
const latex = String(props['data-mx-maths']);
|
||||
const displayMode = name === 'div';
|
||||
const fallback = displayMode ? (
|
||||
<div {...props}>{domToReact(children as unknown as DOMNode[], opts)}</div>
|
||||
) : (
|
||||
<span {...props}>{domToReact(children as unknown as DOMNode[], opts)}</span>
|
||||
);
|
||||
return renderMath(latex, displayMode, latex, fallback);
|
||||
}
|
||||
|
||||
if (name === 'span' && 'data-mx-spoiler' in props) {
|
||||
return (
|
||||
<span
|
||||
@@ -546,20 +587,50 @@ export const getReactCustomHtmlParser = (
|
||||
}
|
||||
|
||||
if (domNode instanceof DOMText) {
|
||||
const linkify =
|
||||
!(domNode.parent && 'name' in domNode.parent && domNode.parent.name === 'code') &&
|
||||
!(domNode.parent && 'name' in domNode.parent && domNode.parent.name === 'a');
|
||||
const parentName =
|
||||
domNode.parent && 'name' in domNode.parent ? domNode.parent.name : undefined;
|
||||
const linkify = parentName !== 'code' && parentName !== 'a';
|
||||
// Never parse `$…$`/`$$…$$` math inside <pre>/<code> (verbatim regions).
|
||||
const mathAllowed = parentName !== 'code' && parentName !== 'pre';
|
||||
|
||||
let jsx = scaleSystemEmoji(domNode.data);
|
||||
const renderTextChunk = (text: string): (string | JSX.Element)[] | JSX.Element => {
|
||||
let jsx = scaleSystemEmoji(text);
|
||||
if (params.highlightRegex) {
|
||||
jsx = highlightText(params.highlightRegex, jsx);
|
||||
}
|
||||
if (linkify) {
|
||||
return <Linkify options={params.linkifyOpts}>{jsx}</Linkify>;
|
||||
}
|
||||
return jsx;
|
||||
};
|
||||
|
||||
if (params.highlightRegex) {
|
||||
jsx = highlightText(params.highlightRegex, jsx);
|
||||
if (mathAllowed) {
|
||||
const segments = splitMathSegments(domNode.data);
|
||||
if (segments.some((segment) => segment.type !== 'text')) {
|
||||
return (
|
||||
<>
|
||||
{segments.map((segment, index) => {
|
||||
if (segment.type === 'text') {
|
||||
// eslint-disable-next-line react/no-array-index-key
|
||||
return (
|
||||
<React.Fragment key={index}>{renderTextChunk(segment.value)}</React.Fragment>
|
||||
);
|
||||
}
|
||||
const raw =
|
||||
segment.type === 'block' ? `$$${segment.value}$$` : `$${segment.value}$`;
|
||||
return (
|
||||
// eslint-disable-next-line react/no-array-index-key
|
||||
<React.Fragment key={index}>
|
||||
{renderMath(segment.value, segment.type === 'block', raw, raw)}
|
||||
</React.Fragment>
|
||||
);
|
||||
})}
|
||||
</>
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (linkify) {
|
||||
return <Linkify options={params.linkifyOpts}>{jsx}</Linkify>;
|
||||
}
|
||||
return jsx;
|
||||
return renderTextChunk(domNode.data);
|
||||
}
|
||||
return undefined;
|
||||
},
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
import {
|
||||
atomWithLocalStorage,
|
||||
getLocalStorageItem,
|
||||
setLocalStorageItem,
|
||||
} from './utils/atomWithLocalStorage';
|
||||
|
||||
const SEARCH_CACHE_ENABLED = 'searchCacheEnabled';
|
||||
|
||||
/**
|
||||
* P4-8 — persistent encrypted-search cache opt-in flag (default `false`).
|
||||
*
|
||||
* Standalone, `localStorage`-backed boolean atom kept separate from
|
||||
* `state/settings.ts` on purpose. When `true`, encrypted-room search persists a
|
||||
* decrypted plaintext index to IndexedDB (`lotus-search-cache`) so coverage
|
||||
* survives reloads. Because this writes decrypted plaintext at rest it must be
|
||||
* explicitly opted into; the cache is clearable from the search UI and wiped on
|
||||
* logout. Toggling this atom off stops all reads/writes but does NOT wipe
|
||||
* existing data — that is the explicit "Clear cached index" button / logout.
|
||||
*/
|
||||
export const searchCacheEnabledAtom = atomWithLocalStorage<boolean>(
|
||||
SEARCH_CACHE_ENABLED,
|
||||
(key) => getLocalStorageItem<boolean>(key, false),
|
||||
(key, value) => setLocalStorageItem(key, value),
|
||||
);
|
||||
@@ -1,6 +1,14 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { setFallbackSession, removeFallbackSession, getFallbackSession } from './sessions';
|
||||
import {
|
||||
setFallbackSession,
|
||||
removeFallbackSession,
|
||||
getFallbackSession,
|
||||
subscribeSessionChanges,
|
||||
} from './sessions';
|
||||
|
||||
// The single-key atomic blob (kept in sync with SESSION_BLOB_KEY in sessions.ts).
|
||||
const SESSION_BLOB_KEY = 'cinny_session_v1';
|
||||
|
||||
// The fallback-session helpers read/write specific `cinny_*` keys directly on
|
||||
// `localStorage`. node has none, so install a controllable in-memory mock per
|
||||
@@ -47,8 +55,9 @@ test('getFallbackSession returns undefined when nothing is stored', () => {
|
||||
assert.equal(getFallbackSession(), undefined);
|
||||
});
|
||||
|
||||
test('getFallbackSession returns undefined when a single key is missing', () => {
|
||||
// Every one of the four keys is required; missing any one yields undefined.
|
||||
test('legacy path: undefined when a single legacy key is missing (no blob)', () => {
|
||||
// With no atomic blob, every one of the four legacy keys is required; missing
|
||||
// any one yields undefined (the pre-blob behaviour).
|
||||
const keys = [
|
||||
'cinny_access_token',
|
||||
'cinny_device_id',
|
||||
@@ -59,11 +68,26 @@ test('getFallbackSession returns undefined when a single key is missing', () =>
|
||||
keys.forEach((missing) => {
|
||||
installStorage();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
localStorage.removeItem(SESSION_BLOB_KEY);
|
||||
localStorage.removeItem(missing);
|
||||
assert.equal(getFallbackSession(), undefined, `missing ${missing} should yield undefined`);
|
||||
});
|
||||
});
|
||||
|
||||
test('blob wins: a torn legacy key does NOT tear the session while the blob exists', () => {
|
||||
installStorage();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
// Simulate a torn legacy write — the authoritative blob must still resolve.
|
||||
localStorage.removeItem('cinny_access_token');
|
||||
assert.deepEqual(getFallbackSession(), {
|
||||
baseUrl: 'https://hs.example.org',
|
||||
userId: '@alice:example.org',
|
||||
deviceId: 'DEVICE1',
|
||||
accessToken: 'token-1',
|
||||
fallbackSdkStores: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('removeFallbackSession clears all keys', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
@@ -113,4 +137,298 @@ test('a password session carries no OIDC fields, and re-saving clears stale OIDC
|
||||
assert.ok(s);
|
||||
assert.equal(s.oidc, undefined);
|
||||
assert.equal(s.refreshToken, undefined);
|
||||
// The overwritten blob must not retain the stale OIDC state either.
|
||||
const blob = JSON.parse(localStorage.getItem(SESSION_BLOB_KEY)!);
|
||||
assert.equal(blob.oidc, undefined);
|
||||
assert.equal(blob.refreshToken, undefined);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Atomic blob: write/read round-trip
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('setFallbackSession writes a single atomic blob under cinny_session_v1', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
|
||||
const raw = store.get(SESSION_BLOB_KEY);
|
||||
assert.ok(raw, 'blob key must be written');
|
||||
assert.deepEqual(JSON.parse(raw!), {
|
||||
accessToken: 'token-1',
|
||||
deviceId: 'DEVICE1',
|
||||
userId: '@alice:example.org',
|
||||
baseUrl: 'https://hs.example.org',
|
||||
});
|
||||
});
|
||||
|
||||
test('blob round-trips a full OIDC session (absolute expiry stored, remaining read back)', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('tok', 'DEV', '@bob:mozilla.org', 'https://hs', {
|
||||
refreshToken: 'refresh-xyz',
|
||||
expiresInMs: 3_600_000,
|
||||
oidc: {
|
||||
issuer: 'https://i',
|
||||
clientId: 'c',
|
||||
redirectUri: 'https://cb',
|
||||
idTokenClaims: { sub: '@bob:mozilla.org' },
|
||||
},
|
||||
});
|
||||
|
||||
const blob = JSON.parse(store.get(SESSION_BLOB_KEY)!);
|
||||
assert.equal(blob.refreshToken, 'refresh-xyz');
|
||||
assert.ok(typeof blob.expiresAt === 'number' && blob.expiresAt > Date.now());
|
||||
assert.deepEqual(blob.oidc, {
|
||||
issuer: 'https://i',
|
||||
clientId: 'c',
|
||||
redirectUri: 'https://cb',
|
||||
idTokenClaims: { sub: '@bob:mozilla.org' },
|
||||
});
|
||||
|
||||
const s = getFallbackSession();
|
||||
assert.ok(s);
|
||||
assert.equal(s.refreshToken, 'refresh-xyz');
|
||||
assert.ok(s.expiresInMs! > 0 && s.expiresInMs! <= 3_600_000);
|
||||
assert.deepEqual(s.oidc, {
|
||||
issuer: 'https://i',
|
||||
clientId: 'c',
|
||||
redirectUri: 'https://cb',
|
||||
idTokenClaims: { sub: '@bob:mozilla.org' },
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Migration: legacy-only storage → transparent read → blob persisted on write
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('legacy-only storage (no blob) is read transparently', () => {
|
||||
const store = installStorage();
|
||||
// Simulate an older build: legacy keys present, no blob.
|
||||
store.set('cinny_access_token', 'tok');
|
||||
store.set('cinny_device_id', 'DEV');
|
||||
store.set('cinny_user_id', '@carol:example.org');
|
||||
store.set('cinny_hs_base_url', 'https://hs');
|
||||
assert.equal(store.has(SESSION_BLOB_KEY), false);
|
||||
|
||||
assert.deepEqual(getFallbackSession(), {
|
||||
baseUrl: 'https://hs',
|
||||
userId: '@carol:example.org',
|
||||
deviceId: 'DEV',
|
||||
accessToken: 'tok',
|
||||
fallbackSdkStores: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('first write after a legacy-only read persists the blob (migration)', () => {
|
||||
const store = installStorage();
|
||||
store.set('cinny_access_token', 'old');
|
||||
store.set('cinny_device_id', 'DEV');
|
||||
store.set('cinny_user_id', '@carol:example.org');
|
||||
store.set('cinny_hs_base_url', 'https://hs');
|
||||
|
||||
// Reads are side-effect free — no blob yet.
|
||||
getFallbackSession();
|
||||
assert.equal(store.has(SESSION_BLOB_KEY), false);
|
||||
|
||||
// The next write (e.g. a token refresh) persists the atomic blob.
|
||||
setFallbackSession('new', 'DEV', '@carol:example.org', 'https://hs');
|
||||
assert.ok(store.has(SESSION_BLOB_KEY));
|
||||
assert.equal(getFallbackSession()?.accessToken, 'new');
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Corruption / partial blob → legacy fallback; blob wins on disagreement
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('corrupt blob (bad JSON) falls back to the legacy keys', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
// Corrupt the blob but keep the legacy keys intact.
|
||||
store.set(SESSION_BLOB_KEY, '{not valid json');
|
||||
|
||||
assert.deepEqual(getFallbackSession(), {
|
||||
baseUrl: 'https://hs.example.org',
|
||||
userId: '@alice:example.org',
|
||||
deviceId: 'DEVICE1',
|
||||
accessToken: 'token-1',
|
||||
fallbackSdkStores: true,
|
||||
});
|
||||
});
|
||||
|
||||
test('partial blob (missing a required field) falls back to the legacy keys', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
// A blob missing accessToken is treated as absent.
|
||||
store.set(
|
||||
SESSION_BLOB_KEY,
|
||||
JSON.stringify({ deviceId: 'DEVICE1', userId: '@alice:example.org', baseUrl: 'https://hs' }),
|
||||
);
|
||||
|
||||
assert.equal(getFallbackSession()?.accessToken, 'token-1');
|
||||
});
|
||||
|
||||
test('blob wins when blob and legacy keys disagree', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('blob-token', 'DEVICE1', '@alice:example.org', 'https://hs.example.org');
|
||||
// Legacy keys drift to a stale token; the blob is authoritative.
|
||||
store.set('cinny_access_token', 'stale-legacy-token');
|
||||
|
||||
assert.equal(getFallbackSession()?.accessToken, 'blob-token');
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Dual-write keeps blob + legacy in sync; removal clears both
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('dual-write keeps the legacy keys in sync with the blob', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('tok', 'DEV', '@bob:mozilla.org', 'https://hs', {
|
||||
refreshToken: 'r',
|
||||
expiresInMs: 1000,
|
||||
oidc: { issuer: 'https://i', clientId: 'c', redirectUri: 'https://cb' },
|
||||
});
|
||||
|
||||
// Legacy credential keys
|
||||
assert.equal(store.get('cinny_access_token'), 'tok');
|
||||
assert.equal(store.get('cinny_device_id'), 'DEV');
|
||||
assert.equal(store.get('cinny_user_id'), '@bob:mozilla.org');
|
||||
assert.equal(store.get('cinny_hs_base_url'), 'https://hs');
|
||||
// Legacy OIDC keys
|
||||
assert.equal(store.get('cinny_refresh_token'), 'r');
|
||||
assert.ok(store.has('cinny_expires_at'));
|
||||
assert.equal(store.get('cinny_oidc_issuer'), 'https://i');
|
||||
assert.equal(store.get('cinny_oidc_client_id'), 'c');
|
||||
assert.equal(store.get('cinny_oidc_redirect_uri'), 'https://cb');
|
||||
// Blob agrees
|
||||
const blob = JSON.parse(store.get(SESSION_BLOB_KEY)!);
|
||||
assert.equal(blob.accessToken, 'tok');
|
||||
assert.equal(blob.refreshToken, 'r');
|
||||
assert.equal(store.get('cinny_expires_at'), String(blob.expiresAt));
|
||||
});
|
||||
|
||||
test('removeFallbackSession clears BOTH the blob and every legacy key', () => {
|
||||
const store = installStorage();
|
||||
setFallbackSession('tok', 'DEV', '@bob:mozilla.org', 'https://hs', {
|
||||
refreshToken: 'r',
|
||||
expiresInMs: 1000,
|
||||
oidc: { issuer: 'https://i', clientId: 'c', redirectUri: 'https://cb' },
|
||||
});
|
||||
assert.ok(store.size > 0);
|
||||
|
||||
removeFallbackSession();
|
||||
assert.equal(store.size, 0, 'no session key may survive removal');
|
||||
assert.equal(getFallbackSession(), undefined);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Token-refresh update path (the path LotusOidcTokenRefresher uses)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test('token refresh via setFallbackSession updates blob + legacy atomically', () => {
|
||||
const store = installStorage();
|
||||
// Initial OIDC session.
|
||||
setFallbackSession('access-1', 'DEV', '@bob:mozilla.org', 'https://hs', {
|
||||
refreshToken: 'refresh-1',
|
||||
oidc: { issuer: 'https://i', clientId: 'c', redirectUri: 'https://cb' },
|
||||
});
|
||||
|
||||
// LotusOidcTokenRefresher.persistTokens() calls setFallbackSession with the
|
||||
// rotated tokens and the same identity/oidc refs.
|
||||
setFallbackSession('access-2', 'DEV', '@bob:mozilla.org', 'https://hs', {
|
||||
refreshToken: 'refresh-2',
|
||||
oidc: { issuer: 'https://i', clientId: 'c', redirectUri: 'https://cb' },
|
||||
});
|
||||
|
||||
// Blob updated
|
||||
const blob = JSON.parse(store.get(SESSION_BLOB_KEY)!);
|
||||
assert.equal(blob.accessToken, 'access-2');
|
||||
assert.equal(blob.refreshToken, 'refresh-2');
|
||||
// Legacy keys updated in lockstep
|
||||
assert.equal(store.get('cinny_access_token'), 'access-2');
|
||||
assert.equal(store.get('cinny_refresh_token'), 'refresh-2');
|
||||
// Reader sees the fresh token
|
||||
const s = getFallbackSession();
|
||||
assert.equal(s?.accessToken, 'access-2');
|
||||
assert.equal(s?.refreshToken, 'refresh-2');
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cross-tab sync: subscribeSessionChanges
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Minimal window/storage-event harness: node has neither.
|
||||
const installWindow = (): ((evt: { key: string | null }) => void)[] => {
|
||||
const listeners: ((evt: { key: string | null }) => void)[] = [];
|
||||
(globalThis as { window?: unknown }).window = {
|
||||
addEventListener: (type: string, cb: (evt: { key: string | null }) => void) => {
|
||||
if (type === 'storage') listeners.push(cb);
|
||||
},
|
||||
removeEventListener: (type: string, cb: (evt: { key: string | null }) => void) => {
|
||||
if (type !== 'storage') return;
|
||||
const i = listeners.indexOf(cb);
|
||||
if (i !== -1) listeners.splice(i, 1);
|
||||
},
|
||||
};
|
||||
return listeners;
|
||||
};
|
||||
|
||||
test('subscribeSessionChanges fires with the session when a session key changes', () => {
|
||||
installStorage();
|
||||
const listeners = installWindow();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs');
|
||||
|
||||
let received: unknown = 'unset';
|
||||
const unsub = subscribeSessionChanges((s) => {
|
||||
received = s;
|
||||
});
|
||||
|
||||
// Simulate another tab writing a new token, then dispatch the storage event.
|
||||
setFallbackSession('token-2', 'DEVICE1', '@alice:example.org', 'https://hs');
|
||||
listeners.forEach((cb) => cb({ key: SESSION_BLOB_KEY }));
|
||||
|
||||
assert.notEqual(received, 'unset');
|
||||
assert.equal((received as { accessToken?: string })?.accessToken, 'token-2');
|
||||
|
||||
unsub();
|
||||
assert.equal(listeners.length, 0, 'unsubscribe removes the listener');
|
||||
});
|
||||
|
||||
test('subscribeSessionChanges fires with null when the session is removed', () => {
|
||||
installStorage();
|
||||
const listeners = installWindow();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs');
|
||||
subscribeSessionChanges((s) => {
|
||||
assert.equal(s, null);
|
||||
});
|
||||
|
||||
removeFallbackSession();
|
||||
listeners.forEach((cb) => cb({ key: SESSION_BLOB_KEY }));
|
||||
});
|
||||
|
||||
test('subscribeSessionChanges treats a null key (localStorage.clear) as a change', () => {
|
||||
const store = installStorage();
|
||||
const listeners = installWindow();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs');
|
||||
|
||||
let fired = false;
|
||||
subscribeSessionChanges(() => {
|
||||
fired = true;
|
||||
});
|
||||
|
||||
store.clear();
|
||||
listeners.forEach((cb) => cb({ key: null }));
|
||||
assert.equal(fired, true);
|
||||
});
|
||||
|
||||
test('subscribeSessionChanges ignores unrelated storage keys', () => {
|
||||
installStorage();
|
||||
const listeners = installWindow();
|
||||
setFallbackSession('token-1', 'DEVICE1', '@alice:example.org', 'https://hs');
|
||||
|
||||
let fired = false;
|
||||
subscribeSessionChanges(() => {
|
||||
fired = true;
|
||||
});
|
||||
|
||||
listeners.forEach((cb) => cb({ key: 'some_unrelated_preference' }));
|
||||
assert.equal(fired, false);
|
||||
});
|
||||
|
||||
+230
-76
@@ -26,6 +26,15 @@ export type Session = {
|
||||
oidc?: OidcSessionMeta;
|
||||
};
|
||||
|
||||
// Legacy per-field localStorage keys. Kept for dual-write (see below) so a
|
||||
// rollback to an older build that only understands these keys still works.
|
||||
const LEGACY_KEYS = {
|
||||
accessToken: 'cinny_access_token',
|
||||
deviceId: 'cinny_device_id',
|
||||
userId: 'cinny_user_id',
|
||||
baseUrl: 'cinny_hs_base_url',
|
||||
} as const;
|
||||
|
||||
// OIDC-only localStorage keys (absent for password/legacy-SSO sessions).
|
||||
const OIDC_KEYS = {
|
||||
refreshToken: 'cinny_refresh_token',
|
||||
@@ -36,6 +45,174 @@ const OIDC_KEYS = {
|
||||
idTokenClaims: 'cinny_oidc_id_token_claims',
|
||||
} as const;
|
||||
|
||||
// Single-key atomic session blob. The whole session is serialised and written
|
||||
// in ONE `setItem`, so a reader can never observe a torn/partial session the
|
||||
// way the multi-key legacy layout could. Bumping the schema means bumping the
|
||||
// `_v1` suffix.
|
||||
const SESSION_BLOB_KEY = 'cinny_session_v1';
|
||||
|
||||
// The exact shape stored inside SESSION_BLOB_KEY. Note it stores an ABSOLUTE
|
||||
// `expiresAt` (ms since epoch) rather than a relative lifetime — identical to
|
||||
// the legacy `cinny_expires_at` semantics — so reads stay drift-free.
|
||||
type PersistedSession = {
|
||||
accessToken: string;
|
||||
deviceId: string;
|
||||
userId: string;
|
||||
baseUrl: string;
|
||||
refreshToken?: string;
|
||||
expiresAt?: number;
|
||||
oidc?: OidcSessionMeta;
|
||||
};
|
||||
|
||||
// Build the persisted shape from the public setFallbackSession arguments. This
|
||||
// is the single source of truth written to BOTH the blob and the legacy keys.
|
||||
const buildPersisted = (
|
||||
accessToken: string,
|
||||
deviceId: string,
|
||||
userId: string,
|
||||
baseUrl: string,
|
||||
extra?: FallbackSessionExtra,
|
||||
): PersistedSession => {
|
||||
const persisted: PersistedSession = { accessToken, deviceId, userId, baseUrl };
|
||||
if (extra?.refreshToken) persisted.refreshToken = extra.refreshToken;
|
||||
// Store ABSOLUTE expiry to avoid drift across reloads.
|
||||
if (typeof extra?.expiresInMs === 'number') persisted.expiresAt = Date.now() + extra.expiresInMs;
|
||||
if (extra?.oidc) persisted.oidc = extra.oidc;
|
||||
return persisted;
|
||||
};
|
||||
|
||||
// Convert a persisted shape into the public Session returned to callers. Keeps
|
||||
// behaviour identical to the original getFallbackSession assembly: derives the
|
||||
// REMAINING lifetime from the absolute expiry, and only surfaces `oidc` when the
|
||||
// three required OIDC fields are present.
|
||||
const sessionFromPersisted = (p: PersistedSession): Session => {
|
||||
const session: Session = {
|
||||
baseUrl: p.baseUrl,
|
||||
userId: p.userId,
|
||||
deviceId: p.deviceId,
|
||||
accessToken: p.accessToken,
|
||||
fallbackSdkStores: true,
|
||||
};
|
||||
|
||||
if (p.refreshToken) session.refreshToken = p.refreshToken;
|
||||
|
||||
if (typeof p.expiresAt === 'number' && Number.isFinite(p.expiresAt)) {
|
||||
// Expose the REMAINING lifetime (clamped at 0); the SDK refreshes on 401.
|
||||
session.expiresInMs = Math.max(0, p.expiresAt - Date.now());
|
||||
}
|
||||
|
||||
if (p.oidc && p.oidc.issuer && p.oidc.clientId && p.oidc.redirectUri) {
|
||||
session.oidc = {
|
||||
issuer: p.oidc.issuer,
|
||||
clientId: p.oidc.clientId,
|
||||
redirectUri: p.oidc.redirectUri,
|
||||
idTokenClaims: p.oidc.idTokenClaims,
|
||||
};
|
||||
}
|
||||
|
||||
return session;
|
||||
};
|
||||
|
||||
// Read the atomic blob. Returns undefined when absent, unparseable, or missing
|
||||
// any of the four required credential fields — callers then fall back to the
|
||||
// legacy keys.
|
||||
const readSessionBlob = (): PersistedSession | undefined => {
|
||||
const raw = localStorage.getItem(SESSION_BLOB_KEY);
|
||||
if (!raw) return undefined;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
// Corrupt JSON — treat as absent and let the legacy path take over.
|
||||
return undefined;
|
||||
}
|
||||
if (!parsed || typeof parsed !== 'object') return undefined;
|
||||
const p = parsed as Partial<PersistedSession>;
|
||||
if (
|
||||
typeof p.accessToken !== 'string' ||
|
||||
typeof p.deviceId !== 'string' ||
|
||||
typeof p.userId !== 'string' ||
|
||||
typeof p.baseUrl !== 'string'
|
||||
) {
|
||||
// Partial/corrupt blob — fall back to legacy assembly.
|
||||
return undefined;
|
||||
}
|
||||
return p as PersistedSession;
|
||||
};
|
||||
|
||||
// Assemble a session from the legacy per-field keys, or undefined when the four
|
||||
// required keys are not all present. Used for transparent migration from builds
|
||||
// that predate the atomic blob.
|
||||
const readLegacyKeys = (): PersistedSession | undefined => {
|
||||
const baseUrl = localStorage.getItem(LEGACY_KEYS.baseUrl);
|
||||
const userId = localStorage.getItem(LEGACY_KEYS.userId);
|
||||
const deviceId = localStorage.getItem(LEGACY_KEYS.deviceId);
|
||||
const accessToken = localStorage.getItem(LEGACY_KEYS.accessToken);
|
||||
|
||||
if (!(baseUrl && userId && deviceId && accessToken)) return undefined;
|
||||
|
||||
const persisted: PersistedSession = { accessToken, deviceId, userId, baseUrl };
|
||||
|
||||
const refreshToken = localStorage.getItem(OIDC_KEYS.refreshToken);
|
||||
if (refreshToken) persisted.refreshToken = refreshToken;
|
||||
|
||||
const expiresAtRaw = localStorage.getItem(OIDC_KEYS.expiresAt);
|
||||
if (expiresAtRaw) {
|
||||
const expiresAt = Number(expiresAtRaw);
|
||||
if (Number.isFinite(expiresAt)) persisted.expiresAt = expiresAt;
|
||||
}
|
||||
|
||||
const issuer = localStorage.getItem(OIDC_KEYS.issuer);
|
||||
const clientId = localStorage.getItem(OIDC_KEYS.clientId);
|
||||
const redirectUri = localStorage.getItem(OIDC_KEYS.redirectUri);
|
||||
if (issuer && clientId && redirectUri) {
|
||||
let idTokenClaims: Record<string, unknown> | undefined;
|
||||
const claimsRaw = localStorage.getItem(OIDC_KEYS.idTokenClaims);
|
||||
if (claimsRaw) {
|
||||
try {
|
||||
idTokenClaims = JSON.parse(claimsRaw);
|
||||
} catch {
|
||||
/* corrupt claims — ignore, the refresher will re-validate on use */
|
||||
}
|
||||
}
|
||||
persisted.oidc = { issuer, clientId, redirectUri, idTokenClaims };
|
||||
}
|
||||
|
||||
return persisted;
|
||||
};
|
||||
|
||||
// Write the legacy per-field keys (dual-write half). Mirrors the original
|
||||
// setFallbackSession body so a rollback to an older build keeps working.
|
||||
const writeLegacyKeys = (p: PersistedSession): void => {
|
||||
localStorage.setItem(LEGACY_KEYS.accessToken, p.accessToken);
|
||||
localStorage.setItem(LEGACY_KEYS.deviceId, p.deviceId);
|
||||
localStorage.setItem(LEGACY_KEYS.userId, p.userId);
|
||||
localStorage.setItem(LEGACY_KEYS.baseUrl, p.baseUrl);
|
||||
|
||||
// OIDC fields — written only when present; otherwise cleared so a password
|
||||
// session never carries stale OIDC state.
|
||||
if (p.refreshToken) localStorage.setItem(OIDC_KEYS.refreshToken, p.refreshToken);
|
||||
else localStorage.removeItem(OIDC_KEYS.refreshToken);
|
||||
|
||||
if (typeof p.expiresAt === 'number')
|
||||
localStorage.setItem(OIDC_KEYS.expiresAt, String(p.expiresAt));
|
||||
else localStorage.removeItem(OIDC_KEYS.expiresAt);
|
||||
|
||||
if (p.oidc) {
|
||||
localStorage.setItem(OIDC_KEYS.issuer, p.oidc.issuer);
|
||||
localStorage.setItem(OIDC_KEYS.clientId, p.oidc.clientId);
|
||||
localStorage.setItem(OIDC_KEYS.redirectUri, p.oidc.redirectUri);
|
||||
if (p.oidc.idTokenClaims) {
|
||||
localStorage.setItem(OIDC_KEYS.idTokenClaims, JSON.stringify(p.oidc.idTokenClaims));
|
||||
} else localStorage.removeItem(OIDC_KEYS.idTokenClaims);
|
||||
} else {
|
||||
localStorage.removeItem(OIDC_KEYS.issuer);
|
||||
localStorage.removeItem(OIDC_KEYS.clientId);
|
||||
localStorage.removeItem(OIDC_KEYS.redirectUri);
|
||||
localStorage.removeItem(OIDC_KEYS.idTokenClaims);
|
||||
}
|
||||
};
|
||||
|
||||
export type FallbackSessionExtra = {
|
||||
refreshToken?: string;
|
||||
expiresInMs?: number;
|
||||
@@ -56,6 +233,10 @@ export type SessionStoreName = {
|
||||
// crypto: 'crypto-store',
|
||||
// } as const;
|
||||
|
||||
// Persist the session. Writes the atomic blob FIRST (so the consistent,
|
||||
// never-torn copy is established before the multi-key legacy write), then
|
||||
// dual-writes the legacy keys for rollback safety. Signature is unchanged —
|
||||
// callers (login/register/OIDC callback/token refresher) are untouched.
|
||||
export function setFallbackSession(
|
||||
accessToken: string,
|
||||
deviceId: string,
|
||||
@@ -63,92 +244,65 @@ export function setFallbackSession(
|
||||
baseUrl: string,
|
||||
extra?: FallbackSessionExtra,
|
||||
) {
|
||||
localStorage.setItem('cinny_access_token', accessToken);
|
||||
localStorage.setItem('cinny_device_id', deviceId);
|
||||
localStorage.setItem('cinny_user_id', userId);
|
||||
localStorage.setItem('cinny_hs_base_url', baseUrl);
|
||||
|
||||
// OIDC fields — written only when present; otherwise cleared so a password
|
||||
// session never carries stale OIDC state.
|
||||
if (extra?.refreshToken) localStorage.setItem(OIDC_KEYS.refreshToken, extra.refreshToken);
|
||||
else localStorage.removeItem(OIDC_KEYS.refreshToken);
|
||||
|
||||
if (typeof extra?.expiresInMs === 'number') {
|
||||
// Store ABSOLUTE expiry to avoid drift across reloads.
|
||||
localStorage.setItem(OIDC_KEYS.expiresAt, String(Date.now() + extra.expiresInMs));
|
||||
} else localStorage.removeItem(OIDC_KEYS.expiresAt);
|
||||
|
||||
if (extra?.oidc) {
|
||||
localStorage.setItem(OIDC_KEYS.issuer, extra.oidc.issuer);
|
||||
localStorage.setItem(OIDC_KEYS.clientId, extra.oidc.clientId);
|
||||
localStorage.setItem(OIDC_KEYS.redirectUri, extra.oidc.redirectUri);
|
||||
if (extra.oidc.idTokenClaims) {
|
||||
localStorage.setItem(OIDC_KEYS.idTokenClaims, JSON.stringify(extra.oidc.idTokenClaims));
|
||||
} else localStorage.removeItem(OIDC_KEYS.idTokenClaims);
|
||||
} else {
|
||||
localStorage.removeItem(OIDC_KEYS.issuer);
|
||||
localStorage.removeItem(OIDC_KEYS.clientId);
|
||||
localStorage.removeItem(OIDC_KEYS.redirectUri);
|
||||
localStorage.removeItem(OIDC_KEYS.idTokenClaims);
|
||||
}
|
||||
const persisted = buildPersisted(accessToken, deviceId, userId, baseUrl, extra);
|
||||
// ONE setItem — the blob can never be observed half-written.
|
||||
localStorage.setItem(SESSION_BLOB_KEY, JSON.stringify(persisted));
|
||||
// Dual-write the legacy keys (removal of this half is a future release).
|
||||
writeLegacyKeys(persisted);
|
||||
}
|
||||
|
||||
// Clear BOTH the atomic blob and every legacy key so no reader (blob-preferring
|
||||
// or legacy-fallback) can resurrect a logged-out session.
|
||||
export const removeFallbackSession = () => {
|
||||
localStorage.removeItem('cinny_hs_base_url');
|
||||
localStorage.removeItem('cinny_user_id');
|
||||
localStorage.removeItem('cinny_device_id');
|
||||
localStorage.removeItem('cinny_access_token');
|
||||
localStorage.removeItem(SESSION_BLOB_KEY);
|
||||
Object.values(LEGACY_KEYS).forEach((key) => localStorage.removeItem(key));
|
||||
Object.values(OIDC_KEYS).forEach((key) => localStorage.removeItem(key));
|
||||
};
|
||||
|
||||
// Read the session, preferring the atomic blob. If the blob is absent or
|
||||
// corrupt/partial we transparently assemble from the legacy keys (migration);
|
||||
// the next setFallbackSession then persists the blob. When both exist the blob
|
||||
// wins by construction.
|
||||
export const getFallbackSession = (): Session | undefined => {
|
||||
const baseUrl = localStorage.getItem('cinny_hs_base_url');
|
||||
const userId = localStorage.getItem('cinny_user_id');
|
||||
const deviceId = localStorage.getItem('cinny_device_id');
|
||||
const accessToken = localStorage.getItem('cinny_access_token');
|
||||
|
||||
if (baseUrl && userId && deviceId && accessToken) {
|
||||
const session: Session = {
|
||||
baseUrl,
|
||||
userId,
|
||||
deviceId,
|
||||
accessToken,
|
||||
fallbackSdkStores: true,
|
||||
};
|
||||
|
||||
const refreshToken = localStorage.getItem(OIDC_KEYS.refreshToken);
|
||||
if (refreshToken) session.refreshToken = refreshToken;
|
||||
|
||||
const expiresAtRaw = localStorage.getItem(OIDC_KEYS.expiresAt);
|
||||
if (expiresAtRaw) {
|
||||
const expiresAt = Number(expiresAtRaw);
|
||||
// Expose the REMAINING lifetime (clamped at 0); the SDK refreshes on 401.
|
||||
if (Number.isFinite(expiresAt)) session.expiresInMs = Math.max(0, expiresAt - Date.now());
|
||||
}
|
||||
|
||||
const issuer = localStorage.getItem(OIDC_KEYS.issuer);
|
||||
const clientId = localStorage.getItem(OIDC_KEYS.clientId);
|
||||
const redirectUri = localStorage.getItem(OIDC_KEYS.redirectUri);
|
||||
if (issuer && clientId && redirectUri) {
|
||||
let idTokenClaims: Record<string, unknown> | undefined;
|
||||
const claimsRaw = localStorage.getItem(OIDC_KEYS.idTokenClaims);
|
||||
if (claimsRaw) {
|
||||
try {
|
||||
idTokenClaims = JSON.parse(claimsRaw);
|
||||
} catch {
|
||||
/* corrupt claims — ignore, the refresher will re-validate on use */
|
||||
}
|
||||
}
|
||||
session.oidc = { issuer, clientId, redirectUri, idTokenClaims };
|
||||
}
|
||||
|
||||
return session;
|
||||
}
|
||||
|
||||
return undefined;
|
||||
const persisted = readSessionBlob() ?? readLegacyKeys();
|
||||
if (!persisted) return undefined;
|
||||
return sessionFromPersisted(persisted);
|
||||
};
|
||||
/**
|
||||
* End of migration code for old session
|
||||
*/
|
||||
|
||||
// Session keys whose cross-tab change indicates a login/logout/token-rotation
|
||||
// in another tab. localStorage.clear() dispatches a storage event with a null
|
||||
// key, which we also treat as a session change.
|
||||
const SESSION_STORAGE_KEYS = new Set<string>([
|
||||
SESSION_BLOB_KEY,
|
||||
...Object.values(LEGACY_KEYS),
|
||||
...Object.values(OIDC_KEYS),
|
||||
]);
|
||||
|
||||
/**
|
||||
* Subscribe to session changes made in OTHER tabs/windows. The browser only
|
||||
* dispatches `storage` events to tabs that did NOT perform the write, so this
|
||||
* is inherently guarded against reacting to our own same-tab writes — no
|
||||
* echo-suppression needed. The callback receives the freshly-read session, or
|
||||
* `null` when the session was removed (logout in another tab, or a full
|
||||
* localStorage.clear()). Returns an unsubscribe function.
|
||||
*/
|
||||
export const subscribeSessionChanges = (
|
||||
callback: (session: Session | null) => void,
|
||||
): (() => void) => {
|
||||
const handleStorage = (evt: StorageEvent) => {
|
||||
// A null key means localStorage.clear(); otherwise only react to our keys.
|
||||
if (evt.key !== null && !SESSION_STORAGE_KEYS.has(evt.key)) return;
|
||||
callback(getFallbackSession() ?? null);
|
||||
};
|
||||
window.addEventListener('storage', handleStorage);
|
||||
return () => {
|
||||
window.removeEventListener('storage', handleStorage);
|
||||
};
|
||||
};
|
||||
|
||||
// export const getSessionStoreName = (session: Session): SessionStoreName => {
|
||||
// if (session.fallbackSdkStores) {
|
||||
// return FALLBACK_STORE_NAME;
|
||||
|
||||
@@ -9,6 +9,8 @@ import {
|
||||
contrastingText,
|
||||
varNameFromToken,
|
||||
derivePrimaryPalette,
|
||||
deriveAccentExtras,
|
||||
buildAccentCss,
|
||||
} from './accentColor';
|
||||
|
||||
test('hexToRgb parses 6-digit hex (with/without #, trimmed)', () => {
|
||||
@@ -66,3 +68,22 @@ test('derivePrimaryPalette produces the full Primary token set', () => {
|
||||
assert.match(palette.MainHover, /^#[0-9a-f]{6}$/);
|
||||
assert.match(palette.MainActive, /^#[0-9a-f]{6}$/);
|
||||
});
|
||||
|
||||
test('deriveAccentExtras derives focus ring, link and selection from one base', () => {
|
||||
const base = { r: 255, g: 136, b: 0 };
|
||||
const extras = deriveAccentExtras(base);
|
||||
// focus ring keeps the translucent character in the accent hue
|
||||
assert.equal(extras.focusRing, 'rgba(255, 136, 0, 0.5)');
|
||||
// link + selection background are the solid base hex
|
||||
assert.equal(extras.link, '#ff8800');
|
||||
assert.equal(extras.selectionBg, '#ff8800');
|
||||
// selection text is WCAG-aware contrasting text over the base
|
||||
assert.equal(extras.selectionText, contrastingText(base));
|
||||
});
|
||||
|
||||
test('buildAccentCss emits selection rules using the derived palette', () => {
|
||||
const base = { r: 0, g: 0, b: 0 };
|
||||
const css = buildAccentCss(base);
|
||||
assert.match(css, /::selection\{background:#000000;color:#fff;\}/);
|
||||
assert.match(css, /::-moz-selection\{background:#000000;color:#fff;\}/);
|
||||
});
|
||||
|
||||
@@ -74,6 +74,45 @@ const PRIMARY_TOKENS: Record<string, string> = {
|
||||
OnContainer: color.Primary.OnContainer,
|
||||
};
|
||||
|
||||
// The neutral focus-ring token folds uses for the outline on inputs, buttons,
|
||||
// switches, checkboxes and radios. Its default is a semi-transparent grey/black,
|
||||
// so tinting it in the accent hue themes every focus ring without touching the
|
||||
// neutral Secondary family (see below). We keep the same translucent character
|
||||
// so it reads as a ring rather than a fill.
|
||||
const FOCUS_RING_TOKEN = color.Other.FocusRing;
|
||||
|
||||
// `--tc-link` is the global anchor color (index.css `a { color: var(--tc-link) }`);
|
||||
// overriding it themes plain links inside messages, room topics and URL previews.
|
||||
const LINK_VAR = '--tc-link';
|
||||
|
||||
// Injected stylesheet id — carries rules that cannot be expressed as a single
|
||||
// CSS variable (currently text ::selection).
|
||||
const ACCENT_STYLE_ID = 'lotus-accent-style';
|
||||
|
||||
export type AccentExtras = {
|
||||
focusRing: string;
|
||||
link: string;
|
||||
selectionBg: string;
|
||||
selectionText: string;
|
||||
};
|
||||
|
||||
// Derive the extra (non-Primary) accent values from the single base color, using
|
||||
// the same helpers as the Primary palette so everything stays in one hue.
|
||||
export const deriveAccentExtras = (base: Rgb): AccentExtras => ({
|
||||
focusRing: rgba(base, 0.5),
|
||||
link: rgbToHex(base),
|
||||
selectionBg: rgbToHex(base),
|
||||
selectionText: contrastingText(base),
|
||||
});
|
||||
|
||||
// Build the injected stylesheet body. Selection uses a solid accent fill with
|
||||
// WCAG-aware contrasting text so highlighted text stays readable.
|
||||
export const buildAccentCss = (base: Rgb): string => {
|
||||
const { selectionBg, selectionText } = deriveAccentExtras(base);
|
||||
const selection = `background:${selectionBg};color:${selectionText};`;
|
||||
return `::selection{${selection}}::-moz-selection{${selection}}`;
|
||||
};
|
||||
|
||||
// Derive the 10 Primary sub-token values from a single chosen base color.
|
||||
export const derivePrimaryPalette = (base: Rgb): Record<string, string> => {
|
||||
const baseHex = rgbToHex(base);
|
||||
@@ -96,22 +135,46 @@ export const derivePrimaryPalette = (base: Rgb): Record<string, string> => {
|
||||
};
|
||||
|
||||
// Apply a custom accent color by overriding the folds Primary CSS variables on
|
||||
// `document.body`. Returns true when applied, false when the input is invalid.
|
||||
// `document.body`, tinting the focus-ring and link vars, and injecting a small
|
||||
// stylesheet for text selection. Returns true when applied, false when the input
|
||||
// is invalid.
|
||||
export const applyCustomAccent = (hex: string): boolean => {
|
||||
const base = hexToRgb(hex);
|
||||
if (!base) return false;
|
||||
|
||||
const palette = derivePrimaryPalette(base);
|
||||
Object.entries(PRIMARY_TOKENS).forEach(([key, token]) => {
|
||||
const varName = varNameFromToken(token);
|
||||
if (varName) document.body.style.setProperty(varName, palette[key]);
|
||||
});
|
||||
|
||||
const extras = deriveAccentExtras(base);
|
||||
const focusRingVar = varNameFromToken(FOCUS_RING_TOKEN);
|
||||
if (focusRingVar) document.body.style.setProperty(focusRingVar, extras.focusRing);
|
||||
document.body.style.setProperty(LINK_VAR, extras.link);
|
||||
|
||||
let styleEl = document.getElementById(ACCENT_STYLE_ID) as HTMLStyleElement | null;
|
||||
if (!styleEl) {
|
||||
styleEl = document.createElement('style');
|
||||
styleEl.id = ACCENT_STYLE_ID;
|
||||
document.head.appendChild(styleEl);
|
||||
}
|
||||
styleEl.textContent = buildAccentCss(base);
|
||||
|
||||
return true;
|
||||
};
|
||||
|
||||
// Remove all custom accent overrides, reverting to the active theme's defaults.
|
||||
// Idempotent — safe to call even when nothing was applied.
|
||||
export const removeCustomAccent = (): void => {
|
||||
Object.values(PRIMARY_TOKENS).forEach((token) => {
|
||||
const varName = varNameFromToken(token);
|
||||
if (varName) document.body.style.removeProperty(varName);
|
||||
});
|
||||
|
||||
const focusRingVar = varNameFromToken(FOCUS_RING_TOKEN);
|
||||
if (focusRingVar) document.body.style.removeProperty(focusRingVar);
|
||||
document.body.style.removeProperty(LINK_VAR);
|
||||
|
||||
document.getElementById(ACCENT_STYLE_ID)?.remove();
|
||||
};
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
import type { MatrixClient } from 'matrix-js-sdk';
|
||||
import pkg from '../../../package.json';
|
||||
|
||||
// Lotus E2EE investigation kit — capture-only console diagnostics.
|
||||
//
|
||||
// Installs pass-through wrappers around `console.warn` / `console.error` that
|
||||
// ring-buffer any log line matching the KE-1..KE-4 bug-cluster signatures
|
||||
// (see LOTUS_E2EE_INVESTIGATION.md). It NEVER swallows a log call — the
|
||||
// original console method is always invoked — and it performs NO network I/O.
|
||||
// The report metadata is limited to SDK version / device id / user id / sync
|
||||
// state; the captured log lines themselves are intentional evidence and may
|
||||
// contain event ids or matrix ids exactly as the SDK logged them.
|
||||
|
||||
export type CryptoDiagLevel = 'warn' | 'error';
|
||||
|
||||
export type CryptoDiagEntry = {
|
||||
/** ISO-8601 UTC timestamp of when the line was captured. */
|
||||
ts: string;
|
||||
level: CryptoDiagLevel;
|
||||
/** Which KE bucket the signature belongs to, e.g. `KE-1`. */
|
||||
ke: string;
|
||||
/** Human-readable label of the matched signature. */
|
||||
signature: string;
|
||||
/** The serialized console line (best-effort). */
|
||||
message: string;
|
||||
};
|
||||
|
||||
type Signature = {
|
||||
ke: string;
|
||||
label: string;
|
||||
re: RegExp;
|
||||
};
|
||||
|
||||
// Ordered most-specific-first so the recorded label is the tightest match.
|
||||
const SIGNATURES: Signature[] = [
|
||||
{ ke: 'KE-1', label: 'already exists', re: /already exists/i },
|
||||
{ ke: 'KE-2', label: 'missing key at index', re: /missing key at index/i },
|
||||
{
|
||||
ke: 'KE-2',
|
||||
label: 'io.element.call.encryption_keys',
|
||||
re: /io\.element\.call\.encryption_keys/,
|
||||
},
|
||||
{ ke: 'KE-2', label: 'MissingKey', re: /MissingKey/ },
|
||||
{ ke: 'KE-3', label: 'DecryptionError', re: /DecryptionError/ },
|
||||
{ ke: 'KE-4', label: 'update_delayed_event', re: /update_delayed_event/ },
|
||||
{ ke: 'KE-4', label: 'delayed event', re: /delayed event/i },
|
||||
];
|
||||
|
||||
const MAX_ENTRIES = 200;
|
||||
|
||||
const entries: CryptoDiagEntry[] = [];
|
||||
|
||||
let installed = false;
|
||||
let originalWarn: ((...data: unknown[]) => void) | undefined;
|
||||
let originalError: ((...data: unknown[]) => void) | undefined;
|
||||
|
||||
const stringifyArg = (arg: unknown): string => {
|
||||
if (typeof arg === 'string') return arg;
|
||||
if (arg instanceof Error) return `${arg.name}: ${arg.message}`;
|
||||
try {
|
||||
return JSON.stringify(arg);
|
||||
} catch {
|
||||
return String(arg);
|
||||
}
|
||||
};
|
||||
|
||||
const capture = (level: CryptoDiagLevel, args: unknown[]): void => {
|
||||
const message = args.map(stringifyArg).join(' ');
|
||||
const sig = SIGNATURES.find((s) => s.re.test(message));
|
||||
if (!sig) return;
|
||||
|
||||
entries.push({
|
||||
ts: new Date().toISOString(),
|
||||
level,
|
||||
ke: sig.ke,
|
||||
signature: sig.label,
|
||||
message,
|
||||
});
|
||||
// Ring-buffer: keep only the most recent MAX_ENTRIES.
|
||||
while (entries.length > MAX_ENTRIES) {
|
||||
entries.shift();
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Install the capture-only console wrappers. Idempotent — calling it more than
|
||||
* once is a no-op. Safe to call as early as possible during app boot.
|
||||
*/
|
||||
export const installCryptoDiagLog = (): void => {
|
||||
if (installed) return;
|
||||
installed = true;
|
||||
|
||||
originalWarn = console.warn.bind(console);
|
||||
originalError = console.error.bind(console);
|
||||
|
||||
console.warn = (...args: unknown[]): void => {
|
||||
capture('warn', args);
|
||||
originalWarn?.(...args);
|
||||
};
|
||||
console.error = (...args: unknown[]): void => {
|
||||
capture('error', args);
|
||||
originalError?.(...args);
|
||||
};
|
||||
};
|
||||
|
||||
/** A snapshot copy of the current capture buffer (most-recent-last). */
|
||||
export const getCryptoDiagEntries = (): CryptoDiagEntry[] => entries.slice();
|
||||
|
||||
const readSdkVersion = (mx?: MatrixClient): string => {
|
||||
// Prefer the value the running client reports; fall back to the declared pin.
|
||||
const declared = (pkg.dependencies as Record<string, string> | undefined)?.['matrix-js-sdk'];
|
||||
const clientVersion = (mx as unknown as { getSdkVersion?: () => string } | undefined)
|
||||
?.getSdkVersion;
|
||||
if (typeof clientVersion === 'function') {
|
||||
try {
|
||||
return clientVersion.call(mx) || declared || 'unknown';
|
||||
} catch {
|
||||
// fall through to the declared pin
|
||||
}
|
||||
}
|
||||
return declared ?? 'unknown';
|
||||
};
|
||||
|
||||
/**
|
||||
* Build a self-contained JSON diagnostic report string. Contains only the SDK
|
||||
* version, device id, user id, sync state, crypto readiness, and the captured
|
||||
* KE signature buffer — no message content, tokens, or other PII.
|
||||
*/
|
||||
export const buildCryptoDiagReport = (mx?: MatrixClient): string => {
|
||||
const buffer = getCryptoDiagEntries();
|
||||
const countsByKe: Record<string, number> = {};
|
||||
buffer.forEach((entry) => {
|
||||
countsByKe[entry.ke] = (countsByKe[entry.ke] ?? 0) + 1;
|
||||
});
|
||||
|
||||
const report = {
|
||||
kind: 'lotus-crypto-diag',
|
||||
generatedAt: new Date().toISOString(),
|
||||
sdkVersion: readSdkVersion(mx),
|
||||
deviceId: mx?.getDeviceId() ?? null,
|
||||
userId: mx?.getUserId() ?? null,
|
||||
syncState: mx?.getSyncState() ?? null,
|
||||
cryptoReady: Boolean(mx?.getCrypto()),
|
||||
entryCount: buffer.length,
|
||||
maxEntries: MAX_ENTRIES,
|
||||
countsByKe,
|
||||
entries: buffer,
|
||||
};
|
||||
|
||||
return JSON.stringify(report, null, 2);
|
||||
};
|
||||
@@ -0,0 +1,83 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { splitMathSegments } from './mathParse';
|
||||
|
||||
test('plain text with no dollars is a single text segment', () => {
|
||||
assert.deepEqual(splitMathSegments('hello world'), [{ type: 'text', value: 'hello world' }]);
|
||||
});
|
||||
|
||||
test('empty string yields no segments', () => {
|
||||
assert.deepEqual(splitMathSegments(''), []);
|
||||
});
|
||||
|
||||
test('inline $…$ is extracted between surrounding text', () => {
|
||||
assert.deepEqual(splitMathSegments('a $x^2$ b'), [
|
||||
{ type: 'text', value: 'a ' },
|
||||
{ type: 'inline', value: 'x^2' },
|
||||
{ type: 'text', value: ' b' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('block $$…$$ is extracted', () => {
|
||||
assert.deepEqual(splitMathSegments('$$block$$'), [{ type: 'block', value: 'block' }]);
|
||||
});
|
||||
|
||||
test('block math may span newlines', () => {
|
||||
assert.deepEqual(splitMathSegments('$$\na=b\n$$'), [{ type: 'block', value: '\na=b\n' }]);
|
||||
});
|
||||
|
||||
test('currency "$5 and $10" is NOT treated as math', () => {
|
||||
assert.deepEqual(splitMathSegments('$5 and $10'), [{ type: 'text', value: '$5 and $10' }]);
|
||||
});
|
||||
|
||||
test('escaped \\$ never opens or closes math', () => {
|
||||
assert.deepEqual(splitMathSegments('cost \\$5 today'), [
|
||||
{ type: 'text', value: 'cost $5 today' },
|
||||
]);
|
||||
assert.deepEqual(splitMathSegments('\\$x\\$'), [{ type: 'text', value: '$x$' }]);
|
||||
});
|
||||
|
||||
test('unbalanced single $ stays as text', () => {
|
||||
assert.deepEqual(splitMathSegments('price is $ here'), [
|
||||
{ type: 'text', value: 'price is $ here' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('unbalanced $$ stays as text', () => {
|
||||
assert.deepEqual(splitMathSegments('$$x'), [{ type: 'text', value: '$$x' }]);
|
||||
});
|
||||
|
||||
test('inline requires non-space adjacency on both delimiters', () => {
|
||||
// Space right after opening $ -> not math.
|
||||
assert.deepEqual(splitMathSegments('$ x$'), [{ type: 'text', value: '$ x$' }]);
|
||||
// Space right before closing $ -> not math.
|
||||
assert.deepEqual(splitMathSegments('$x $'), [{ type: 'text', value: '$x $' }]);
|
||||
});
|
||||
|
||||
test('multiple inline spans on one line', () => {
|
||||
assert.deepEqual(splitMathSegments('$a$ and $b$'), [
|
||||
{ type: 'inline', value: 'a' },
|
||||
{ type: 'text', value: ' and ' },
|
||||
{ type: 'inline', value: 'b' },
|
||||
]);
|
||||
});
|
||||
|
||||
test('escaped dollar inside inline math is preserved in LaTeX', () => {
|
||||
assert.deepEqual(splitMathSegments('$a\\$b$'), [{ type: 'inline', value: 'a\\$b' }]);
|
||||
});
|
||||
|
||||
test('closing $ followed by a digit is skipped (currency guard) then recovers', () => {
|
||||
// The first candidate closer is followed by `2` so it is skipped; the later
|
||||
// `$` closes the span.
|
||||
assert.deepEqual(splitMathSegments('$x$2 + y$'), [{ type: 'inline', value: 'x$2 + y' }]);
|
||||
});
|
||||
|
||||
test('block and inline mixed with text', () => {
|
||||
assert.deepEqual(splitMathSegments('see $$E=mc^2$$ and $a$ ok'), [
|
||||
{ type: 'text', value: 'see ' },
|
||||
{ type: 'block', value: 'E=mc^2' },
|
||||
{ type: 'text', value: ' and ' },
|
||||
{ type: 'inline', value: 'a' },
|
||||
{ type: 'text', value: ' ok' },
|
||||
]);
|
||||
});
|
||||
@@ -0,0 +1,136 @@
|
||||
export type MathSegmentType = 'text' | 'inline' | 'block';
|
||||
|
||||
export type MathSegment = {
|
||||
type: MathSegmentType;
|
||||
/**
|
||||
* For `text` segments this is the literal text. For `inline`/`block` segments
|
||||
* this is the LaTeX source WITHOUT its surrounding `$`/`$$` delimiters.
|
||||
*/
|
||||
value: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* Attempt to match an inline `$…$` span starting at `start` (the index of the
|
||||
* opening `$`).
|
||||
*
|
||||
* Conservative rules (chosen to keep false positives low for prose that merely
|
||||
* mentions currency, e.g. `$5 and $10`):
|
||||
* - The char immediately AFTER the opening `$` must exist, be non-space and not
|
||||
* another `$` (a lone `$` before whitespace, or `$$`, never opens inline math).
|
||||
* - The char immediately BEFORE the closing `$` must be non-space (so `x $` is
|
||||
* not a valid close; we keep scanning for a better `$`).
|
||||
* - The char immediately AFTER the closing `$` must not be a digit (so
|
||||
* `$5 and $10` reads as currency, never math).
|
||||
* - A backslash escapes the following char inside the span, so `\$` is not
|
||||
* treated as a delimiter and stays part of the LaTeX.
|
||||
* - Inline math may not span a newline.
|
||||
* - The LaTeX content must be non-empty.
|
||||
*/
|
||||
const matchInline = (text: string, start: number): { value: string; end: number } | null => {
|
||||
const nextChar = text[start + 1];
|
||||
if (nextChar === undefined || /\s/.test(nextChar) || nextChar === '$') return null;
|
||||
|
||||
let j = start + 1;
|
||||
while (j < text.length) {
|
||||
const c = text[j];
|
||||
if (c === '\\') {
|
||||
// Skip the escaped char (covers `\$` inside the span).
|
||||
j += 2;
|
||||
continue;
|
||||
}
|
||||
if (c === '\n') return null;
|
||||
if (c === '$') {
|
||||
const prev = text[j - 1];
|
||||
// Closing `$` must hug non-space; otherwise this `$` cannot close, keep scanning.
|
||||
if (prev !== undefined && /\s/.test(prev)) {
|
||||
j += 1;
|
||||
continue;
|
||||
}
|
||||
const after = text[j + 1];
|
||||
// A `$` directly followed by a digit is treated as currency, not a closer.
|
||||
if (after !== undefined && /\d/.test(after)) {
|
||||
j += 1;
|
||||
continue;
|
||||
}
|
||||
const value = text.slice(start + 1, j);
|
||||
if (value.length === 0) return null;
|
||||
return { value, end: j + 1 };
|
||||
}
|
||||
j += 1;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Split a plain-text string into text/inline-math/block-math segments.
|
||||
*
|
||||
* Delimiter rules:
|
||||
* - `$$…$$` (possibly multi-line) is block math; the first following `$$` closes it.
|
||||
* - `$…$` is inline math, subject to the conservative adjacency rules in
|
||||
* {@link matchInline}.
|
||||
* - `\$` is an escaped literal dollar: it never acts as a delimiter and is
|
||||
* emitted as a plain `$` in the surrounding text.
|
||||
* - Any `$`/`$$` run that cannot be balanced is left verbatim as text.
|
||||
*
|
||||
* This is a PURE function used by the HTML parser to render math with KaTeX. It
|
||||
* must never be applied to text inside `<pre>`/`<code>` (the caller guards that).
|
||||
*/
|
||||
export const splitMathSegments = (text: string): MathSegment[] => {
|
||||
const segments: MathSegment[] = [];
|
||||
let buffer = '';
|
||||
let i = 0;
|
||||
|
||||
const flushText = () => {
|
||||
if (buffer.length > 0) {
|
||||
segments.push({ type: 'text', value: buffer });
|
||||
buffer = '';
|
||||
}
|
||||
};
|
||||
|
||||
while (i < text.length) {
|
||||
// Escaped dollar: consume `\$` and emit a literal `$` as text.
|
||||
if (text[i] === '\\' && text[i + 1] === '$') {
|
||||
buffer += '$';
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Block math `$$…$$`.
|
||||
if (text.startsWith('$$', i)) {
|
||||
const close = text.indexOf('$$', i + 2);
|
||||
if (close !== -1) {
|
||||
const value = text.slice(i + 2, close);
|
||||
if (value.trim().length > 0) {
|
||||
flushText();
|
||||
segments.push({ type: 'block', value });
|
||||
i = close + 2;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// Unbalanced/empty `$$` — emit a single `$` and continue scanning.
|
||||
buffer += text[i];
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Inline math `$…$`.
|
||||
if (text[i] === '$') {
|
||||
const match = matchInline(text, i);
|
||||
if (match) {
|
||||
flushText();
|
||||
segments.push({ type: 'inline', value: match.value });
|
||||
i = match.end;
|
||||
continue;
|
||||
}
|
||||
buffer += text[i];
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
buffer += text[i];
|
||||
i += 1;
|
||||
}
|
||||
|
||||
flushText();
|
||||
return segments;
|
||||
};
|
||||
@@ -0,0 +1,130 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
computeCoverage,
|
||||
mergeSearchResults,
|
||||
putRows,
|
||||
queryRoom,
|
||||
getCoverage,
|
||||
saveRoomIndex,
|
||||
clearRoom,
|
||||
clearAll,
|
||||
deleteSearchCacheDatabase,
|
||||
SearchCacheRow,
|
||||
} from './searchCache';
|
||||
|
||||
// --- Pure helpers: mergeSearchResults ---------------------------------------
|
||||
|
||||
type Item = { event: { event_id: string; origin_server_ts?: number } };
|
||||
const item = (eventId: string, ts?: number): Item => ({
|
||||
event: { event_id: eventId, origin_server_ts: ts },
|
||||
});
|
||||
|
||||
test('mergeSearchResults: sorts by origin_server_ts descending', () => {
|
||||
const out = mergeSearchResults([item('$a', 10), item('$b', 30), item('$c', 20)], []);
|
||||
assert.deepEqual(
|
||||
out.map((i) => i.event.event_id),
|
||||
['$b', '$c', '$a'],
|
||||
);
|
||||
});
|
||||
|
||||
test('mergeSearchResults: dedupes by event_id with in-memory winning', () => {
|
||||
const memory = [{ event: { event_id: '$dup', origin_server_ts: 5 }, tag: 'memory' }];
|
||||
const cached = [
|
||||
{ event: { event_id: '$dup', origin_server_ts: 5 }, tag: 'cached' },
|
||||
{ event: { event_id: '$only', origin_server_ts: 9 }, tag: 'cached' },
|
||||
];
|
||||
const out = mergeSearchResults(memory, cached);
|
||||
assert.equal(out.length, 2);
|
||||
const dup = out.find((i) => i.event.event_id === '$dup');
|
||||
assert.equal(dup?.tag, 'memory');
|
||||
});
|
||||
|
||||
test('mergeSearchResults: cached-only hits are included', () => {
|
||||
const out = mergeSearchResults<Item>([], [item('$c1', 1), item('$c2', 2)]);
|
||||
assert.equal(out.length, 2);
|
||||
});
|
||||
|
||||
test('mergeSearchResults: missing ts sorts as 0 (last)', () => {
|
||||
const out = mergeSearchResults([item('$noTs'), item('$withTs', 100)], []);
|
||||
assert.deepEqual(
|
||||
out.map((i) => i.event.event_id),
|
||||
['$withTs', '$noTs'],
|
||||
);
|
||||
});
|
||||
|
||||
// --- Pure helpers: computeCoverage ------------------------------------------
|
||||
|
||||
const row = (ts: number): Pick<SearchCacheRow, 'ts'> => ({ ts });
|
||||
|
||||
test('computeCoverage: derives oldest/newest from rows', () => {
|
||||
const cov = computeCoverage('!r', [row(30), row(10), row(20)], 3);
|
||||
assert.deepEqual(cov, { roomId: '!r', oldestTs: 10, newestTs: 30, count: 3 });
|
||||
});
|
||||
|
||||
test('computeCoverage: widens the window against previous coverage', () => {
|
||||
const prev = { roomId: '!r', oldestTs: 5, newestTs: 25, count: 2 };
|
||||
const cov = computeCoverage('!r', [row(15), row(40)], 4, prev);
|
||||
assert.equal(cov.oldestTs, 5); // previous oldest kept
|
||||
assert.equal(cov.newestTs, 40); // batch newest wins
|
||||
assert.equal(cov.count, 4); // authoritative count from caller
|
||||
});
|
||||
|
||||
test('computeCoverage: empty rows with no previous yields zeroed window', () => {
|
||||
const cov = computeCoverage('!r', [], 0);
|
||||
assert.deepEqual(cov, { roomId: '!r', oldestTs: 0, newestTs: 0, count: 0 });
|
||||
});
|
||||
|
||||
// --- IDB round-trip: skip when IndexedDB is unavailable (e.g. node --test) ---
|
||||
|
||||
const hasIdb = typeof indexedDB !== 'undefined';
|
||||
|
||||
test('searchCache IDB round-trip', { skip: !hasIdb }, async () => {
|
||||
await clearAll();
|
||||
const rows: SearchCacheRow[] = [
|
||||
{ roomId: '!r1', eventId: '$1', ts: 100, sender: '@a', body: 'hello world' },
|
||||
{
|
||||
roomId: '!r1',
|
||||
eventId: '$2',
|
||||
ts: 200,
|
||||
sender: '@b',
|
||||
body: 'goodbye',
|
||||
formattedBody: '<b>x</b>',
|
||||
},
|
||||
{ roomId: '!r2', eventId: '$3', ts: 300, sender: '@a', body: 'other room' },
|
||||
];
|
||||
await putRows(rows);
|
||||
|
||||
const r1 = await queryRoom('!r1');
|
||||
assert.equal(r1.length, 2);
|
||||
assert.deepEqual(r1.map((x) => x.eventId).sort(), ['$1', '$2']);
|
||||
|
||||
await saveRoomIndex(
|
||||
'!r1',
|
||||
rows.filter((x) => x.roomId === '!r1'),
|
||||
);
|
||||
const cov = await getCoverage('!r1');
|
||||
assert.equal(cov?.count, 2);
|
||||
assert.equal(cov?.oldestTs, 100);
|
||||
assert.equal(cov?.newestTs, 200);
|
||||
|
||||
await clearRoom('!r1');
|
||||
assert.equal((await queryRoom('!r1')).length, 0);
|
||||
assert.equal((await queryRoom('!r2')).length, 1);
|
||||
|
||||
await deleteSearchCacheDatabase();
|
||||
});
|
||||
|
||||
test('resilient helpers never throw when IDB is unavailable', { skip: hasIdb }, async () => {
|
||||
// In this environment IndexedDB is absent; every call must degrade to a
|
||||
// cache-miss rather than throwing.
|
||||
await assert.doesNotReject(
|
||||
putRows([{ roomId: '!r', eventId: '$1', ts: 1, sender: '@a', body: 'x' }]),
|
||||
);
|
||||
assert.deepEqual(await queryRoom('!r'), []);
|
||||
assert.equal(await getCoverage('!r'), null);
|
||||
await assert.doesNotReject(saveRoomIndex('!r', []));
|
||||
await assert.doesNotReject(clearRoom('!r'));
|
||||
await assert.doesNotReject(clearAll());
|
||||
await assert.doesNotReject(deleteSearchCacheDatabase());
|
||||
});
|
||||
@@ -0,0 +1,308 @@
|
||||
/**
|
||||
* P4-8 — persistent encrypted-search cache (raw IndexedDB, no new deps).
|
||||
*
|
||||
* The homeserver cannot search E2EE message content, so encrypted-room search
|
||||
* only ever covers what the client has paginated + decrypted this session. This
|
||||
* module persists a local plaintext index so coverage survives reloads.
|
||||
*
|
||||
* PRIVACY: this stores decrypted plaintext at rest. It is opt-in (default OFF),
|
||||
* clearable, and wiped on logout via `deleteSearchCacheDatabase()`.
|
||||
*
|
||||
* Resilience contract: every entry point swallows IndexedDB errors and behaves
|
||||
* as a cache-miss. Nothing here ever throws to the UI.
|
||||
*/
|
||||
|
||||
const DB_NAME = 'lotus-search-cache';
|
||||
const DB_VERSION = 1;
|
||||
const MESSAGES_STORE = 'messages';
|
||||
const COVERAGE_STORE = 'coverage';
|
||||
const ROOM_TS_INDEX = 'roomTs';
|
||||
|
||||
/** A single cached, decrypted message row. Keyed on `[roomId, eventId]`. */
|
||||
export type SearchCacheRow = {
|
||||
roomId: string;
|
||||
eventId: string;
|
||||
ts: number;
|
||||
sender: string;
|
||||
body: string;
|
||||
formattedBody?: string;
|
||||
pollText?: string;
|
||||
};
|
||||
|
||||
/** Per-room coverage stats for the "X / Y cached" UI counters. */
|
||||
export type SearchCacheCoverage = {
|
||||
roomId: string;
|
||||
oldestTs: number;
|
||||
newestTs: number;
|
||||
count: number;
|
||||
};
|
||||
|
||||
// A key range that matches every `[roomId, *]` entry in a composite-key store
|
||||
// or `[roomId, ts]` index: an empty array sorts after all other key types, so
|
||||
// `[roomId]` .. `[roomId, []]` brackets the whole room partition.
|
||||
const roomRange = (roomId: string): IDBKeyRange => IDBKeyRange.bound([roomId], [roomId, []]);
|
||||
|
||||
let dbPromise: Promise<IDBDatabase | null> | null = null;
|
||||
|
||||
const openDb = (): Promise<IDBDatabase | null> => {
|
||||
if (dbPromise) return dbPromise;
|
||||
dbPromise = new Promise<IDBDatabase | null>((resolve) => {
|
||||
try {
|
||||
if (typeof indexedDB === 'undefined') {
|
||||
resolve(null);
|
||||
return;
|
||||
}
|
||||
const req = indexedDB.open(DB_NAME, DB_VERSION);
|
||||
req.onupgradeneeded = () => {
|
||||
const db = req.result;
|
||||
if (!db.objectStoreNames.contains(MESSAGES_STORE)) {
|
||||
const store = db.createObjectStore(MESSAGES_STORE, {
|
||||
keyPath: ['roomId', 'eventId'],
|
||||
});
|
||||
store.createIndex(ROOM_TS_INDEX, ['roomId', 'ts']);
|
||||
}
|
||||
if (!db.objectStoreNames.contains(COVERAGE_STORE)) {
|
||||
db.createObjectStore(COVERAGE_STORE, { keyPath: 'roomId' });
|
||||
}
|
||||
};
|
||||
req.onsuccess = () => resolve(req.result);
|
||||
req.onerror = () => {
|
||||
dbPromise = null; // allow a later retry
|
||||
resolve(null);
|
||||
};
|
||||
req.onblocked = () => {
|
||||
dbPromise = null;
|
||||
resolve(null);
|
||||
};
|
||||
} catch {
|
||||
dbPromise = null;
|
||||
resolve(null);
|
||||
}
|
||||
});
|
||||
return dbPromise;
|
||||
};
|
||||
|
||||
/** Resolve once a write transaction commits (or reject/abort → caller swallows). */
|
||||
const awaitTx = (tx: IDBTransaction): Promise<void> =>
|
||||
new Promise<void>((resolve, reject) => {
|
||||
tx.oncomplete = () => resolve();
|
||||
tx.onerror = () => reject(tx.error);
|
||||
tx.onabort = () => reject(tx.error);
|
||||
});
|
||||
|
||||
/** Upsert message rows. No-op on empty input or when IDB is unavailable. */
|
||||
export const putRows = async (rows: SearchCacheRow[]): Promise<void> => {
|
||||
if (rows.length === 0) return;
|
||||
const db = await openDb();
|
||||
if (!db) return;
|
||||
try {
|
||||
const tx = db.transaction(MESSAGES_STORE, 'readwrite');
|
||||
const store = tx.objectStore(MESSAGES_STORE);
|
||||
rows.forEach((row) => store.put(row));
|
||||
await awaitTx(tx);
|
||||
} catch {
|
||||
// Cache write failures must never surface to the UI.
|
||||
}
|
||||
};
|
||||
|
||||
/** All cached rows for a room, ordered oldest→newest by the `[roomId, ts]` index. */
|
||||
export const queryRoom = async (roomId: string): Promise<SearchCacheRow[]> => {
|
||||
const db = await openDb();
|
||||
if (!db) return [];
|
||||
try {
|
||||
return await new Promise<SearchCacheRow[]>((resolve, reject) => {
|
||||
const tx = db.transaction(MESSAGES_STORE, 'readonly');
|
||||
const index = tx.objectStore(MESSAGES_STORE).index(ROOM_TS_INDEX);
|
||||
const req = index.getAll(roomRange(roomId));
|
||||
req.onsuccess = () => resolve((req.result as SearchCacheRow[]) ?? []);
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
};
|
||||
|
||||
/** Cursor variant: stream a room's rows through a matcher, collecting hits. */
|
||||
export const searchRoom = async (
|
||||
roomId: string,
|
||||
matcher: (row: SearchCacheRow) => boolean,
|
||||
): Promise<SearchCacheRow[]> => {
|
||||
const db = await openDb();
|
||||
if (!db) return [];
|
||||
try {
|
||||
return await new Promise<SearchCacheRow[]>((resolve, reject) => {
|
||||
const hits: SearchCacheRow[] = [];
|
||||
const tx = db.transaction(MESSAGES_STORE, 'readonly');
|
||||
const index = tx.objectStore(MESSAGES_STORE).index(ROOM_TS_INDEX);
|
||||
const req = index.openCursor(roomRange(roomId));
|
||||
req.onsuccess = () => {
|
||||
const cursor = req.result;
|
||||
if (!cursor) {
|
||||
resolve(hits);
|
||||
return;
|
||||
}
|
||||
const row = cursor.value as SearchCacheRow;
|
||||
if (matcher(row)) hits.push(row);
|
||||
cursor.continue();
|
||||
};
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
};
|
||||
|
||||
/** Number of cached rows for a room. */
|
||||
export const countRoom = async (roomId: string): Promise<number> => {
|
||||
const db = await openDb();
|
||||
if (!db) return 0;
|
||||
try {
|
||||
return await new Promise<number>((resolve, reject) => {
|
||||
const tx = db.transaction(MESSAGES_STORE, 'readonly');
|
||||
const index = tx.objectStore(MESSAGES_STORE).index(ROOM_TS_INDEX);
|
||||
const req = index.count(roomRange(roomId));
|
||||
req.onsuccess = () => resolve(req.result);
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
} catch {
|
||||
return 0;
|
||||
}
|
||||
};
|
||||
|
||||
export const getCoverage = async (roomId: string): Promise<SearchCacheCoverage | null> => {
|
||||
const db = await openDb();
|
||||
if (!db) return null;
|
||||
try {
|
||||
return await new Promise<SearchCacheCoverage | null>((resolve, reject) => {
|
||||
const tx = db.transaction(COVERAGE_STORE, 'readonly');
|
||||
const req = tx.objectStore(COVERAGE_STORE).get(roomId);
|
||||
req.onsuccess = () => resolve((req.result as SearchCacheCoverage) ?? null);
|
||||
req.onerror = () => reject(req.error);
|
||||
});
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
};
|
||||
|
||||
export const putCoverage = async (coverage: SearchCacheCoverage): Promise<void> => {
|
||||
const db = await openDb();
|
||||
if (!db) return;
|
||||
try {
|
||||
const tx = db.transaction(COVERAGE_STORE, 'readwrite');
|
||||
tx.objectStore(COVERAGE_STORE).put(coverage);
|
||||
await awaitTx(tx);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Pure helper: fold a batch of rows into a coverage record, widening the
|
||||
* `oldestTs`/`newestTs` window against any previous coverage. `count` is
|
||||
* supplied by the caller (authoritative store count) so dedup across sessions
|
||||
* is handled correctly. Exported for testing without IDB.
|
||||
*/
|
||||
export const computeCoverage = (
|
||||
roomId: string,
|
||||
rows: ReadonlyArray<Pick<SearchCacheRow, 'ts'>>,
|
||||
count: number,
|
||||
previous?: SearchCacheCoverage | null,
|
||||
): SearchCacheCoverage => {
|
||||
let oldestTs = previous?.oldestTs ?? Number.POSITIVE_INFINITY;
|
||||
let newestTs = previous?.newestTs ?? Number.NEGATIVE_INFINITY;
|
||||
rows.forEach((row) => {
|
||||
if (row.ts < oldestTs) oldestTs = row.ts;
|
||||
if (row.ts > newestTs) newestTs = row.ts;
|
||||
});
|
||||
if (!Number.isFinite(oldestTs)) oldestTs = 0;
|
||||
if (!Number.isFinite(newestTs)) newestTs = 0;
|
||||
return { roomId, oldestTs, newestTs, count };
|
||||
};
|
||||
|
||||
/**
|
||||
* Convenience persist path used by the search hook: upsert a batch of rows for
|
||||
* a room, then recompute + store the room's coverage from the authoritative
|
||||
* store count. Fire-and-forget; never throws.
|
||||
*/
|
||||
export const saveRoomIndex = async (roomId: string, rows: SearchCacheRow[]): Promise<void> => {
|
||||
if (rows.length === 0) return;
|
||||
await putRows(rows);
|
||||
const [count, previous] = await Promise.all([countRoom(roomId), getCoverage(roomId)]);
|
||||
await putCoverage(computeCoverage(roomId, rows, count, previous));
|
||||
};
|
||||
|
||||
/**
|
||||
* Pure helper: merge in-memory result items with cache-derived result items,
|
||||
* deduping by `event.event_id` (in-memory wins), sorted by `origin_server_ts`
|
||||
* descending. Generic over the minimal shape it reads so it is fully testable
|
||||
* without matrix-js-sdk types. Exported for testing.
|
||||
*/
|
||||
export const mergeSearchResults = <
|
||||
T extends { event: { event_id: string; origin_server_ts?: number } },
|
||||
>(
|
||||
memory: ReadonlyArray<T>,
|
||||
cached: ReadonlyArray<T>,
|
||||
): T[] => {
|
||||
const byId = new Map<string, T>();
|
||||
// Seed with cached, then let in-memory overwrite so in-memory always wins.
|
||||
cached.forEach((item) => byId.set(item.event.event_id, item));
|
||||
memory.forEach((item) => byId.set(item.event.event_id, item));
|
||||
return Array.from(byId.values()).sort(
|
||||
(a, b) => (b.event.origin_server_ts ?? 0) - (a.event.origin_server_ts ?? 0),
|
||||
);
|
||||
};
|
||||
|
||||
export const clearRoom = async (roomId: string): Promise<void> => {
|
||||
const db = await openDb();
|
||||
if (!db) return;
|
||||
try {
|
||||
const tx = db.transaction([MESSAGES_STORE, COVERAGE_STORE], 'readwrite');
|
||||
tx.objectStore(MESSAGES_STORE).delete(roomRange(roomId));
|
||||
tx.objectStore(COVERAGE_STORE).delete(roomId);
|
||||
await awaitTx(tx);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
};
|
||||
|
||||
export const clearAll = async (): Promise<void> => {
|
||||
const db = await openDb();
|
||||
if (!db) return;
|
||||
try {
|
||||
const tx = db.transaction([MESSAGES_STORE, COVERAGE_STORE], 'readwrite');
|
||||
tx.objectStore(MESSAGES_STORE).clear();
|
||||
tx.objectStore(COVERAGE_STORE).clear();
|
||||
await awaitTx(tx);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Drop the entire on-disk database. Wired into the logout path by the
|
||||
* coordinator (initMatrix) so no decrypted plaintext lingers after sign-out.
|
||||
* Closes any open handle first so the delete is not blocked. Never throws.
|
||||
*/
|
||||
export const deleteSearchCacheDatabase = async (): Promise<void> => {
|
||||
try {
|
||||
const existing = dbPromise ? await dbPromise : null;
|
||||
if (existing) existing.close();
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
dbPromise = null;
|
||||
return new Promise<void>((resolve) => {
|
||||
try {
|
||||
if (typeof indexedDB === 'undefined') {
|
||||
resolve();
|
||||
return;
|
||||
}
|
||||
const req = indexedDB.deleteDatabase(DB_NAME);
|
||||
req.onsuccess = () => resolve();
|
||||
req.onerror = () => resolve();
|
||||
req.onblocked = () => resolve();
|
||||
} catch {
|
||||
resolve();
|
||||
}
|
||||
});
|
||||
};
|
||||
@@ -6,6 +6,7 @@ import { getFallbackSession, removeFallbackSession, Session } from '../app/state
|
||||
import { LotusOidcTokenRefresher } from './oidcTokenRefresher';
|
||||
import { revokeOidcTokens } from './oidcLogout';
|
||||
import { pushSessionToSW } from '../sw-session';
|
||||
import { deleteSearchCacheDatabase } from '../app/utils/searchCache';
|
||||
|
||||
// Thrown when the local IndexedDB has a higher schema version than this SDK expects.
|
||||
// This happens after a downgrade (e.g. matrix-js-sdk was briefly upgraded and then reverted).
|
||||
@@ -87,6 +88,9 @@ export const logoutClient = async (mx: MatrixClient) => {
|
||||
// ignore if failed to logout
|
||||
}
|
||||
await mx.clearStores();
|
||||
// The opt-in local search index stores decrypted plaintext — always wipe it
|
||||
// on logout. (clearLoginData below nukes all IDB databases, covering it too.)
|
||||
await deleteSearchCacheDatabase();
|
||||
// Remove only the session credential keys, preserving user preferences and
|
||||
// unsent drafts (N98). The factory-reset path is clearLoginData() below.
|
||||
removeFallbackSession();
|
||||
|
||||
Reference in New Issue
Block a user