From 049472e25faca94c8bf5b1b2211014fa002be780 Mon Sep 17 00:00:00 2001 From: Jared Vititoe Date: Thu, 2 Jul 2026 15:28:09 -0400 Subject: [PATCH] feat(crypto) + docs: request persistent storage; consolidate docs to 3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - index.tsx: request navigator.storage.persist() for logged-in sessions so the browser can't evict the IndexedDB rust-crypto store (eviction while the localStorage session survives resurrects the device with a blank store → the KE-1 "one time key already exists" upload storm). Guarded, checks persisted() first, best-effort. - Docs: remove HANDOFF_ELEMENT_CALL_FORK.md, LOTUS_E2EE_INVESTIGATION.md, and LOTUS_BUGS.md. Port their live content into the three kept docs — verification backlog → LOTUS_TESTING; open bugs + E2EE (KE-1..4) + an Element Call fork operational reference (publish steps + io.lotus action catalog) → LOTUS_TODO. Fix all dangling references (README, code comments, cross-doc links). Full history of the removed docs remains in git. Gates: tsc/eslint/prettier clean, build OK, 665 tests. Co-Authored-By: Claude Opus 4.8 --- HANDOFF_ELEMENT_CALL_FORK.md | 686 ---------------------------- LOTUS_BUGS.md | 188 -------- LOTUS_E2EE_INVESTIGATION.md | 504 -------------------- LOTUS_FEATURES.md | 4 +- LOTUS_TESTING.md | 50 +- LOTUS_TODO.md | 139 +++++- README.md | 8 +- src/app/plugins/call/CallControl.ts | 2 +- src/app/utils/cryptoDiagLog.ts | 2 +- src/index.tsx | 14 + 10 files changed, 205 insertions(+), 1392 deletions(-) delete mode 100644 HANDOFF_ELEMENT_CALL_FORK.md delete mode 100644 LOTUS_BUGS.md delete mode 100644 LOTUS_E2EE_INVESTIGATION.md diff --git a/HANDOFF_ELEMENT_CALL_FORK.md b/HANDOFF_ELEMENT_CALL_FORK.md deleted file mode 100644 index abfe7d2fd..000000000 --- a/HANDOFF_ELEMENT_CALL_FORK.md +++ /dev/null @@ -1,686 +0,0 @@ -# HANDOFF — Forking & Self-Building Element Call ("Lotus Call") - -> **Audience:** a fresh Claude/engineer session with **no prior context** on this -> project. Read this top-to-bottom before touching anything. This document is the -> single source of truth for the Element Call (EC) fork initiative. -> -> **Status:** **PHASE 0–2 IMPLEMENTED (build-verified, not yet live-tested)** -> (2026-06-30). The fork exists, builds, is published, and cinny consumes it -> (Phase 0/1). **All 7 Phase-2 EC features are implemented on the fork's `lotus` -> branch**, each additive + flag-gated, build+typecheck-clean, per-feature -> reviewed (+ a holistic multi-agent review), and pushed. **None are live-tested -> yet** — every one needs the `LOTUS_TESTING.md` §D sweep, and the **cinny host -> side must be wired** (set flags / send actions / handle call_state) — see §12. -> See **§9** Phase 0/1 results, **§10** cutover, **§11** Phase-2 seams, **§12** -> Phase-2 status + cinny integration checklist. Created 2026-06 from `LotusGuild/cinny`. - ---- - -## 9. Phase 0 Results (verified 2026-06-29) - -**Decisions taken with the user:** scope = Phase 0 recon; consumption model = -**private npm package** (§5 option 1). Recommended registry = **Gitea's built-in -npm registry** (`code.lotusguild.org`) — zero new infra. - -### 9.1 Version → tag → commit mapping (LOCKED) - -| Source | Value | -| :--------------------------------------------------- | :----------------------------------------- | -| cinny `package.json` pin | `@element-hq/element-call-embedded@0.20.1` | -| Bundle self-report (`VITE_APP_VERSION`/`appVersion`) | `embedded-v0.20.1` | -| npm registry `gitHead` for 0.20.1 | `2d74c48151d9edc01c65a22a91478aac81bf24d0` | -| GitHub tag `v0.20.1` → commit | `2d74c48…` ✅ **same commit** | - -→ **Fork from upstream tag `v0.20.1` (commit `2d74c48`).** The embedded package -version equals the element-call release tag; repo `package.json` version is -`0.0.0` and the real version is stamped at publish time from the tag. - -### 9.2 The shipped npm dist is a CLEAN upstream build - -No `lotus`/`denoise`/`rnnoise` strings anywhere in -`node_modules/@element-hq/element-call-embedded/dist`. **All Lotus customization -(denoise shim) is injected at cinny build time, not baked into the package** — so -swapping the source does not disturb cinny's denoise injection layer. The -ringtone/reaction assets (`baduntss`, `cat`, `clap`, `call_declined`, …) are -upstream EC's own, not ours. - -### 9.3 Build toolchain & mechanism - -- **Node `24`** (`.node-version`), **pnpm `10.33.0`** (`packageManager` field, - via corepack). -- Build: **`pnpm run build:embedded`** = `vite build --config -vite-embedded.config.ts` with `NODE_OPTIONS=--max-old-space-size=16384`. -- Output dir is **repo-root `dist/`**; CI stages it into **`embedded/web/dist`** - (the `embedded/web/` dir holds the publish template: `package.json`, README, - both LICENSE files). -- Publish workflow upstream = `.github/workflows/publish-embedded-packages.yaml`: - builds → `npm version --no-git-tag-version` → `npm publish --provenance ---access public` to npmjs as `@element-hq/element-call-embedded`. (Also - Android/Maven + iOS/SwiftPM — irrelevant; we are web-only.) - -### 9.4 Build reproduction — PARITY CONFIRMED - -Cloned `element-call@v0.20.1` to `/root/code/element-call` (shallow), built with -isolated Node 24 / pnpm 10.33.0 (system Node 20 / cinny untouched). Result vs the -shipped npm dist: - -- **137 of 147 files byte-identical** (same Vite content-hash): all CSS, fonts, - wasm, audio, JSON locale files, and `IndexedDBWorker`. -- **Only 5 JS chunks differ** (`index`, `pako.esm`, `polyfill-force`, - `rust-crypto`, `spa`) — **cause isolated to the version define**: our local - build baked `appVersion:\`dev\``(because`VITE_APP_VERSION`was unset) vs the -npm build's`appVersion:\`embedded-v0.20.1\``. `index.html` is identical modulo - the hashed asset filenames. **Benign** — our CI sets the version from the git - tag, so a tagged CI build will match. - -### 9.5 Fork CI (drafted) - -`.gitea/workflows/ci.yml` is staged in the clone (models cinny's -`.gitea/workflows/ci.yml` + upstream's publish flow). Linux-only (`ubuntu-latest`) -— the Windows worker is for cinny-desktop/Tauri, not the EC web bundle. Build job -on PR/push to `lotus`; publish job on `v*` tag → `@lotusguild/element-call-embedded` -to the Gitea npm registry (needs `secrets.GITEA_NPM_TOKEN`). - -### 9.6 Phase 1 — DONE (2026-06-29) - -1. ✅ **Fork repo live:** `code.lotusguild.org/LotusGuild/element-call` (public, - AGPL), default branch `lotus`, full history (7018 commits) + tag `v0.20.1`. - Branch `lotus` = `v0.20.1` + 2-file diff (CI workflow + embedded package - rename). -2. ✅ **Package published:** `@lotusguild/element-call-embedded@0.20.1` on the - Gitea npm registry (published manually from the version-faithful build while - the admin token was available). **Publicly readable** (unauth `npm install` - works → devs/CI need no token to consume; only publishing needs one). -3. ✅ **cinny wired & built clean** (Node 24): `.npmrc` scope line + - `package.json` dep + `vite.config.js` `viteStaticCopy` src. `npm install` - swapped the package (resolved from Gitea), `npm run build` succeeded, - `dist/public/element-call/` populated, bundle reports `appVersion: -embedded-v0.20.1`, **denoise shim injected + all denoise assets copied** - (injection layer unchanged). **These cinny edits are staged in the working - tree, NOT committed/pushed** — pushing triggers CI → desktop → deploy, so it's - gated on the §D live test (see §10). - -### 9.8 Reproducibility note (important) - -A from-source rebuild is **NOT byte-identical** to upstream's npm tarball. -137/147 files match exactly (CSS, fonts, wasm, audio, worker); the 5 JS chunks -(`index`, `pako.esm`, `polyfill-force`, `rust-crypto`, `spa`) differ because the -rolldown/oxc **minifier mangles export names differently** across build -environments (and the version-define is one input). This is normal and benign — -the code is functionally equivalent. **Do not chase byte-parity; the §D live call -test is the real parity gate.** - -### 9.9 Remaining follow-ups (not blocking the cutover) - -- **CI publishing:** `.gitea/workflows/ci.yml` publishes on a `v*` tag but needs - (a) a Gitea Actions runner for `LotusGuild/element-call`, and (b) a **durable** - `GITEA_NPM_TOKEN` repo secret with package read/write (the admin token used for - the manual publish is being deleted, so it was deliberately NOT baked in). Until - then, publishing is manual (`npm version ` in `embedded/web` → - `npm publish`). -- Decide rebase cadence vs upstream (0.20.2 / 0.20.3 already out — see §9.1). - -### 9.7 Ready-to-apply artifacts (staged 2026-06-29) - -**Fork side — already committed** on branch `lotus` in `/root/code/element-call` -(remote `lotus` = `code.lotusguild.org/LotusGuild/element-call.git`, push deferred -until the repo exists). Minimal 2-file diff vs tag `v0.20.1`: -`.gitea/workflows/ci.yml` (new) + `embedded/web/package.json` (rename to -`@lotusguild/element-call-embedded`). Push with: -`git push -u lotus lotus && git push lotus v0.20.1` (and tag `v0.20.1` on our side -to trigger the first publish, or push our own `v0.20.1` tag). - -**cinny side — NOT yet applied** (applying before the package is published breaks -`npm ci`). Exactly 3 edits + a lockfile regen: - -1. `.npmrc` — append the scoped-registry line: - ``` - @lotusguild:registry=https://code.lotusguild.org/api/packages/LotusGuild/npm/ - ``` - (CI/auth: `//code.lotusguild.org/api/packages/LotusGuild/npm/:_authToken=${GITEA_NPM_TOKEN}` - — inject via env in CI, do not commit a plaintext token.) -2. `package.json:104` — - `"@element-hq/element-call-embedded": "0.20.1"` → - `"@lotusguild/element-call-embedded": "0.20.1"`. -3. `vite.config.js:25` — `viteStaticCopy` src: - `node_modules/@element-hq/element-call-embedded/dist` → - `node_modules/@lotusguild/element-call-embedded/dist`. - **`stripBase: 4` stays unchanged** — `node_modules/@lotusguild/element-call-embedded/dist` - is still exactly 4 leading segments. (Update the comment's path reference too.) -4. `package-lock.json` — regenerated by `npm install`, not hand-edited (drops the - `registry.npmjs.org/@element-hq/...` resolved URL for the Gitea one). - -The denoise injection (`lotusDenoise()` in `vite.config.js`) is **unchanged** — it -keys off `dist/public/element-call/index.html`, which our fork's bundle still -produces identically (verified: `index.html` byte-identical modulo asset hashes). - ---- - -## 0. TL;DR / The Goal - -We embed **Element Call** (the Matrix group-VoIP/video app) inside Lotus Chat to -power voice/video channels. Today we consume Element's **pre-compiled npm -bundle** and can only steer it from the outside (a limited widget API + fragile -same-origin DOM hacks). Several in-call problems are **unfixable from outside** -because they live in EC's compiled JS. - -**We want true ownership: fork `element-hq/element-call`, build it from source -ourselves, host our build, and replace the npm bundle with our fork.** Then -every in-call behavior becomes editable code. - -**This requires standing up a brand-new repo and build pipeline for our EC fork.** - ---- - -## 1. Why fork? (What we cannot fix today) - -These came out of live testing and are documented in `LOTUS_BUGS.md` → -"Known Element Call iframe limitations": - -| Issue | What's wrong | Why outside-fixes fail | -| :----------------------------------------------------- | :------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **A6** — avatar decorations in-call | Our profile-decoration overlays don't appear on in-call video tiles | The video grid is rendered by EC's React app inside the iframe. We can only inject overlay DOM (fragile) — we can't make it a first-class part of the tile. | -| **A5** — focus camera / fullscreen during screenshare | Can't reliably spotlight a participant's camera while someone screenshares | EC's **layout logic** (screenshare priority, spotlight) is compiled JS we don't control. We currently DOM-click tiles as a hack. | -| **A7** — mic dead after EC's "Reconnect" | After EC's own mid-call reconnect, the local mic isn't re-published | EC's reconnect/track-republish path is internal. (Partly entangled with our denoise shim — see §6.) | -| Native theming | EC's UI doesn't match Lotus design; we inject CSS hacks | Real theming needs source-level component/token changes. | -| Decorations, custom controls, custom layouts, branding | all blocked | all require source access | - -**Bottom line:** the iframe is **same-origin** (we self-host it), so we can read -and even write its DOM — but we **do not own its source**, so we can't change its -**behavior/logic**, only poke at its rendered output. Forking removes that wall. - ---- - -## 2. How EC is integrated TODAY (the current architecture) - -Understand this fully before changing it — the fork must slot into the same -integration seams. - -### 2.1 Where the EC bundle comes from - -- npm package: **`@element-hq/element-call-embedded`**, pinned to **`0.20.1`** in - `cinny/package.json` (line ~104). -- It ships a **pre-built `dist/`**. At cinny build time, - `vite-plugin-static-copy` copies that `dist/` flat into - **`public/element-call/`** (see `cinny/vite.config.js`, the `copyFiles` - target with `rename: { stripBase: 4 }` — note the stripBase gotcha documented - there; getting this wrong 404s the widget). -- It is **NOT committed** to git (`git ls-files public/element-call` → 0). It's a - build artifact materialized from `node_modules`. - -### 2.2 How EC is loaded & controlled - -- The widget iframe `src` is **same-origin**: - `${BASE_URL}/public/element-call/index.html?` (see - `cinny/src/app/plugins/call/CallEmbed.ts`, `getWidget()` / - `getIframe()`). Sandbox: `allow-forms allow-scripts allow-same-origin -allow-popups allow-modals allow-downloads`; `allow="microphone; camera; -display-capture; autoplay; clipboard-write;"`. -- **Control surface #1 — the official widget API** (`matrix-widget-api`): - `ClientWidgetApi` + a custom `CallWidgetDriver`. This is the robust, - version-stable channel (theme change, hangup, capabilities, timeline events). - Files: `plugins/call/CallEmbed.ts`, `plugins/call/CallWidgetDriver.ts`, - `plugins/call/utils.ts` (capabilities), `plugins/call/CallControl.ts`. -- **Control surface #2 — same-origin DOM poking** (fragile, version-coupled): - reading `iframe.contentDocument` to detect speakers/mute state and - `.click()`-ing tiles to focus a camera. Files: - `hooks/useCallSpeakers.ts` (reads `[data-muted]`, `[data-video-fit]`), - `plugins/call/CallControl.ts` (`focusCameraParticipant` — tile selectors). - **These selectors break on every EC version bump.** A fork lets us replace - these hacks with real APIs/props. -- **Control surface #3 — URL params + build-time injection** for our denoise - shim (see §6). - -### 2.3 Full file inventory (everything that touches EC in cinny) - -Plugin / core: - -- `src/app/plugins/call/CallEmbed.ts` — iframe creation, widget API wiring, theme sync, hangup, load watchdog/self-heal, denoise URL params. -- `src/app/plugins/call/CallControl.ts` — control state + **DOM-poking** (`focusCameraParticipant`, spotlight). -- `src/app/plugins/call/CallControl.tsx` _(call-status variant)_ and `features/call-status/CallControl.tsx`. -- `src/app/plugins/call/CallWidgetDriver.ts` — widget driver (capabilities, event relay). -- `src/app/plugins/call/utils.ts` — widget capabilities set. -- `src/app/plugins/call/hooks.ts`, `index.ts` — plugin exports/hooks. -- `src/app/state/callEmbed.ts` — jotai atoms for the active embed. - -React / UI: - -- `src/app/components/CallEmbedProvider.tsx` — the big one: incoming-call ring/banner, RTCNotification + **RTCDecline** listeners, PiP, mute badges, fullscreen, ringtones. -- `src/app/features/call/CallView.tsx` — prescreen lobby vs joined (the iframe placement target), load-error recovery UI. -- `src/app/features/call/CallControls.tsx` — in-call control bar (mic/cam/deafen/screenshare/fullscreen/more/PiP). -- `src/app/features/call/CallMemberCard.tsx` — **lobby** participant roster (this is where `AvatarDecoration` works today; in-call grid is EC's). -- `src/app/features/call/PrescreenControls.tsx` — join controls. -- `src/app/features/call-status/*` — `CallStatus.tsx`, `MemberGlance.tsx` (the "Focus camera" menu lives here), `LiveChip.tsx`. -- `src/app/features/room-nav/RoomNavItem.tsx`, `features/room/Room.tsx`, `features/room/RoomViewHeader.tsx`, `pages/client/space/Space.tsx`, `pages/CallStatusRenderer.tsx`, `pages/Router.tsx` — call entry points / status surfacing. - -Hooks: - -- `src/app/hooks/useCallEmbed.ts`, `useCall.ts`, `useCallSpeakers.ts` (DOM-poking), `useCallJoinLeaveSounds.ts`, `useAfkAutoMute.ts`. - -Build: - -- `cinny/vite.config.js` — `copyFiles` (EC dist copy) + `lotusDenoise()` plugin (denoise asset copy + index.html shim injection, in `closeBundle`). - -Utils: - -- `src/app/utils/ringtones.ts`, `utils/denoisePipeline.ts`, `utils/lotusDenoiseUtils.ts`. - ---- - -## 3. Hosting / infra context (the OTHER repo) - -There are **two repos**: - -1. **`LotusGuild/cinny`** (`/root/code/cinny`) — this Lotus Chat fork. Consumes EC. -2. **`LotusGuild/matrix`** (`/root/code/matrix`) — the **infra/homeserver** repo. - Subdirs: `livekit/` (the SFU EC talks to), `deploy/`, `draupnir/`, - `hookshot/`, `landing/`, `matrixbot/`, `systemd/`. Gitea remote - `code.lotusguild.org/LotusGuild/matrix`, branch `main`. - -EC needs a **LiveKit SFU** + the **livekit-jwt-service**; those live in -`matrix/livekit/`. A self-hosted EC build must be configured to point at our -homeserver (`matrix.lotusguild.org` / synapse) and our LiveKit. EC's runtime -`config.json` (homeserver, livekit URL, feature flags) is part of what we'll own -once we build it ourselves. - -Deployment today: `chat.lotusguild.org` (the cinny web build, which embeds EC at -`/public/element-call/`). cinny-desktop (`LotusGuild/cinny-desktop`, a Tauri -wrapper, bumped by cinny CI) embeds the same. - ---- - -## 4. The plan (proposed — confirm with the user before executing) - -### Decision: **YES, create a new repo.** `LotusGuild/element-call` - -Rationale: EC is a large standalone app (React + LiveKit client SDK + matrixRTC + -its own Vite build + heavy deps). Keep it out of cinny so cinny's build stays -clean — cinny keeps consuming a **built EC `dist/`**, exactly as today, just -sourced from **our fork** instead of npm. - -### Phase 0 — Recon (no code) - -- Fork `github.com/element-hq/element-call` → `LotusGuild/element-call` on Gitea. -- Pin to the upstream tag matching **0.20.1** (`element-call-embedded` 0.20.1's - corresponding `element-call` release) so behavior matches what's shipping now. - Verify the embedded-package version ↔ element-call repo tag mapping. -- Read EC's own build docs: it builds the "embedded" widget bundle (the thing - currently published as `@element-hq/element-call-embedded`). Reproduce that - build locally and confirm the output matches `public/element-call/` today. -- **License:** element-call is **AGPL-3.0**, same as Lotus Chat — compatible. - Our fork must remain AGPL and publish source. - -### Phase 1 — Reproduce current behavior from our fork (parity, no features) - -- Build our fork's embedded bundle; wire cinny to consume it instead of the npm - package (see §5 for the consumption options). Smoke-test: a call works exactly - as today (web + desktop), denoise shim still injects, widget API + theme still - work. **No behavior change yet** — this de-risks the swap. - -### Phase 2 — Replace the outside hacks with source-level features - -Tackle the §1 issues in EC's source: - -- **A6:** render avatar decorations as part of the video-tile component - (read decoration data we pass in via widget data / URL param / a small bridge). -- **A5:** fix focus/spotlight + screenshare-coexistence in EC's layout code; - expose a clean widget action so cinny can trigger it (kill the DOM `.click()`). -- **A7:** fix mic re-publish on reconnect; reconcile with our denoise shim (§6) — - ideally move denoise INTO the fork as a real audio-processing step instead of a - `getUserMedia` monkeypatch. -- Native Lotus theming/branding at the source (kill the injected-CSS hacks). -- Then retire the DOM-poking in `useCallSpeakers.ts` / `CallControl.ts` in favor - of real widget messages. - -### Phase 3 — Maintenance posture - -- Decide rebase cadence vs. upstream element-call releases. Keep customizations - isolated (feature flags / minimal-diff patches) to ease rebasing. -- CI in the new repo builds + publishes the embedded dist as a versioned - artifact; cinny CI consumes a pinned version. - ---- - -## 5. How cinny should consume the fork (pick one — decide with user) - -1. **Private npm package** (mirror the current model): our fork's CI publishes - `@lotusguild/element-call-embedded` to a registry; cinny depends on it and - `viteStaticCopy` keeps working almost unchanged. _Cleanest swap; needs a - registry._ -2. **Git submodule + build in cinny CI:** add the fork as a submodule, build it - during cinny's build, copy its `dist/` to `public/element-call/`. _No - registry; heavier cinny CI._ -3. **CI artifact copy:** fork CI uploads a `dist` tarball; cinny CI downloads a - pinned version at build. _Decoupled; needs artifact plumbing._ - -**Recommendation: Option 1** — it changes the least in cinny (just swap the -package name in `package.json` + the `viteStaticCopy` src path) and preserves the -clean cinny/EC separation. - ---- - -## 6. The denoise shim — critical interaction (don't break this) - -Lotus ships ML noise suppression by **injecting a same-origin pre-init shim into -EC's `index.html` at build time** (cinny `vite.config.js` → `lotusDenoise()`, -`closeBundle`). The shim monkeypatches `getUserMedia` **before EC captures the -mic** and routes audio through RNNoise/Speex/DTLN AudioWorklets, then EC/LiveKit -publishes the processed track. It's activated via URL params -(`lotusDenoise=ml&lotusModel=…&lotusGate=…`) set in `CallEmbed.ts`. - -- Assets copied to `public/element-call/denoise/` at build (sapphi RNNoise/Speex/ - gate worklets + `@workadventure/noise-suppression` DTLN tree). -- Related: `utils/denoisePipeline.ts`, `utils/lotusDenoiseUtils.ts`, - `settings/general/DenoiseTester.tsx`, `VoiceMessageRecorder.tsx`. -- **Known issues:** denoise quality is still poor (tracked separately); and the - mic-after-reconnect bug (A7) is suspected to involve the shim's getUserMedia - patch handing back a stale processed stream when EC re-acquires the mic. - -**Once we own the fork, the right move is to make denoise a first-class -audio-processing stage inside EC** (not an index.html monkeypatch) — more robust, -survives reconnects, and removes the build-time injection hack. Until then, the -fork's `index.html` must remain injectable the same way, or the shim must be -re-homed into the fork. - ---- - -## 7. Doc-accuracy notes / corrections for the new session - -- `LOTUS_TODO.md` (~line 533) calls EC a **"cross-origin iframe"** — **outdated.** - EC is **same-origin** today (self-hosted under our domain; - `iframe.sandbox` includes `allow-same-origin`; we read `contentDocument`), and - **as of 2026-06-29 we own the fork's source** (`@lotusguild/element-call-embedded`). - The _practical_ point it made still holds _until we ship the audio-inject API_: - **LiveKit's `LocalAudioTrack` lives in EC's module scope**, not on `window`, so - cinny can't reach it even same-origin — which is why the in-call soundboard had - to be local-playback-only. **The fork removes this wall:** EC can expose a real - `io.lotus.inject_audio` widget action (Phase 2) that mixes into the published - track from inside its own module scope. -- `LOTUS_FEATURES.md` documents the EC upgrade history (0.16.3 → 0.19.4 → - 0.20.1), the dark-mode CSS injection, and AFK auto-mute — all relevant prior - art for what the fork must preserve. -- `LOTUS_TESTING.md` §D is the **EC regression sweep** to re-run after the fork - swap (Phase 1 parity check). - ---- - -## 8. First actions for the new session - -1. Read this file, then skim §2.3's files in `cinny` to internalize the seams. -2. Confirm with the user: new repo name, consumption model (§5), rebase cadence. -3. Phase 0: fork element-call, map 0.20.1 ↔ element-call tag, reproduce the - embedded build locally, diff against `public/element-call/`. -4. Phase 1: wire cinny to the fork, run `LOTUS_TESTING.md` §D parity sweep. -5. Only then start Phase 2 features (A5/A6/A7, theming, denoise-in-source). - -**Cross-references:** `LOTUS_BUGS.md` (EC limitations + verify queue), -`LOTUS_TODO.md` (denoise/soundboard constraints), `LOTUS_FEATURES.md` (EC history), -`LOTUS_TESTING.md` §D (regression sweep). Infra: `/root/code/matrix` (`livekit/`, -`deploy/`). - ---- - -## 10. Live cutover — the remaining steps (Phase 1 finish) - -The fork is published and cinny builds against it locally (§9.6). What's left to -go live: - -1. **Run `LOTUS_TESTING.md` §D** against a local cinny build (`npm run build` is - already proven; serve `dist/` or `npm run dev`). Verify a real call: join, - mic/cam, screenshare, theme sync, denoise on, widget hangup — web first. -2. **Commit the cinny edits** (currently staged, uncommitted in the working tree): - `.npmrc`, `package.json`, `package-lock.json`, `vite.config.js`. Suggested - message: `chore(call): consume self-built @lotusguild/element-call-embedded`. -3. **Push to `lotus`** → cinny CI builds, then `trigger-desktop` bumps - cinny-desktop → Tauri release. Re-run §D on **cinny-desktop** (the path where - the old `stripBase` bug bit — verify the widget loads, not a 404). -4. Only then start **Phase 2** (A5/A6/A7, theming, denoise-in-source). - ---- - -## 11. Phase 2 — implementation seams (mapped 2026-06-29) - -The exact integration points for each Phase 2 item, found by reading the EC fork - -- cinny source. **All of these are media-path / in-call features that cannot be - functionally verified without a live Matrix + LiveKit call** — implement each as - a minimal, **feature-flagged, additive** diff (no behavior change unless cinny - opts in), build-verify the fork (`pnpm build:embedded`, ~15s) AND cinny - (`npm run build`), then gate shipping on `LOTUS_TESTING.md` §D. - -**Shared widget channel (the backbone for #2/#3/#4/#7):** - -- EC→cinny: `widget.api.transport.send("io.lotus.", data)` (see - `element-call/src/widget.ts`). -- cinny→EC actions: add the action name to the `lazyActions` allow-list in - `widget.ts` (the array at ~L101) and handle it in EC; cinny sends via - `this.call.transport.send(...)`. -- cinny receives EC→cinny actions via the existing `listenAction(type, cb)` - helper in `plugins/call/CallEmbed.ts:626` (auto-replies `{}` so the transport - doesn't time out — same pattern as `io.element.device_mute`). - -**#2 mute/speaker events** — Source: subscribe to `vm.userMedia$` -(`CallViewModel`), per member `speaking$` + `audioEnabled$` -(`state/media/UserMediaViewModel.ts:47-48`); aggregate and -`transport.send("io.lotus.call_state", {participants:[{id,speaking,audioEnabled}]})`. -Mount in `room/InCallView.tsx` via `useEffect` guarded by `widget !== null`. -cinny: `listenAction("io.lotus.call_state")` in `CallEmbed.ts`, feed -`hooks/useCallSpeakers.ts` → delete its `contentDocument` `[data-muted]` / -`[data-video-fit]` scrape. _Additive, low risk._ - -**#4 spotlight/focus** — EC: add `io.lotus.focus_participant` to the `lazyActions` -list (`widget.ts`), drive `vm`'s spotlight (`spotlightSpeaker$` / -`spotlight$` in `CallViewModel.ts:898/1001`) to pin a given identity, coexisting -with `hasRemoteScreenShares$` (L1008). cinny: replace -`CallControl.ts` `focusCameraParticipant` `.click()` walk with -`transport.send("io.lotus.focus_participant", {userId})`. _Additive, low risk._ - -**#3 audio-inject** — EC: add `io.lotus.inject_audio` action; mix an -`AudioBufferSourceNode` into the published mic track. The local publish path is -`state/CallViewModel/localMember/Publisher.ts` + `LocalMember.ts` (LiveKit -`localParticipant`); create a `MediaStreamAudioDestinationNode`, mix mic + clip, -`replaceTrack`. cinny soundboard calls the action instead of local-only playback. -_Medium; touches publish path → live-test carefully._ - -**#1 denoise-in-source** — replace the cinny `lotusDenoise()` `getUserMedia` -monkeypatch with a real processing stage in EC's mic capture -(`Publisher.ts`/`LocalMember.ts`; note EC has a `TrackProcessorContext` + -`BlurBackgroundTransformer` precedent in `livekit/`). EC re-runs it on every -(re)publish → fixes A7. Remove `vite.config.js` `lotusDenoise()` + URL params in -`CallEmbed.ts`; move `denoise/` assets into the fork. _Highest value, highest -risk — most live testing._ - -**#5 theming** — add a Lotus/TDS theme in EC's theme system (`src/useTheme.ts` + -EC theme tokens / CSS); driven by the existing `setTheme()` channel cinny already -calls (`CallEmbed.ts:277`). Bake transparent background. Delete cinny's -`applyStyles()` injection + `background:none !important`. _Medium._ - -**#6 in-call decorations** — render the decoration APNG in EC's tile component -(`tile/GridTile.tsx`); pass slugs via widget member data. cinny already has the -decoration data + `AvatarDecoration` (lobby `CallMemberCard.tsx`). _Medium-Large._ - -**#7 quality controls** — set audio `maxBitrate` via -`RTCRtpSender.setParameters` and screenshare `getDisplayMedia` constraints in -EC's publish path (`Publisher.ts`); configurable via `config.json` / a widget -message. Keep the server `voice-limit-guard` as enforcement. _Medium._ - -**Rollback:** revert the 4 cinny files (restores `@element-hq/...@0.20.1` from -npmjs). The fork repo/package can stay; nothing else depends on it until pushed. - -### Local repro/build environment (this session, 2026-06-29) - -- Upstream cloned + our `lotus` branch at `/root/code/element-call` (remote - `lotus` → Gitea; origin → github upstream, now un-shallowed/full history). -- Isolated **Node 24.18.0** lives in the session scratchpad (system Node is 20); - cinny's `.node-version` is `24.13.1`, so use Node 24 to build cinny too. -- Build the embedded bundle: in `/root/code/element-call`, with Node 24 + pnpm - 10.33.0 on PATH, `VITE_APP_VERSION=embedded-v0.20.1 pnpm run build:embedded` - → output in `dist/`; stage to `embedded/web/dist` before publishing. - ---- - -## 12. Phase 2 — IMPLEMENTED on the fork (2026-06-30) - -All 7 EC features are on the `lotus` branch of `LotusGuild/element-call`, each -**additive + feature-flagged** (a vanilla call with no `lotus*` params / no Lotus -actions behaves exactly like upstream), build + `tsc` clean, per-feature reviewed -(fixes applied) and holistically reviewed. **Not yet live-tested** — all need the -`LOTUS_TESTING.md` §D sweep. - -Fork modules live under `element-call/src/lotus/*`; mounts are `useEffect`s in -`src/room/InCallView.tsx`. Custom widget actions are in `src/lotus/lotusActions.ts` -(toWidget ones allow-listed in `src/widget.ts`). - -| # | Feature | Enable via | EC module | -| :--- | :------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------ | ---------------------------------------------------- | -| 2 | Speaker/mute/camera state → host | URL `lotusCallState=1` | `lotusCallState.ts` (sends `io.lotus.call_state`) | -| 4 | Focus/spotlight a participant (works during screenshare) | action `io.lotus.focus_participant {userId | null}` | `lotusFocus.ts` + `CallViewModel` spotlight override | -| 3 | Soundboard audio-inject (heard by peers) | URL `lotusAudioInject=1` + action `io.lotus.inject_audio {url,volume?}` | `lotusAudioInject.ts` | -| 7 | Audio/screenshare quality caps | action `io.lotus.set_quality {audioMaxBitrate?,screenshareMaxBitrate?,screenshareMaxFramerate?}` | `lotusQuality.ts` | -| 5 | Transparent bg + Lotus theme | URL `lotusTransparent=1` / `lotusTheme=1` | `useTheme.ts` + `index.css` | -| 6 | In-call avatar decorations | action `io.lotus.decorations {decorations:{userId:url}}` | `lotusDecorations.ts` + `MediaView.tsx` | -| 1 | ML denoise in-source (fixes A7) | URL **`lotusDenoiseSource=1`** (+`lotusModel`,`lotusGate`,`lotusGateThreshold`,`lotusDenoiseBase`) — deliberately NOT the existing `lotusDenoise=ml` (that drives the host shim; reusing it would double-process) | `lotusDenoise.ts` + `lotusDenoiseProcessor.ts` | -| P6-2 | Deafen / screenshare-audio-mute at the LiveKit source | action `io.lotus.set_deafen {deafened,screenshareAudioMuted}` — sets remote `RemoteParticipant.setVolume(0/1)` per source (Microphone + ScreenShareAudio), persists to late joiners via `RoomEvent.ParticipantConnected` | `lotusDeafen.ts` | - -### 12.4 P6-2 — deafen action (retires cinny's iframe-DOM `.muted` hack) - -`io.lotus.set_deafen` (fork commit, folded into unpublished **`0.20.1-lotus.2`**) replaces -cinny's `CallControl.setSound`/`applyScreenshareAudioMuted` DOM `