2026-06-29 20:50:10 -04:00
# HANDOFF — Forking & Self-Building Element Call ("Lotus Call")
> **Audience:** a fresh Claude/engineer session with **no prior context** on this
> project. Read this top-to-bottom before touching anything. This document is the
> single source of truth for the Element Call (EC) fork initiative.
>
2026-06-30 01:33:52 -04:00
> **Status:** **PHASE 0–
> (2026-06-30). The fork exists, builds, is published, and cinny consumes it
> (Phase 0/1). **All 7 Phase-2 EC features are implemented on the fork's `lotus`
> branch**, each additive + flag-gated, build+typecheck-clean, per-feature
> reviewed (+ a holistic multi-agent review), and pushed. **None are live-tested
> yet** — every one needs the `LOTUS_TESTING.md` §D sweep, and the **cinny host
> side must be wired** (set flags / send actions / handle call_state) — see §12.
> See **§9** Phase 0/1 results, **§10** cutover, **§11** Phase-2 seams, **§12**
> Phase-2 status + cinny integration checklist. Created 2026-06 from `LotusGuild/cinny`.
---
## 9. Phase 0 Results (verified 2026-06-29)
**Decisions taken with the user: ** scope = Phase 0 recon; consumption model =
**private npm package ** (§5 option 1). Recommended registry = **Gitea's built-in
npm registry** (`code.lotusguild.org` ) — zero new infra.
### 9.1 Version → tag → commit mapping (LOCKED)
2026-06-30 01:52:45 -04:00
| Source | Value |
| :--------------------------------------------------- | :----------------------------------------- |
| cinny `package.json` pin | `@element-hq/element-call-embedded@0.20.1` |
| Bundle self-report (`VITE_APP_VERSION` /`appVersion` ) | `embedded-v0.20.1` |
| npm registry `gitHead` for 0.20.1 | `2d74c48151d9edc01c65a22a91478aac81bf24d0` |
| GitHub tag `v0.20.1` → commit | `2d74c48…` ✅ **same commit ** |
2026-06-30 01:33:52 -04:00
→ **Fork from upstream tag `v0.20.1` (commit `2d74c48`). ** The embedded package
version equals the element-call release tag; repo `package.json` version is
`0.0.0` and the real version is stamped at publish time from the tag.
### 9.2 The shipped npm dist is a CLEAN upstream build
No `lotus` /`denoise` /`rnnoise` strings anywhere in
`node_modules/@element-hq/element-call-embedded/dist` . **All Lotus customization
(denoise shim) is injected at cinny build time, not baked into the package** — so
swapping the source does not disturb cinny's denoise injection layer. The
ringtone/reaction assets (`baduntss` , `cat` , `clap` , `call_declined` , …) are
upstream EC's own, not ours.
### 9.3 Build toolchain & mechanism
- **Node `24` ** (`.node-version` ), **pnpm `10.33.0` ** (`packageManager` field,
via corepack).
- Build: * * `pnpm run build:embedded` ** = `vite build --config
2026-06-30 01:52:45 -04:00
vite-embedded.config.ts` with ` NODE_OPTIONS=--max-old-space-size=16384`.
2026-06-30 01:33:52 -04:00
- Output dir is **repo-root ` dist/`**; CI stages it into **` embedded/web/dist`**
(the ` embedded/web/` dir holds the publish template: ` package.json`, README,
both LICENSE files).
- Publish workflow upstream = ` .github/workflows/publish-embedded-packages.yaml`:
builds → ` npm version <tag> --no-git-tag-version` → ` npm publish --provenance
2026-06-30 01:52:45 -04:00
--access public` to npmjs as ` @element -hq/element-call-embedded`. (Also
2026-06-30 01:33:52 -04:00
Android/Maven + iOS/SwiftPM — irrelevant; we are web-only.)
### 9.4 Build reproduction — PARITY CONFIRMED
Cloned ` element-call@v0 .20.1` to ` /root/code/element-call` (shallow), built with
isolated Node 24 / pnpm 10.33.0 (system Node 20 / cinny untouched). Result vs the
shipped npm dist:
- **137 of 147 files byte-identical** (same Vite content-hash): all CSS, fonts,
wasm, audio, JSON locale files, and ` IndexedDBWorker`.
- **Only 5 JS chunks differ** (` index`, ` pako.esm`, ` polyfill-force`,
` rust-crypto`, ` spa`) — **cause isolated to the version define**: our local
2026-06-30 01:52:45 -04:00
build baked ` appVersion:\`dev\``(because` VITE_APP_VERSION`was unset) vs the
npm build's` appVersion:\`embedded-v0.20.1\``. ` index.html` is identical modulo
2026-06-30 01:33:52 -04:00
the hashed asset filenames. **Benign** — our CI sets the version from the git
tag, so a tagged CI build will match.
### 9.5 Fork CI (drafted)
` .gitea/workflows/ci.yml` is staged in the clone (models cinny's
` .gitea/workflows/ci.yml` + upstream's publish flow). Linux-only (` ubuntu-latest`)
— the Windows worker is for cinny-desktop/Tauri, not the EC web bundle. Build job
on PR/push to ` lotus`; publish job on ` v*` tag → ` @lotusguild/element -call-embedded`
to the Gitea npm registry (needs ` secrets.GITEA_NPM_TOKEN`).
### 9.6 Phase 1 — DONE (2026-06-29)
1. ✅ **Fork repo live:** ` code.lotusguild.org/LotusGuild/element-call` (public,
AGPL), default branch ` lotus`, full history (7018 commits) + tag ` v0.20.1`.
Branch ` lotus` = ` v0.20.1` + 2-file diff (CI workflow + embedded package
rename).
2. ✅ **Package published:** ` @lotusguild/element -call-embedded@0 .20.1` on the
Gitea npm registry (published manually from the version-faithful build while
the admin token was available). **Publicly readable** (unauth ` npm install`
works → devs/CI need no token to consume; only publishing needs one).
3. ✅ **cinny wired & built clean** (Node 24): ` .npmrc` scope line +
` package.json` dep + ` vite.config.js` ` viteStaticCopy` src. ` npm install`
swapped the package (resolved from Gitea), ` npm run build` succeeded,
` dist/public/element-call/` populated, bundle reports ` appVersion:
2026-06-30 01:52:45 -04:00
embedded-v0.20.1`, **denoise shim injected + all denoise assets copied**
2026-06-30 01:33:52 -04:00
(injection layer unchanged). **These cinny edits are staged in the working
tree, NOT committed/pushed** — pushing triggers CI → desktop → deploy, so it's
gated on the §D live test (see §10).
### 9.8 Reproducibility note (important)
A from-source rebuild is **NOT byte-identical** to upstream's npm tarball.
137/147 files match exactly (CSS, fonts, wasm, audio, worker); the 5 JS chunks
(` index`, ` pako.esm`, ` polyfill-force`, ` rust-crypto`, ` spa`) differ because the
rolldown/oxc **minifier mangles export names differently** across build
environments (and the version-define is one input). This is normal and benign —
the code is functionally equivalent. **Do not chase byte-parity; the §D live call
test is the real parity gate.**
### 9.9 Remaining follow-ups (not blocking the cutover)
- **CI publishing:** ` .gitea/workflows/ci.yml` publishes on a ` v*` tag but needs
(a) a Gitea Actions runner for ` LotusGuild/element-call`, and (b) a **durable**
` GITEA_NPM_TOKEN` repo secret with package read/write (the admin token used for
the manual publish is being deleted, so it was deliberately NOT baked in). Until
then, publishing is manual (` npm version <tag>` in ` embedded/web` →
` npm publish`).
- Decide rebase cadence vs upstream (0.20.2 / 0.20.3 already out — see §9.1).
### 9.7 Ready-to-apply artifacts (staged 2026-06-29)
**Fork side — already committed** on branch ` lotus` in ` /root/code/element-call`
(remote ` lotus` = ` code.lotusguild.org/LotusGuild/element-call.git`, push deferred
until the repo exists). Minimal 2-file diff vs tag ` v0.20.1`:
` .gitea/workflows/ci.yml` (new) + ` embedded/web/package.json` (rename to
` @lotusguild/element -call-embedded`). Push with:
` git push -u lotus lotus && git push lotus v0.20.1` (and tag ` v0.20.1` on our side
to trigger the first publish, or push our own ` v0.20.1` tag).
**cinny side — NOT yet applied** (applying before the package is published breaks
` npm ci`). Exactly 3 edits + a lockfile regen:
1. ` .npmrc` — append the scoped-registry line:
` ``
@lotusguild:registry=https://code.lotusguild.org/api/packages/LotusGuild/npm/
` ``
(CI/auth: ` //code.lotusguild.org/api/packages/LotusGuild/npm/:_authToken=${GITEA_NPM_TOKEN}`
— inject via env in CI, do not commit a plaintext token.)
2. ` package.json:104` —
` "@element -hq/element-call-embedded": "0.20.1"` →
` "@lotusguild/element -call-embedded": "0.20.1"`.
3. ` vite.config.js:25` — ` viteStaticCopy` src:
` node_modules/@element -hq/element-call-embedded/dist` →
` node_modules/@lotusguild/element -call-embedded/dist`.
**` stripBase: 4` stays unchanged** — ` node_modules/@lotusguild/element -call-embedded/dist`
is still exactly 4 leading segments. (Update the comment's path reference too.)
4. ` package-lock.json` — regenerated by ` npm install`, not hand-edited (drops the
` registry.npmjs.org/@element -hq/...` resolved URL for the Gitea one).
The denoise injection (` lotusDenoise()` in ` vite.config.js`) is **unchanged** — it
keys off ` dist/public/element-call/index.html`, which our fork's bundle still
produces identically (verified: ` index.html` byte-identical modulo asset hashes).
2026-06-29 20:50:10 -04:00
---
## 0. TL;DR / The Goal
We embed **Element Call** (the Matrix group-VoIP/video app) inside Lotus Chat to
power voice/video channels. Today we consume Element's **pre-compiled npm
bundle** and can only steer it from the outside (a limited widget API + fragile
same-origin DOM hacks). Several in-call problems are **unfixable from outside**
because they live in EC's compiled JS.
**We want true ownership: fork ` element-hq/element-call`, build it from source
ourselves, host our build, and replace the npm bundle with our fork.** Then
every in-call behavior becomes editable code.
**This requires standing up a brand-new repo and build pipeline for our EC fork.**
---
## 1. Why fork? (What we cannot fix today)
These came out of live testing and are documented in ` LOTUS_BUGS.md` →
"Known Element Call iframe limitations":
| Issue | What's wrong | Why outside-fixes fail |
| :----------------------------------------------------- | :------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **A6** — avatar decorations in-call | Our profile-decoration overlays don't appear on in-call video tiles | The video grid is rendered by EC's React app inside the iframe. We can only inject overlay DOM (fragile) — we can't make it a first-class part of the tile. |
| **A5** — focus camera / fullscreen during screenshare | Can't reliably spotlight a participant's camera while someone screenshares | EC's **layout logic** (screenshare priority, spotlight) is compiled JS we don't control. We currently DOM-click tiles as a hack. |
| **A7** — mic dead after EC's "Reconnect" | After EC's own mid-call reconnect, the local mic isn't re-published | EC's reconnect/track-republish path is internal. (Partly entangled with our denoise shim — see §6.) |
| Native theming | EC's UI doesn't match Lotus design; we inject CSS hacks | Real theming needs source-level component/token changes. |
| Decorations, custom controls, custom layouts, branding | all blocked | all require source access |
**Bottom line:** the iframe is **same-origin** (we self-host it), so we can read
and even write its DOM — but we **do not own its source**, so we can't change its
**behavior/logic**, only poke at its rendered output. Forking removes that wall.
---
## 2. How EC is integrated TODAY (the current architecture)
Understand this fully before changing it — the fork must slot into the same
integration seams.
### 2.1 Where the EC bundle comes from
- npm package: **` @element -hq/element-call-embedded`**, pinned to **` 0.20.1`** in
` cinny/package.json` (line ~104).
- It ships a **pre-built ` dist/`**. At cinny build time,
` vite-plugin-static-copy` copies that ` dist/` flat into
**` public/element-call/`** (see ` cinny/vite.config.js`, the ` copyFiles`
target with ` rename: { stripBase: 4 }` — note the stripBase gotcha documented
there; getting this wrong 404s the widget).
- It is **NOT committed** to git (` git ls-files public/element-call` → 0). It's a
build artifact materialized from ` node_modules`.
### 2.2 How EC is loaded & controlled
- The widget iframe ` src` is **same-origin**:
` ${BASE_URL}/public/element-call/index.html?<params>` (see
` cinny/src/app/plugins/call/CallEmbed.ts`, ` getWidget()` /
` getIframe()`). Sandbox: ` allow-forms allow-scripts allow-same-origin
allow-popups allow-modals allow-downloads`; ` allow="microphone; camera;
display-capture; autoplay; clipboard-write;"`.
- **Control surface #1 — the official widget API** (` matrix-widget-api`):
` ClientWidgetApi` + a custom ` CallWidgetDriver`. This is the robust,
version-stable channel (theme change, hangup, capabilities, timeline events).
Files: ` plugins/call/CallEmbed.ts`, ` plugins/call/CallWidgetDriver.ts`,
` plugins/call/utils.ts` (capabilities), ` plugins/call/CallControl.ts`.
- **Control surface #2 — same-origin DOM poking** (fragile, version-coupled):
reading ` iframe.contentDocument` to detect speakers/mute state and
` .click()`-ing tiles to focus a camera. Files:
` hooks/useCallSpeakers.ts` (reads ` [data-muted]`, ` [data-video-fit]`),
` plugins/call/CallControl.ts` (` focusCameraParticipant` — tile selectors).
**These selectors break on every EC version bump.** A fork lets us replace
these hacks with real APIs/props.
- **Control surface #3 — URL params + build-time injection** for our denoise
shim (see §6).
### 2.3 Full file inventory (everything that touches EC in cinny)
Plugin / core:
- ` src/app/plugins/call/CallEmbed.ts` — iframe creation, widget API wiring, theme sync, hangup, load watchdog/self-heal, denoise URL params.
- ` src/app/plugins/call/CallControl.ts` — control state + **DOM-poking** (` focusCameraParticipant`, spotlight).
- ` src/app/plugins/call/CallControl.tsx` _(call-status variant)_ and ` features/call-status/CallControl.tsx`.
- ` src/app/plugins/call/CallWidgetDriver.ts` — widget driver (capabilities, event relay).
- ` src/app/plugins/call/utils.ts` — widget capabilities set.
- ` src/app/plugins/call/hooks.ts`, ` index.ts` — plugin exports/hooks.
- ` src/app/state/callEmbed.ts` — jotai atoms for the active embed.
React / UI:
- ` src/app/components/CallEmbedProvider.tsx` — the big one: incoming-call ring/banner, RTCNotification + **RTCDecline** listeners, PiP, mute badges, fullscreen, ringtones.
- ` src/app/features/call/CallView.tsx` — prescreen lobby vs joined (the iframe placement target), load-error recovery UI.
- ` src/app/features/call/CallControls.tsx` — in-call control bar (mic/cam/deafen/screenshare/fullscreen/more/PiP).
- ` src/app/features/call/CallMemberCard.tsx` — **lobby** participant roster (this is where ` AvatarDecoration` works today; in-call grid is EC's).
- ` src/app/features/call/PrescreenControls.tsx` — join controls.
- ` src/app/features/call-status/*` — ` CallStatus.tsx`, ` MemberGlance.tsx` (the "Focus camera" menu lives here), ` LiveChip.tsx`.
- ` src/app/features/room-nav/RoomNavItem.tsx`, ` features/room/Room.tsx`, ` features/room/RoomViewHeader.tsx`, ` pages/client/space/Space.tsx`, ` pages/CallStatusRenderer.tsx`, ` pages/Router.tsx` — call entry points / status surfacing.
Hooks:
- ` src/app/hooks/useCallEmbed.ts`, ` useCall.ts`, ` useCallSpeakers.ts` (DOM-poking), ` useCallJoinLeaveSounds.ts`, ` useAfkAutoMute.ts`.
Build:
- ` cinny/vite.config.js` — ` copyFiles` (EC dist copy) + ` lotusDenoise()` plugin (denoise asset copy + index.html shim injection, in ` closeBundle`).
Utils:
- ` src/app/utils/ringtones.ts`, ` utils/denoisePipeline.ts`, ` utils/lotusDenoiseUtils.ts`.
---
## 3. Hosting / infra context (the OTHER repo)
There are **two repos**:
1. **` LotusGuild/cinny`** (` /root/code/cinny`) — this Lotus Chat fork. Consumes EC.
2. **` LotusGuild/matrix`** (` /root/code/matrix`) — the **infra/homeserver** repo.
Subdirs: ` livekit/` (the SFU EC talks to), ` deploy/`, ` draupnir/`,
` hookshot/`, ` landing/`, ` matrixbot/`, ` systemd/`. Gitea remote
` code.lotusguild.org/LotusGuild/matrix`, branch ` main`.
EC needs a **LiveKit SFU** + the **livekit-jwt-service**; those live in
` matrix/livekit/`. A self-hosted EC build must be configured to point at our
homeserver (` matrix.lotusguild.org` / synapse) and our LiveKit. EC's runtime
` config.json` (homeserver, livekit URL, feature flags) is part of what we'll own
once we build it ourselves.
Deployment today: ` chat.lotusguild.org` (the cinny web build, which embeds EC at
` /public/element-call/`). cinny-desktop (` LotusGuild/cinny-desktop`, a Tauri
wrapper, bumped by cinny CI) embeds the same.
---
## 4. The plan (proposed — confirm with the user before executing)
### Decision: **YES, create a new repo.** ` LotusGuild/element-call`
Rationale: EC is a large standalone app (React + LiveKit client SDK + matrixRTC +
its own Vite build + heavy deps). Keep it out of cinny so cinny's build stays
clean — cinny keeps consuming a **built EC ` dist/`**, exactly as today, just
sourced from **our fork** instead of npm.
### Phase 0 — Recon (no code)
- Fork ` github.com/element-hq/element-call` → ` LotusGuild/element-call` on Gitea.
- Pin to the upstream tag matching **0.20.1** (` element-call-embedded` 0.20.1's
corresponding ` element-call` release) so behavior matches what's shipping now.
Verify the embedded-package version ↔ element-call repo tag mapping.
- Read EC's own build docs: it builds the "embedded" widget bundle (the thing
currently published as ` @element -hq/element-call-embedded`). Reproduce that
build locally and confirm the output matches ` public/element-call/` today.
- **License:** element-call is **AGPL-3.0**, same as Lotus Chat — compatible.
Our fork must remain AGPL and publish source.
### Phase 1 — Reproduce current behavior from our fork (parity, no features)
- Build our fork's embedded bundle; wire cinny to consume it instead of the npm
package (see §5 for the consumption options). Smoke-test: a call works exactly
as today (web + desktop), denoise shim still injects, widget API + theme still
work. **No behavior change yet** — this de-risks the swap.
### Phase 2 — Replace the outside hacks with source-level features
Tackle the §1 issues in EC's source:
- **A6:** render avatar decorations as part of the video-tile component
(read decoration data we pass in via widget data / URL param / a small bridge).
- **A5:** fix focus/spotlight + screenshare-coexistence in EC's layout code;
expose a clean widget action so cinny can trigger it (kill the DOM ` .click()`).
- **A7:** fix mic re-publish on reconnect; reconcile with our denoise shim (§6) —
ideally move denoise INTO the fork as a real audio-processing step instead of a
` getUserMedia` monkeypatch.
- Native Lotus theming/branding at the source (kill the injected-CSS hacks).
- Then retire the DOM-poking in ` useCallSpeakers.ts` / ` CallControl.ts` in favor
of real widget messages.
### Phase 3 — Maintenance posture
- Decide rebase cadence vs. upstream element-call releases. Keep customizations
isolated (feature flags / minimal-diff patches) to ease rebasing.
- CI in the new repo builds + publishes the embedded dist as a versioned
artifact; cinny CI consumes a pinned version.
---
## 5. How cinny should consume the fork (pick one — decide with user)
1. **Private npm package** (mirror the current model): our fork's CI publishes
` @lotusguild/element -call-embedded` to a registry; cinny depends on it and
` viteStaticCopy` keeps working almost unchanged. _Cleanest swap; needs a
registry._
2. **Git submodule + build in cinny CI:** add the fork as a submodule, build it
during cinny's build, copy its ` dist/` to ` public/element-call/`. _No
registry; heavier cinny CI._
3. **CI artifact copy:** fork CI uploads a ` dist` tarball; cinny CI downloads a
pinned version at build. _Decoupled; needs artifact plumbing._
**Recommendation: Option 1** — it changes the least in cinny (just swap the
package name in ` package.json` + the ` viteStaticCopy` src path) and preserves the
clean cinny/EC separation.
---
## 6. The denoise shim — critical interaction (don't break this)
Lotus ships ML noise suppression by **injecting a same-origin pre-init shim into
EC's ` index.html` at build time** (cinny ` vite.config.js` → ` lotusDenoise()`,
` closeBundle`). The shim monkeypatches ` getUserMedia` **before EC captures the
mic** and routes audio through RNNoise/Speex/DTLN AudioWorklets, then EC/LiveKit
publishes the processed track. It's activated via URL params
(` lotusDenoise=ml&lotusModel=…&lotusGate=…`) set in ` CallEmbed.ts`.
- Assets copied to ` public/element-call/denoise/` at build (sapphi RNNoise/Speex/
gate worklets + ` @workadventure/noise -suppression` DTLN tree).
- Related: ` utils/denoisePipeline.ts`, ` utils/lotusDenoiseUtils.ts`,
` settings/general/DenoiseTester.tsx`, ` VoiceMessageRecorder.tsx`.
- **Known issues:** denoise quality is still poor (tracked separately); and the
mic-after-reconnect bug (A7) is suspected to involve the shim's getUserMedia
patch handing back a stale processed stream when EC re-acquires the mic.
**Once we own the fork, the right move is to make denoise a first-class
audio-processing stage inside EC** (not an index.html monkeypatch) — more robust,
survives reconnects, and removes the build-time injection hack. Until then, the
fork's ` index.html` must remain injectable the same way, or the shim must be
re-homed into the fork.
---
## 7. Doc-accuracy notes / corrections for the new session
- ` LOTUS_TODO.md` (~line 533) calls EC a **"cross-origin iframe"** — **outdated.**
EC is **same-origin** today (self-hosted under our domain;
2026-06-30 01:33:52 -04:00
` iframe.sandbox` includes ` allow-same-origin`; we read ` contentDocument`), and
**as of 2026-06-29 we own the fork's source** (` @lotusguild/element -call-embedded`).
2026-06-30 01:52:45 -04:00
The _practical_ point it made still holds _until we ship the audio-inject API_:
2026-06-30 01:33:52 -04:00
**LiveKit's ` LocalAudioTrack` lives in EC's module scope**, not on ` window`, so
cinny can't reach it even same-origin — which is why the in-call soundboard had
to be local-playback-only. **The fork removes this wall:** EC can expose a real
` io.lotus.inject_audio` widget action (Phase 2) that mixes into the published
track from inside its own module scope.
2026-06-29 20:50:10 -04:00
- ` LOTUS_FEATURES.md` documents the EC upgrade history (0.16.3 → 0.19.4 →
0.20.1), the dark-mode CSS injection, and AFK auto-mute — all relevant prior
art for what the fork must preserve.
- ` LOTUS_TESTING.md` §D is the **EC regression sweep** to re-run after the fork
swap (Phase 1 parity check).
---
## 8. First actions for the new session
1. Read this file, then skim §2.3's files in ` cinny` to internalize the seams.
2. Confirm with the user: new repo name, consumption model (§5), rebase cadence.
3. Phase 0: fork element-call, map 0.20.1 ↔ element-call tag, reproduce the
embedded build locally, diff against ` public/element-call/`.
4. Phase 1: wire cinny to the fork, run ` LOTUS_TESTING.md` §D parity sweep.
5. Only then start Phase 2 features (A5/A6/A7, theming, denoise-in-source).
**Cross-references:** ` LOTUS_BUGS.md` (EC limitations + verify queue),
` LOTUS_TODO.md` (denoise/soundboard constraints), ` LOTUS_FEATURES.md` (EC history),
` LOTUS_TESTING.md` §D (regression sweep). Infra: ` /root/code/matrix` (` livekit/`,
` deploy/`).
2026-06-30 01:33:52 -04:00
---
## 10. Live cutover — the remaining steps (Phase 1 finish)
The fork is published and cinny builds against it locally (§9.6). What's left to
go live:
1. **Run ` LOTUS_TESTING.md` §D** against a local cinny build (` npm run build` is
already proven; serve ` dist/` or ` npm run dev`). Verify a real call: join,
mic/cam, screenshare, theme sync, denoise on, widget hangup — web first.
2. **Commit the cinny edits** (currently staged, uncommitted in the working tree):
` .npmrc`, ` package.json`, ` package-lock.json`, ` vite.config.js`. Suggested
message: ` chore(call): consume self-built @lotusguild/element -call-embedded`.
3. **Push to ` lotus`** → cinny CI builds, then ` trigger-desktop` bumps
cinny-desktop → Tauri release. Re-run §D on **cinny-desktop** (the path where
the old ` stripBase` bug bit — verify the widget loads, not a 404).
4. Only then start **Phase 2** (A5/A6/A7, theming, denoise-in-source).
---
## 11. Phase 2 — implementation seams (mapped 2026-06-29)
The exact integration points for each Phase 2 item, found by reading the EC fork
2026-06-30 01:52:45 -04:00
- cinny source. **All of these are media-path / in-call features that cannot be
functionally verified without a live Matrix + LiveKit call** — implement each as
a minimal, **feature-flagged, additive** diff (no behavior change unless cinny
opts in), build-verify the fork (` pnpm build:embedded`, ~15s) AND cinny
(` npm run build`), then gate shipping on ` LOTUS_TESTING.md` §D.
2026-06-30 01:33:52 -04:00
**Shared widget channel (the backbone for #2/#3/#4/#7):**
2026-06-30 01:52:45 -04:00
2026-06-30 01:33:52 -04:00
- EC→cinny: ` widget.api.transport.send("io.lotus.<x>", data)` (see
` element-call/src/widget.ts`).
- cinny→EC actions: add the action name to the ` lazyActions` allow-list in
` widget.ts` (the array at ~L101) and handle it in EC; cinny sends via
` this.call.transport.send(...)`.
- cinny receives EC→cinny actions via the existing ` listenAction(type, cb)`
helper in ` plugins/call/CallEmbed.ts:626` (auto-replies ` {}` so the transport
doesn't time out — same pattern as ` io.element.device_mute`).
**#2 mute/speaker events** — Source: subscribe to ` vm.userMedia$`
(` CallViewModel`), per member ` speaking$` + ` audioEnabled$`
(` state/media/UserMediaViewModel.ts:47-48`); aggregate and
` transport.send("io.lotus.call_state", {participants:[{id,speaking,audioEnabled}]})`.
Mount in ` room/InCallView.tsx` via ` useEffect` guarded by ` widget !== null`.
cinny: ` listenAction("io.lotus.call_state")` in ` CallEmbed.ts`, feed
` hooks/useCallSpeakers.ts` → delete its ` contentDocument` ` [data-muted]` /
2026-06-30 01:52:45 -04:00
` [data-video-fit]` scrape. _Additive, low risk._
2026-06-30 01:33:52 -04:00
**#4 spotlight/focus** — EC: add ` io.lotus.focus_participant` to the ` lazyActions`
list (` widget.ts`), drive ` vm`'s spotlight (` spotlightSpeaker$` /
` spotlight$` in ` CallViewModel.ts:898/1001`) to pin a given identity, coexisting
with ` hasRemoteScreenShares$` (L1008). cinny: replace
` CallControl.ts` ` focusCameraParticipant` ` .click()` walk with
2026-06-30 01:52:45 -04:00
` transport.send("io.lotus.focus_participant", {userId})`. _Additive, low risk._
2026-06-30 01:33:52 -04:00
**#3 audio-inject** — EC: add ` io.lotus.inject_audio` action; mix an
` AudioBufferSourceNode` into the published mic track. The local publish path is
` state/CallViewModel/localMember/Publisher.ts` + ` LocalMember.ts` (LiveKit
` localParticipant`); create a ` MediaStreamAudioDestinationNode`, mix mic + clip,
` replaceTrack`. cinny soundboard calls the action instead of local-only playback.
2026-06-30 01:52:45 -04:00
_Medium; touches publish path → live-test carefully._
2026-06-30 01:33:52 -04:00
**#1 denoise-in-source** — replace the cinny ` lotusDenoise()` ` getUserMedia`
monkeypatch with a real processing stage in EC's mic capture
(` Publisher.ts`/` LocalMember.ts`; note EC has a ` TrackProcessorContext` +
` BlurBackgroundTransformer` precedent in ` livekit/`). EC re-runs it on every
(re)publish → fixes A7. Remove ` vite.config.js` ` lotusDenoise()` + URL params in
2026-06-30 01:52:45 -04:00
` CallEmbed.ts`; move ` denoise/` assets into the fork. _Highest value, highest
risk — most live testing._
2026-06-30 01:33:52 -04:00
**#5 theming** — add a Lotus/TDS theme in EC's theme system (` src/useTheme.ts` +
EC theme tokens / CSS); driven by the existing ` setTheme()` channel cinny already
calls (` CallEmbed.ts:277`). Bake transparent background. Delete cinny's
2026-06-30 01:52:45 -04:00
` applyStyles()` injection + ` background:none !important`. _Medium._
2026-06-30 01:33:52 -04:00
**#6 in-call decorations** — render the decoration APNG in EC's tile component
(` tile/GridTile.tsx`); pass slugs via widget member data. cinny already has the
2026-06-30 01:52:45 -04:00
decoration data + ` AvatarDecoration` (lobby ` CallMemberCard.tsx`). _Medium-Large._
2026-06-30 01:33:52 -04:00
**#7 quality controls** — set audio ` maxBitrate` via
` RTCRtpSender.setParameters` and screenshare ` getDisplayMedia` constraints in
EC's publish path (` Publisher.ts`); configurable via ` config.json` / a widget
2026-06-30 01:52:45 -04:00
message. Keep the server ` voice-limit-guard` as enforcement. _Medium._
2026-06-30 01:33:52 -04:00
**Rollback:** revert the 4 cinny files (restores ` @element -hq/...@0 .20.1` from
npmjs). The fork repo/package can stay; nothing else depends on it until pushed.
### Local repro/build environment (this session, 2026-06-29)
- Upstream cloned + our ` lotus` branch at ` /root/code/element-call` (remote
` lotus` → Gitea; origin → github upstream, now un-shallowed/full history).
- Isolated **Node 24.18.0** lives in the session scratchpad (system Node is 20);
cinny's ` .node-version` is ` 24.13.1`, so use Node 24 to build cinny too.
- Build the embedded bundle: in ` /root/code/element-call`, with Node 24 + pnpm
10.33.0 on PATH, ` VITE_APP_VERSION=embedded-v0.20.1 pnpm run build:embedded`
→ output in ` dist/`; stage to ` embedded/web/dist` before publishing.
---
## 12. Phase 2 — IMPLEMENTED on the fork (2026-06-30)
All 7 EC features are on the ` lotus` branch of ` LotusGuild/element-call`, each
**additive + feature-flagged** (a vanilla call with no ` lotus*` params / no Lotus
actions behaves exactly like upstream), build + ` tsc` clean, per-feature reviewed
(fixes applied) and holistically reviewed. **Not yet live-tested** — all need the
` LOTUS_TESTING.md` §D sweep.
Fork modules live under ` element-call/src/lotus/*`; mounts are ` useEffect`s in
` src/room/InCallView.tsx`. Custom widget actions are in ` src/lotus/lotusActions.ts`
(toWidget ones allow-listed in ` src/widget.ts`).
2026-06-30 01:52:45 -04:00
| # | Feature | Enable via | EC module |
| :-- | :------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------ | ---------------------------------------------------- |
| 2 | Speaker/mute/camera state → host | URL ` lotusCallState=1` | ` lotusCallState.ts` (sends ` io.lotus.call_state`) |
| 4 | Focus/spotlight a participant (works during screenshare) | action ` io.lotus.focus_participant {userId | null}` | ` lotusFocus.ts` + ` CallViewModel` spotlight override |
| 3 | Soundboard audio-inject (heard by peers) | URL ` lotusAudioInject=1` + action ` io.lotus.inject_audio {url,volume?}` | ` lotusAudioInject.ts` |
| 7 | Audio/screenshare quality caps | action ` io.lotus.set_quality {audioMaxBitrate?,screenshareMaxBitrate?,screenshareMaxFramerate?}` | ` lotusQuality.ts` |
| 5 | Transparent bg + Lotus theme | URL ` lotusTransparent=1` / ` lotusTheme=1` | ` useTheme.ts` + ` index.css` |
| 6 | In-call avatar decorations | action ` io.lotus.decorations {decorations:{userId:url}}` | ` lotusDecorations.ts` + ` MediaView.tsx` |
| 1 | ML denoise in-source (fixes A7) | URL **` lotusDenoiseSource=1`** (+` lotusModel`,` lotusGate`,` lotusGateThreshold`,` lotusDenoiseBase`) — deliberately NOT the existing ` lotusDenoise=ml` (that drives the host shim; reusing it would double-process) | ` lotusDenoise.ts` + ` lotusDenoiseProcessor.ts` |
2026-06-30 01:33:52 -04:00
**Security hardening applied** (holistic audit): ` lotusDenoiseBase` forced
same-origin before ` audioWorklet.addModule` (was an arbitrary-code-load vector
via a crafted link); audio-inject gated behind ` lotusAudioInject=1`; decoration
roster capped. Only ` https`/` blob` URLs accepted for inject/decoration assets.
### 12.1 cinny host integration checklist (REQUIRED to light these up)
The EC side is additive and dormant until cinny opts in. Host work needed (in
` src/app/plugins/call/CallEmbed.ts` unless noted):
> ⚠️ **CRITICAL TIMING (protocol audit F1):** only send ` io.lotus.*` **toWidget**
> actions (#3 focus, #6 decorations, #7 quality, audio-inject) **after** the call
> is joined (` CallEmbed.onCallJoined` / ` this.joined`). Those actions are
> allow-listed at EC app-init (so ` preventDefault` suppresses the auto-error)
> but their handlers only mount with ` InCallView` (post-join). Sending earlier
> leaves the host's ` transport.send` pending until the **10s timeout**. Queue and
> flush on join, or no-op before join.
>
> Also: **F3** — the fork implements only ` rnnoise`/` speex`; cinny's ` dtln`/
> ` deepfilternet` selections silently fall back to rnnoise (now logged). Restrict
> the embedded-call model picker to rnnoise/speex, or implement the others in
> ` lotusDenoiseProcessor.ts`. **F4** — cinny sends ` lotusNativeNS`, which the
2026-06-30 01:52:45 -04:00
> fork ignores; drop it or wire it in. **F7** — no widget _capability_ changes
2026-06-30 01:33:52 -04:00
> needed; custom actions bypass capability checks.
1. **Set the URL flags** on the widget iframe params (the ` URLSearchParams` in
` CallEmbed`): ` lotusCallState=1`, ` lotusTransparent=1`/` lotusTheme=1`,
` lotusAudioInject=1` as desired. (Denoise already sets ` lotusDenoise=ml` etc.)
2. **Ack ` io.lotus.call_state`**: add ` listenAction('io.lotus.call_state', …)` —
without a reply the fork's sends time out every 250ms. Feed the payload into
` useCallSpeakers` and RETIRE its ` contentDocument` DOM scrape.
3. **Send actions** via ` this.call.transport.send(...)`:
` io.lotus.focus_participant` (replace ` CallControl.focusCameraParticipant`’
` .click()`), ` io.lotus.inject_audio` (from the soundboard), ` io.lotus.set_quality`
(from quality settings), ` io.lotus.decorations` (push the MSC4133 decoration
map; resolve mxc→https first).
4. **#1 denoise cutover**: once verified, STOP injecting the ` lotusDenoise()`
shim in ` cinny/vite.config.js` and remove the ` index.html` injection — the
fork now does denoise in-source. Keep shipping the ` denoise/` assets (the
fork loads ` ./denoise/…` at runtime) until those move into the fork build.
5. Re-run ` LOTUS_TESTING.md` §D for each feature; only then ship.
### 12.2 Holistic multi-agent review — outstanding follow-ups (non-blocking)
Four aspect-agents reviewed the whole fork. Criticals were fixed in-branch (the
denoise restart-silence/A7 bug; the ` lotusDenoiseBase` code-load vector;
audio-inject opt-in gate; #6 rendering in the wrong component; #7 simulcast cap).
Remaining, deliberately deferred:
- **Denoise H2 (double-processing):** if cinny is set to ` lotusDenoise=ml` while
ALSO still injecting its build-time ` getUserMedia` shim, audio is denoised
twice. The #1 cutover MUST remove the cinny-side injection (it currently has
none injected into the iframe — keep it that way). Hard requirement, not code.
- **Denoise M1 (perf):** in-source uses non-SIMD ` rnnoise.wasm`; the reference
preferred SIMD with detection. Perf-only; add SIMD detection later.
- **dtln/deepfilternet (F3): RESOLVED** — all four models
(rnnoise/speex/dtln/deepfilternet) are now implemented in
` lotusDenoiseProcessor.ts` (faithful port of cinny's ` build/lotus-denoise.js`
pipeline). This also fixed a real bug (the gate worklet name was ` noiseGate`;
correct is the hyphenated ` noise-gate`) and added per-model sample rates
(DTLN 16 kHz, others 48 kHz), context ` resume()`, and SIMD wasm selection.
Still needs live §D testing per model, and depends on cinny shipping the
DTLN (` denoise/workadventure/`) + DeepFilterNet (` denoise/deepfilternet/`)
asset trees (it already does).
- **Rebase-fragility (build agent MED):** the ` CallViewModel` spotlight override
edits hot upstream lines (renamed ` spotlightSpeaker$`→` autoSpotlightSpeaker$`).
For cheaper future rebases, refactor it into a ` src/lotus/lotusSpotlight.ts`
wrapper that takes the upstream stream and returns the overridden one, leaving
upstream's definition byte-identical (a single import + two token swaps).
- **Denoise asset coupling (build agent HIGH):** the fork loads ` ./denoise/*`
shipped by cinny, not by the fork build (documented in the processor). Add an
integration smoke-check that ` GET …/element-call/denoise/rnnoise.wasm` == 200,
and pin the ` @sapphi -red/web-noise-suppressor` version both repos expect.
- **Unconditional effect registration (build agent LOW):** focus/audio-inject/
quality/decorations register widget handlers on every embedded call (true
no-ops for a non-Lotus host). Intentional; gate behind a coarse ` lotus=1` flag
if strict zero-footprint is desired.
- **Privacy (security agent):** decoration/inject URLs accept any ` https`; ideally
restrict to the homeserver media origin host-side. Call-state exposes
userId/deviceId/speaking to the (trusted, same-origin) host — documented.
**Nothing here blocks the §D live test — but every feature still needs it.**
### 12.3 Safe rollout when prod is the only test environment
Every Phase-2 feature is now **dormant by default** — with the flags cinny sets
today, the fork behaves identically to the parity build (` #1 ` was decoupled onto
` lotusDenoiseSource=1` so it no longer collides with the host's ` lotusDenoise=ml`
shim). This enables a low-risk incremental rollout even without a staging env:
1. **Ship dormant first.** Publish the ` lotus` branch (e.g. ` 0.20.1-lotus.1`),
bump cinny's pin, deploy. With no Lotus flags set / no Lotus actions sent,
this is upstream-equivalent (only inert, holistically-reviewed code runs).
"Testing" here = confirm a normal call still works.
2. **Enable ONE feature at a time**, each independently revertable:
- URL-flag features (#2 ` lotusCallState`, #5 ` lotusTransparent`/` lotusTheme`,
#1 ` lotusDenoiseSource`): add the flag in ` CallEmbed.getWidget`, deploy,
test that one feature, roll back just that flag if needed.
- Action features (#3,#4,#6,#7): wire the host send + (for #2) the
` listenAction` ack, gated on join (§12.1 F1).
3. **#1 denoise cutover is a coordinated 2-step** (do together): set
` lotusDenoiseSource=1` AND remove the ` lotusDenoise()` shim injection +
` lotusDenoise=ml` param in cinny — otherwise audio is denoised twice.
Roll back = revert both.
4. Baseline is always upstream-equivalent, so any single feature can be disabled
by flipping its flag/send off without touching the rest.
**Blocker to step 1:** publishing the ` lotus` branch needs a Gitea npm token
(the admin token used for the ` 0.20.1` parity publish was deleted). Either
provide a token for a manual ` npm publish`, or stand up the Gitea Actions runner
2026-06-30 01:52:45 -04:00
- ` GITEA_NPM_TOKEN` secret so a ` v0.20.1-lotus.1` tag auto-publishes.
