call: consume self-built Element Call fork + activate Lotus features
CI / Build & Quality Checks (push) Successful in 11m5s
CI / Trigger Desktop Build (push) Successful in 25s

Switch to @lotusguild/element-call-embedded@0.20.1-lotus.1 (our self-built
fork) and turn on the source-level features it adds:

- #1 denoise CUTOVER: in-source ML denoise (lotusDenoiseSource=1) replaces
  the build-time getUserMedia shim — removed the shim injection from
  vite.config.js (denoise/ assets still shipped; the processor loads them).
  Survives reconnects (fixes A7).
- #2 call-state: CallEmbed consumes io.lotus.call_state; useCallSpeakers /
  useRemoteAllMuted prefer it over scraping EC's DOM (DOM fallback kept;
  empty payloads ignored).
- #4 focus: CallControl.focusCameraParticipant sends io.lotus.focus_participant
  (works during screenshare), replacing the DOM tile-click hack.
- #5 theming: lotusTransparent=1 (native transparent background).
- #6 decorations: LotusDecorationPusher sends each member's decoration URL
  via io.lotus.decorations -> rendered on in-call tiles.

#3 soundboard / #7 quality ship dormant (EC-ready; no host UI sends them yet).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-30 01:33:52 -04:00
parent 149ec8e4e4
commit 89cf171efc
11 changed files with 640 additions and 135 deletions
+2 -1
View File
@@ -1,2 +1,3 @@
legacy-peer-deps=true
save-exact=true
save-exact=true
@lotusguild:registry=https://code.lotusguild.org/api/packages/LotusGuild/npm/
+390 -8
View File
@@ -4,8 +4,155 @@
> project. Read this top-to-bottom before touching anything. This document is the
> single source of truth for the Element Call (EC) fork initiative.
>
> **Status:** NOT STARTED. This is the planning/handoff artifact. Created
> 2026-06 from the Lotus Chat (`LotusGuild/cinny`) repo.
> **Status:** **PHASE 02 IMPLEMENTED (build-verified, not yet live-tested)**
> (2026-06-30). The fork exists, builds, is published, and cinny consumes it
> (Phase 0/1). **All 7 Phase-2 EC features are implemented on the fork's `lotus`
> branch**, each additive + flag-gated, build+typecheck-clean, per-feature
> reviewed (+ a holistic multi-agent review), and pushed. **None are live-tested
> yet** — every one needs the `LOTUS_TESTING.md` §D sweep, and the **cinny host
> side must be wired** (set flags / send actions / handle call_state) — see §12.
> See **§9** Phase 0/1 results, **§10** cutover, **§11** Phase-2 seams, **§12**
> Phase-2 status + cinny integration checklist. Created 2026-06 from `LotusGuild/cinny`.
---
## 9. Phase 0 Results (verified 2026-06-29)
**Decisions taken with the user:** scope = Phase 0 recon; consumption model =
**private npm package** (§5 option 1). Recommended registry = **Gitea's built-in
npm registry** (`code.lotusguild.org`) — zero new infra.
### 9.1 Version → tag → commit mapping (LOCKED)
| Source | Value |
| :-- | :-- |
| cinny `package.json` pin | `@element-hq/element-call-embedded@0.20.1` |
| Bundle self-report (`VITE_APP_VERSION`/`appVersion`) | `embedded-v0.20.1` |
| npm registry `gitHead` for 0.20.1 | `2d74c48151d9edc01c65a22a91478aac81bf24d0` |
| GitHub tag `v0.20.1` → commit | `2d74c48…`**same commit** |
**Fork from upstream tag `v0.20.1` (commit `2d74c48`).** The embedded package
version equals the element-call release tag; repo `package.json` version is
`0.0.0` and the real version is stamped at publish time from the tag.
### 9.2 The shipped npm dist is a CLEAN upstream build
No `lotus`/`denoise`/`rnnoise` strings anywhere in
`node_modules/@element-hq/element-call-embedded/dist`. **All Lotus customization
(denoise shim) is injected at cinny build time, not baked into the package** — so
swapping the source does not disturb cinny's denoise injection layer. The
ringtone/reaction assets (`baduntss`, `cat`, `clap`, `call_declined`, …) are
upstream EC's own, not ours.
### 9.3 Build toolchain & mechanism
- **Node `24`** (`.node-version`), **pnpm `10.33.0`** (`packageManager` field,
via corepack).
- Build: **`pnpm run build:embedded`** = `vite build --config
vite-embedded.config.ts` with `NODE_OPTIONS=--max-old-space-size=16384`.
- Output dir is **repo-root `dist/`**; CI stages it into **`embedded/web/dist`**
(the `embedded/web/` dir holds the publish template: `package.json`, README,
both LICENSE files).
- Publish workflow upstream = `.github/workflows/publish-embedded-packages.yaml`:
builds → `npm version <tag> --no-git-tag-version` → `npm publish --provenance
--access public` to npmjs as `@element-hq/element-call-embedded`. (Also
Android/Maven + iOS/SwiftPM — irrelevant; we are web-only.)
### 9.4 Build reproduction — PARITY CONFIRMED
Cloned `element-call@v0.20.1` to `/root/code/element-call` (shallow), built with
isolated Node 24 / pnpm 10.33.0 (system Node 20 / cinny untouched). Result vs the
shipped npm dist:
- **137 of 147 files byte-identical** (same Vite content-hash): all CSS, fonts,
wasm, audio, JSON locale files, and `IndexedDBWorker`.
- **Only 5 JS chunks differ** (`index`, `pako.esm`, `polyfill-force`,
`rust-crypto`, `spa`) — **cause isolated to the version define**: our local
build baked `appVersion:\`dev\`` (because `VITE_APP_VERSION` was unset) vs the
npm build's `appVersion:\`embedded-v0.20.1\``. `index.html` is identical modulo
the hashed asset filenames. **Benign** — our CI sets the version from the git
tag, so a tagged CI build will match.
### 9.5 Fork CI (drafted)
`.gitea/workflows/ci.yml` is staged in the clone (models cinny's
`.gitea/workflows/ci.yml` + upstream's publish flow). Linux-only (`ubuntu-latest`)
— the Windows worker is for cinny-desktop/Tauri, not the EC web bundle. Build job
on PR/push to `lotus`; publish job on `v*` tag → `@lotusguild/element-call-embedded`
to the Gitea npm registry (needs `secrets.GITEA_NPM_TOKEN`).
### 9.6 Phase 1 — DONE (2026-06-29)
1. ✅ **Fork repo live:** `code.lotusguild.org/LotusGuild/element-call` (public,
AGPL), default branch `lotus`, full history (7018 commits) + tag `v0.20.1`.
Branch `lotus` = `v0.20.1` + 2-file diff (CI workflow + embedded package
rename).
2. ✅ **Package published:** `@lotusguild/element-call-embedded@0.20.1` on the
Gitea npm registry (published manually from the version-faithful build while
the admin token was available). **Publicly readable** (unauth `npm install`
works → devs/CI need no token to consume; only publishing needs one).
3. ✅ **cinny wired & built clean** (Node 24): `.npmrc` scope line +
`package.json` dep + `vite.config.js` `viteStaticCopy` src. `npm install`
swapped the package (resolved from Gitea), `npm run build` succeeded,
`dist/public/element-call/` populated, bundle reports `appVersion:
embedded-v0.20.1`, **denoise shim injected + all denoise assets copied**
(injection layer unchanged). **These cinny edits are staged in the working
tree, NOT committed/pushed** — pushing triggers CI → desktop → deploy, so it's
gated on the §D live test (see §10).
### 9.8 Reproducibility note (important)
A from-source rebuild is **NOT byte-identical** to upstream's npm tarball.
137/147 files match exactly (CSS, fonts, wasm, audio, worker); the 5 JS chunks
(`index`, `pako.esm`, `polyfill-force`, `rust-crypto`, `spa`) differ because the
rolldown/oxc **minifier mangles export names differently** across build
environments (and the version-define is one input). This is normal and benign —
the code is functionally equivalent. **Do not chase byte-parity; the §D live call
test is the real parity gate.**
### 9.9 Remaining follow-ups (not blocking the cutover)
- **CI publishing:** `.gitea/workflows/ci.yml` publishes on a `v*` tag but needs
(a) a Gitea Actions runner for `LotusGuild/element-call`, and (b) a **durable**
`GITEA_NPM_TOKEN` repo secret with package read/write (the admin token used for
the manual publish is being deleted, so it was deliberately NOT baked in). Until
then, publishing is manual (`npm version <tag>` in `embedded/web` →
`npm publish`).
- Decide rebase cadence vs upstream (0.20.2 / 0.20.3 already out — see §9.1).
### 9.7 Ready-to-apply artifacts (staged 2026-06-29)
**Fork side — already committed** on branch `lotus` in `/root/code/element-call`
(remote `lotus` = `code.lotusguild.org/LotusGuild/element-call.git`, push deferred
until the repo exists). Minimal 2-file diff vs tag `v0.20.1`:
`.gitea/workflows/ci.yml` (new) + `embedded/web/package.json` (rename to
`@lotusguild/element-call-embedded`). Push with:
`git push -u lotus lotus && git push lotus v0.20.1` (and tag `v0.20.1` on our side
to trigger the first publish, or push our own `v0.20.1` tag).
**cinny side — NOT yet applied** (applying before the package is published breaks
`npm ci`). Exactly 3 edits + a lockfile regen:
1. `.npmrc` — append the scoped-registry line:
```
@lotusguild:registry=https://code.lotusguild.org/api/packages/LotusGuild/npm/
```
(CI/auth: `//code.lotusguild.org/api/packages/LotusGuild/npm/:_authToken=${GITEA_NPM_TOKEN}`
— inject via env in CI, do not commit a plaintext token.)
2. `package.json:104` —
`"@element-hq/element-call-embedded": "0.20.1"` →
`"@lotusguild/element-call-embedded": "0.20.1"`.
3. `vite.config.js:25` — `viteStaticCopy` src:
`node_modules/@element-hq/element-call-embedded/dist` →
`node_modules/@lotusguild/element-call-embedded/dist`.
**`stripBase: 4` stays unchanged** — `node_modules/@lotusguild/element-call-embedded/dist`
is still exactly 4 leading segments. (Update the comment's path reference too.)
4. `package-lock.json` — regenerated by `npm install`, not hand-edited (drops the
`registry.npmjs.org/@element-hq/...` resolved URL for the Gitea one).
The denoise injection (`lotusDenoise()` in `vite.config.js`) is **unchanged** — it
keys off `dist/public/element-call/index.html`, which our fork's bundle still
produces identically (verified: `index.html` byte-identical modulo asset hashes).
---
@@ -241,12 +388,14 @@ re-homed into the fork.
- `LOTUS_TODO.md` (~line 533) calls EC a **"cross-origin iframe"** — **outdated.**
EC is **same-origin** today (self-hosted under our domain;
`iframe.sandbox` includes `allow-same-origin`; we read `contentDocument`). The
_practical_ point it makes still holds: **LiveKit's `LocalAudioTrack` lives in
EC's module scope**, not on `window`, so we can't reach it from cinny even
same-origin — which is exactly why the in-call soundboard had to be
local-playback-only, and another reason to fork (a fork could expose a real
audio-inject API).
`iframe.sandbox` includes `allow-same-origin`; we read `contentDocument`), and
**as of 2026-06-29 we own the fork's source** (`@lotusguild/element-call-embedded`).
The _practical_ point it made still holds *until we ship the audio-inject API*:
**LiveKit's `LocalAudioTrack` lives in EC's module scope**, not on `window`, so
cinny can't reach it even same-origin — which is why the in-call soundboard had
to be local-playback-only. **The fork removes this wall:** EC can expose a real
`io.lotus.inject_audio` widget action (Phase 2) that mixes into the published
track from inside its own module scope.
- `LOTUS_FEATURES.md` documents the EC upgrade history (0.16.3 → 0.19.4 →
0.20.1), the dark-mode CSS injection, and AFK auto-mute — all relevant prior
art for what the fork must preserve.
@@ -268,3 +417,236 @@ re-homed into the fork.
`LOTUS_TODO.md` (denoise/soundboard constraints), `LOTUS_FEATURES.md` (EC history),
`LOTUS_TESTING.md` §D (regression sweep). Infra: `/root/code/matrix` (`livekit/`,
`deploy/`).
---
## 10. Live cutover — the remaining steps (Phase 1 finish)
The fork is published and cinny builds against it locally (§9.6). What's left to
go live:
1. **Run `LOTUS_TESTING.md` §D** against a local cinny build (`npm run build` is
already proven; serve `dist/` or `npm run dev`). Verify a real call: join,
mic/cam, screenshare, theme sync, denoise on, widget hangup — web first.
2. **Commit the cinny edits** (currently staged, uncommitted in the working tree):
`.npmrc`, `package.json`, `package-lock.json`, `vite.config.js`. Suggested
message: `chore(call): consume self-built @lotusguild/element-call-embedded`.
3. **Push to `lotus`** → cinny CI builds, then `trigger-desktop` bumps
cinny-desktop → Tauri release. Re-run §D on **cinny-desktop** (the path where
the old `stripBase` bug bit — verify the widget loads, not a 404).
4. Only then start **Phase 2** (A5/A6/A7, theming, denoise-in-source).
---
## 11. Phase 2 — implementation seams (mapped 2026-06-29)
The exact integration points for each Phase 2 item, found by reading the EC fork
+ cinny source. **All of these are media-path / in-call features that cannot be
functionally verified without a live Matrix + LiveKit call** — implement each as
a minimal, **feature-flagged, additive** diff (no behavior change unless cinny
opts in), build-verify the fork (`pnpm build:embedded`, ~15s) AND cinny
(`npm run build`), then gate shipping on `LOTUS_TESTING.md` §D.
**Shared widget channel (the backbone for #2/#3/#4/#7):**
- EC→cinny: `widget.api.transport.send("io.lotus.<x>", data)` (see
`element-call/src/widget.ts`).
- cinny→EC actions: add the action name to the `lazyActions` allow-list in
`widget.ts` (the array at ~L101) and handle it in EC; cinny sends via
`this.call.transport.send(...)`.
- cinny receives EC→cinny actions via the existing `listenAction(type, cb)`
helper in `plugins/call/CallEmbed.ts:626` (auto-replies `{}` so the transport
doesn't time out — same pattern as `io.element.device_mute`).
**#2 mute/speaker events** — Source: subscribe to `vm.userMedia$`
(`CallViewModel`), per member `speaking$` + `audioEnabled$`
(`state/media/UserMediaViewModel.ts:47-48`); aggregate and
`transport.send("io.lotus.call_state", {participants:[{id,speaking,audioEnabled}]})`.
Mount in `room/InCallView.tsx` via `useEffect` guarded by `widget !== null`.
cinny: `listenAction("io.lotus.call_state")` in `CallEmbed.ts`, feed
`hooks/useCallSpeakers.ts` → delete its `contentDocument` `[data-muted]` /
`[data-video-fit]` scrape. *Additive, low risk.*
**#4 spotlight/focus** — EC: add `io.lotus.focus_participant` to the `lazyActions`
list (`widget.ts`), drive `vm`'s spotlight (`spotlightSpeaker$` /
`spotlight$` in `CallViewModel.ts:898/1001`) to pin a given identity, coexisting
with `hasRemoteScreenShares$` (L1008). cinny: replace
`CallControl.ts` `focusCameraParticipant` `.click()` walk with
`transport.send("io.lotus.focus_participant", {userId})`. *Additive, low risk.*
**#3 audio-inject** — EC: add `io.lotus.inject_audio` action; mix an
`AudioBufferSourceNode` into the published mic track. The local publish path is
`state/CallViewModel/localMember/Publisher.ts` + `LocalMember.ts` (LiveKit
`localParticipant`); create a `MediaStreamAudioDestinationNode`, mix mic + clip,
`replaceTrack`. cinny soundboard calls the action instead of local-only playback.
*Medium; touches publish path → live-test carefully.*
**#1 denoise-in-source** — replace the cinny `lotusDenoise()` `getUserMedia`
monkeypatch with a real processing stage in EC's mic capture
(`Publisher.ts`/`LocalMember.ts`; note EC has a `TrackProcessorContext` +
`BlurBackgroundTransformer` precedent in `livekit/`). EC re-runs it on every
(re)publish → fixes A7. Remove `vite.config.js` `lotusDenoise()` + URL params in
`CallEmbed.ts`; move `denoise/` assets into the fork. *Highest value, highest
risk — most live testing.*
**#5 theming** — add a Lotus/TDS theme in EC's theme system (`src/useTheme.ts` +
EC theme tokens / CSS); driven by the existing `setTheme()` channel cinny already
calls (`CallEmbed.ts:277`). Bake transparent background. Delete cinny's
`applyStyles()` injection + `background:none !important`. *Medium.*
**#6 in-call decorations** — render the decoration APNG in EC's tile component
(`tile/GridTile.tsx`); pass slugs via widget member data. cinny already has the
decoration data + `AvatarDecoration` (lobby `CallMemberCard.tsx`). *Medium-Large.*
**#7 quality controls** — set audio `maxBitrate` via
`RTCRtpSender.setParameters` and screenshare `getDisplayMedia` constraints in
EC's publish path (`Publisher.ts`); configurable via `config.json` / a widget
message. Keep the server `voice-limit-guard` as enforcement. *Medium.*
**Rollback:** revert the 4 cinny files (restores `@element-hq/...@0.20.1` from
npmjs). The fork repo/package can stay; nothing else depends on it until pushed.
### Local repro/build environment (this session, 2026-06-29)
- Upstream cloned + our `lotus` branch at `/root/code/element-call` (remote
`lotus` → Gitea; origin → github upstream, now un-shallowed/full history).
- Isolated **Node 24.18.0** lives in the session scratchpad (system Node is 20);
cinny's `.node-version` is `24.13.1`, so use Node 24 to build cinny too.
- Build the embedded bundle: in `/root/code/element-call`, with Node 24 + pnpm
10.33.0 on PATH, `VITE_APP_VERSION=embedded-v0.20.1 pnpm run build:embedded`
→ output in `dist/`; stage to `embedded/web/dist` before publishing.
---
## 12. Phase 2 — IMPLEMENTED on the fork (2026-06-30)
All 7 EC features are on the `lotus` branch of `LotusGuild/element-call`, each
**additive + feature-flagged** (a vanilla call with no `lotus*` params / no Lotus
actions behaves exactly like upstream), build + `tsc` clean, per-feature reviewed
(fixes applied) and holistically reviewed. **Not yet live-tested** — all need the
`LOTUS_TESTING.md` §D sweep.
Fork modules live under `element-call/src/lotus/*`; mounts are `useEffect`s in
`src/room/InCallView.tsx`. Custom widget actions are in `src/lotus/lotusActions.ts`
(toWidget ones allow-listed in `src/widget.ts`).
| # | Feature | Enable via | EC module |
| :- | :- | :- | :- |
| 2 | Speaker/mute/camera state → host | URL `lotusCallState=1` | `lotusCallState.ts` (sends `io.lotus.call_state`) |
| 4 | Focus/spotlight a participant (works during screenshare) | action `io.lotus.focus_participant {userId|null}` | `lotusFocus.ts` + `CallViewModel` spotlight override |
| 3 | Soundboard audio-inject (heard by peers) | URL `lotusAudioInject=1` + action `io.lotus.inject_audio {url,volume?}` | `lotusAudioInject.ts` |
| 7 | Audio/screenshare quality caps | action `io.lotus.set_quality {audioMaxBitrate?,screenshareMaxBitrate?,screenshareMaxFramerate?}` | `lotusQuality.ts` |
| 5 | Transparent bg + Lotus theme | URL `lotusTransparent=1` / `lotusTheme=1` | `useTheme.ts` + `index.css` |
| 6 | In-call avatar decorations | action `io.lotus.decorations {decorations:{userId:url}}` | `lotusDecorations.ts` + `MediaView.tsx` |
| 1 | ML denoise in-source (fixes A7) | URL **`lotusDenoiseSource=1`** (+`lotusModel`,`lotusGate`,`lotusGateThreshold`,`lotusDenoiseBase`) — deliberately NOT the existing `lotusDenoise=ml` (that drives the host shim; reusing it would double-process) | `lotusDenoise.ts` + `lotusDenoiseProcessor.ts` |
**Security hardening applied** (holistic audit): `lotusDenoiseBase` forced
same-origin before `audioWorklet.addModule` (was an arbitrary-code-load vector
via a crafted link); audio-inject gated behind `lotusAudioInject=1`; decoration
roster capped. Only `https`/`blob` URLs accepted for inject/decoration assets.
### 12.1 cinny host integration checklist (REQUIRED to light these up)
The EC side is additive and dormant until cinny opts in. Host work needed (in
`src/app/plugins/call/CallEmbed.ts` unless noted):
> ⚠️ **CRITICAL TIMING (protocol audit F1):** only send `io.lotus.*` **toWidget**
> actions (#3 focus, #6 decorations, #7 quality, audio-inject) **after** the call
> is joined (`CallEmbed.onCallJoined` / `this.joined`). Those actions are
> allow-listed at EC app-init (so `preventDefault` suppresses the auto-error)
> but their handlers only mount with `InCallView` (post-join). Sending earlier
> leaves the host's `transport.send` pending until the **10s timeout**. Queue and
> flush on join, or no-op before join.
>
> Also: **F3** — the fork implements only `rnnoise`/`speex`; cinny's `dtln`/
> `deepfilternet` selections silently fall back to rnnoise (now logged). Restrict
> the embedded-call model picker to rnnoise/speex, or implement the others in
> `lotusDenoiseProcessor.ts`. **F4** — cinny sends `lotusNativeNS`, which the
> fork ignores; drop it or wire it in. **F7** — no widget *capability* changes
> needed; custom actions bypass capability checks.
1. **Set the URL flags** on the widget iframe params (the `URLSearchParams` in
`CallEmbed`): `lotusCallState=1`, `lotusTransparent=1`/`lotusTheme=1`,
`lotusAudioInject=1` as desired. (Denoise already sets `lotusDenoise=ml` etc.)
2. **Ack `io.lotus.call_state`**: add `listenAction('io.lotus.call_state', …)` —
without a reply the fork's sends time out every 250ms. Feed the payload into
`useCallSpeakers` and RETIRE its `contentDocument` DOM scrape.
3. **Send actions** via `this.call.transport.send(...)`:
`io.lotus.focus_participant` (replace `CallControl.focusCameraParticipant`s
`.click()`), `io.lotus.inject_audio` (from the soundboard), `io.lotus.set_quality`
(from quality settings), `io.lotus.decorations` (push the MSC4133 decoration
map; resolve mxc→https first).
4. **#1 denoise cutover**: once verified, STOP injecting the `lotusDenoise()`
shim in `cinny/vite.config.js` and remove the `index.html` injection — the
fork now does denoise in-source. Keep shipping the `denoise/` assets (the
fork loads `./denoise/…` at runtime) until those move into the fork build.
5. Re-run `LOTUS_TESTING.md` §D for each feature; only then ship.
### 12.2 Holistic multi-agent review — outstanding follow-ups (non-blocking)
Four aspect-agents reviewed the whole fork. Criticals were fixed in-branch (the
denoise restart-silence/A7 bug; the `lotusDenoiseBase` code-load vector;
audio-inject opt-in gate; #6 rendering in the wrong component; #7 simulcast cap).
Remaining, deliberately deferred:
- **Denoise H2 (double-processing):** if cinny is set to `lotusDenoise=ml` while
ALSO still injecting its build-time `getUserMedia` shim, audio is denoised
twice. The #1 cutover MUST remove the cinny-side injection (it currently has
none injected into the iframe — keep it that way). Hard requirement, not code.
- **Denoise M1 (perf):** in-source uses non-SIMD `rnnoise.wasm`; the reference
preferred SIMD with detection. Perf-only; add SIMD detection later.
- **dtln/deepfilternet (F3): RESOLVED** — all four models
(rnnoise/speex/dtln/deepfilternet) are now implemented in
`lotusDenoiseProcessor.ts` (faithful port of cinny's `build/lotus-denoise.js`
pipeline). This also fixed a real bug (the gate worklet name was `noiseGate`;
correct is the hyphenated `noise-gate`) and added per-model sample rates
(DTLN 16 kHz, others 48 kHz), context `resume()`, and SIMD wasm selection.
Still needs live §D testing per model, and depends on cinny shipping the
DTLN (`denoise/workadventure/`) + DeepFilterNet (`denoise/deepfilternet/`)
asset trees (it already does).
- **Rebase-fragility (build agent MED):** the `CallViewModel` spotlight override
edits hot upstream lines (renamed `spotlightSpeaker$`→`autoSpotlightSpeaker$`).
For cheaper future rebases, refactor it into a `src/lotus/lotusSpotlight.ts`
wrapper that takes the upstream stream and returns the overridden one, leaving
upstream's definition byte-identical (a single import + two token swaps).
- **Denoise asset coupling (build agent HIGH):** the fork loads `./denoise/*`
shipped by cinny, not by the fork build (documented in the processor). Add an
integration smoke-check that `GET …/element-call/denoise/rnnoise.wasm` == 200,
and pin the `@sapphi-red/web-noise-suppressor` version both repos expect.
- **Unconditional effect registration (build agent LOW):** focus/audio-inject/
quality/decorations register widget handlers on every embedded call (true
no-ops for a non-Lotus host). Intentional; gate behind a coarse `lotus=1` flag
if strict zero-footprint is desired.
- **Privacy (security agent):** decoration/inject URLs accept any `https`; ideally
restrict to the homeserver media origin host-side. Call-state exposes
userId/deviceId/speaking to the (trusted, same-origin) host — documented.
**Nothing here blocks the §D live test — but every feature still needs it.**
### 12.3 Safe rollout when prod is the only test environment
Every Phase-2 feature is now **dormant by default** — with the flags cinny sets
today, the fork behaves identically to the parity build (`#1` was decoupled onto
`lotusDenoiseSource=1` so it no longer collides with the host's `lotusDenoise=ml`
shim). This enables a low-risk incremental rollout even without a staging env:
1. **Ship dormant first.** Publish the `lotus` branch (e.g. `0.20.1-lotus.1`),
bump cinny's pin, deploy. With no Lotus flags set / no Lotus actions sent,
this is upstream-equivalent (only inert, holistically-reviewed code runs).
"Testing" here = confirm a normal call still works.
2. **Enable ONE feature at a time**, each independently revertable:
- URL-flag features (#2 `lotusCallState`, #5 `lotusTransparent`/`lotusTheme`,
#1 `lotusDenoiseSource`): add the flag in `CallEmbed.getWidget`, deploy,
test that one feature, roll back just that flag if needed.
- Action features (#3,#4,#6,#7): wire the host send + (for #2) the
`listenAction` ack, gated on join (§12.1 F1).
3. **#1 denoise cutover is a coordinated 2-step** (do together): set
`lotusDenoiseSource=1` AND remove the `lotusDenoise()` shim injection +
`lotusDenoise=ml` param in cinny — otherwise audio is denoised twice.
Roll back = revert both.
4. Baseline is always upstream-equivalent, so any single feature can be disabled
by flipping its flag/send off without touching the rest.
**Blocker to step 1:** publishing the `lotus` branch needs a Gitea npm token
(the admin token used for the `0.20.1` parity publish was deleted). Either
provide a token for a manual `npm publish`, or stand up the Gitea Actions runner
+ `GITEA_NPM_TOKEN` secret so a `v0.20.1-lotus.1` tag auto-publishes.
+28 -18
View File
@@ -36,30 +36,40 @@ Implemented and gate-green; confirm each per `LOTUS_TESTING.md`, then delete the
---
## 🧩 Known Element Call iframe limitations (not fixable from our side)
## 🧩 Element Call source-level items — now actionable via the fork
> 🔱 **[EC-FORK]** These are the motivating issues for the **Element Call fork
> initiative** — see [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md).
> Once we build EC from our own source, A5/A6/A7 below become normal code fixes.
> (Correction: the iframe is actually **same-origin** / self-hosted — we just
> don't own EC's compiled source today.)
> 🔱 **[EC-FORK]** **UPDATE 2026-06-29: the fork is live.** We now own and
> self-build Element Call (`LotusGuild/element-call` →
> `@lotusguild/element-call-embedded`, Phase 1 done & cinny wired). A5/A6/A7
> below are **no longer "won't fix"** — they are ordinary source changes. See
> [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md) §10 + the Phase
> 2 work list. (The iframe is **same-origin** / self-hosted; the old blocker was
> that we didn't own EC's compiled source — which we now do.)
The in-call participant grid is rendered **inside EC's iframe** (a pre-built npm
bundle we don't own), which we can style/place around but cannot change the logic
of. Consequences from testing:
The in-call participant grid is rendered **inside EC's app**. Previously a
pre-built npm bundle we could only style/place around; now editable source.
Items from testing, with their fork-level fix path:
- **A5 — "Focus camera":** EC already supports native tile-pinning (click a video
tile). Our bottom-bar "Focus camera" is a programmatic wrapper that clicks that
tile; it can't live inside EC's UI. During a screenshare EC spotlights the
shared screen and a camera pin may not override it. _Decision: keep the
shortcut, revisit with the larger call-UI/EC work._
- **A5 — "Focus camera":** EC supports native tile-pinning. Our bottom-bar "Focus
camera" is a programmatic wrapper that **`.click()`s the tile** today
(`CallControl.ts` `focusCameraParticipant`), and during a screenshare EC
spotlights the shared screen so a camera pin may not override it. **Fork fix:**
add an `io.lotus.focus_participant` widget action that pins a participant in
EC's layout (coexisting with / overriding the screenshare spotlight); cinny
sends it via the widget API and the DOM-click hack is deleted. _Status: Open —
Actionable (Phase 2)._
- **A6 — avatar decorations in-call:** decorations render on **our** pre-join
lobby roster (`CallMemberCard`) but cannot be drawn on EC's in-call video
tiles. Working as designed given the iframe boundary.
lobby roster (`CallMemberCard`) but not on EC's in-call video tiles. **Fork
fix:** render the decoration APNG inside EC's participant-tile component, fed
decoration slugs via widget member data. _Status: Open — Actionable (Phase 2)._
- **A7 — mic dead after EC's "Reconnect":** the mid-call "Connection lost /
Reconnect" screen is **EC's own** (our load watchdog only covers an initial
hung load). After EC reconnects, the mic isn't re-published through our denoise
`getUserMedia` shim until a clean End+rejoin. Tied to the denoise rework.
`getUserMedia` shim until a clean End+rejoin. **Fork fix:** move denoise into
EC's mic-capture/publish pipeline as a first-class audio stage — EC re-runs it
on every (re)publish, so reconnects keep denoise alive natively, and the
build-time `index.html` injection is removed. _Status: Open — Actionable
(Phase 2); root cause is the `getUserMedia` monkeypatch, not EC itself._
---
@@ -67,7 +77,7 @@ of. Consequences from testing:
### Calls / Audio
- **N127 — ML denoise shim is never injected in `vite dev`.** The `lotusDenoise` plugin injects only on `closeBundle` (build), so ML noise suppression is silently inactive during local dev. Add a dev-mode injection (`configureServer` / `transformIndexHtml`). Dev-only impact.
- **N127 — ML denoise shim is never injected in `vite dev`.** The `lotusDenoise` plugin injects only on `closeBundle` (build), so ML noise suppression is silently inactive during local dev. Add a dev-mode injection (`configureServer` / `transformIndexHtml`). Dev-only impact. _Note: this **dissolves entirely** once denoise moves in-source in the fork (A7 fix) — there is then no build-time injection to be missing in dev._
### Security & Privacy
+7 -7
View File
@@ -77,7 +77,7 @@
"ua-parser-js": "2.0.10"
},
"devDependencies": {
"@element-hq/element-call-embedded": "0.20.1",
"@lotusguild/element-call-embedded": "0.20.1-lotus.1",
"@rollup/plugin-inject": "5.0.5",
"@rollup/plugin-wasm": "6.2.2",
"@types/chroma-js": "3.1.2",
@@ -1788,12 +1788,6 @@
"node": ">=v18"
}
},
"node_modules/@element-hq/element-call-embedded": {
"version": "0.20.1",
"resolved": "https://registry.npmjs.org/@element-hq/element-call-embedded/-/element-call-embedded-0.20.1.tgz",
"integrity": "sha512-ODg2r7UmR8UjRpapLKbn6v1PS8fu/r58zdbvXMYaAlUEAC2f6L/9Moc9S4noG1+ARgWxY+m2vLmNDK9G9uFZYQ==",
"dev": true
},
"node_modules/@emnapi/core": {
"version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz",
@@ -2693,6 +2687,12 @@
"resolved": "https://registry.npmjs.org/@juggle/resize-observer/-/resize-observer-3.4.0.tgz",
"integrity": "sha512-dfLbk+PwWvFzSxwk3n5ySL0hfBog779o8h68wK/7/APo/7cgyWp5jcXockbxdk5kFRkbeXWm4Fbi9FrdN381sA=="
},
"node_modules/@lotusguild/element-call-embedded": {
"version": "0.20.1-lotus.1",
"resolved": "https://code.lotusguild.org/api/packages/LotusGuild/npm/%40lotusguild%2Felement-call-embedded/-/0.20.1-lotus.1/element-call-embedded-0.20.1-lotus.1.tgz",
"integrity": "sha512-hy1KEnFw4MuwvlactUFPPvvtPZh1y56JMK/ehnficUmJNwdJsOhSwThaYp35RZ/ar6RCuiW86yQqlQBOSpZJVQ==",
"dev": true
},
"node_modules/@matrix-org/matrix-sdk-crypto-wasm": {
"version": "18.3.0",
"resolved": "https://registry.npmjs.org/@matrix-org/matrix-sdk-crypto-wasm/-/matrix-sdk-crypto-wasm-18.3.0.tgz",
+1 -1
View File
@@ -101,7 +101,7 @@
"ua-parser-js": "2.0.10"
},
"devDependencies": {
"@element-hq/element-call-embedded": "0.20.1",
"@lotusguild/element-call-embedded": "0.20.1-lotus.1",
"@rollup/plugin-inject": "5.0.5",
"@rollup/plugin-wasm": "6.2.2",
"@types/chroma-js": "3.1.2",
+3
View File
@@ -17,6 +17,7 @@ import { useRoomPermissions } from '../../hooks/useRoomPermissions';
import { useMatrixClient } from '../../hooks/useMatrixClient';
import { StateEvent } from '../../../types/matrix/room';
import { useCallMembers, useCallSession } from '../../hooks/useCall';
import { LotusDecorationPusher } from '../lotus/LotusDecorationPusher';
import { useStateEvent } from '../../hooks/useStateEvent';
import { VoiceLimitContent } from '../common-settings/general/RoomVoiceLimit';
import { CallMemberRenderer } from './CallMemberCard';
@@ -199,6 +200,8 @@ function CallJoined({ joined, containerRef }: CallJoinedProps) {
<Box grow="Yes" direction="Column">
<Box grow="Yes" ref={containerRef} />
{callEmbed && joined && <CallControls callEmbed={callEmbed} />}
{/* [lotus #6] push avatar decorations to EC's in-call tiles (post-join) */}
{callEmbed && joined && <LotusDecorationPusher callEmbed={callEmbed} />}
</Box>
);
}
@@ -0,0 +1,99 @@
import React, { useCallback, useEffect, useMemo, useRef, type ReactElement } from 'react';
import { type CallEmbed } from '../../plugins/call';
import { useCallMembers, useCallSession } from '../../hooks/useCall';
import { useAvatarDecoration } from '../../hooks/useAvatarDecoration';
import { decorationUrl } from './avatarDecorations';
/**
* [lotus #6] Pushes each call participant's avatar-decoration image URL to the
* forked Element Call (`io.lotus.decorations`), which renders it on the in-call
* video-tile avatars. Mounted only while joined, so the EC-side handler exists.
*
* The decoration roster is per-user slugs resolved via `useAvatarDecoration`;
* we render one invisible probe per member to reuse that hook + its cache, then
* debounce-send the aggregated `{ userId: url }` map whenever it changes.
*/
function DecorationProbe({
userId,
onResolve,
}: {
userId: string;
onResolve: (userId: string, url: string | null) => void;
}): null {
const slug = useAvatarDecoration(userId);
useEffect(() => {
onResolve(userId, slug ? decorationUrl(slug) : null);
}, [userId, slug, onResolve]);
return null;
}
export function LotusDecorationPusher({ callEmbed }: { callEmbed: CallEmbed }): ReactElement {
const session = useCallSession(callEmbed.room);
const members = useCallMembers(session);
const map = useRef<Map<string, string>>(new Map());
const pushTimer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
const userIds = useMemo(
() => Array.from(new Set(members.map((m) => m.userId).filter((u): u is string => !!u))),
[members],
);
const push = useCallback(() => {
const decorations: Record<string, string> = {};
map.current.forEach((url, userId) => {
decorations[userId] = url;
});
void callEmbed.call.transport
.send('io.lotus.decorations', { decorations })
.catch(() => undefined);
}, [callEmbed]);
const schedulePush = useCallback(() => {
if (pushTimer.current) clearTimeout(pushTimer.current);
pushTimer.current = setTimeout(push, 300);
}, [push]);
const onResolve = useCallback(
(userId: string, url: string | null) => {
const prev = map.current.get(userId);
if (url) {
if (prev !== url) {
map.current.set(userId, url);
schedulePush();
}
} else if (prev !== undefined) {
map.current.delete(userId);
schedulePush();
}
},
[schedulePush],
);
// Drop decorations for participants who left the call.
useEffect(() => {
const present = new Set(userIds);
let changed = false;
map.current.forEach((_url, userId) => {
if (!present.has(userId)) {
map.current.delete(userId);
changed = true;
}
});
if (changed) schedulePush();
}, [userIds, schedulePush]);
useEffect(
() => () => {
if (pushTimer.current) clearTimeout(pushTimer.current);
},
[],
);
return (
<>
{userIds.map((userId) => (
<DecorationProbe key={userId} userId={userId} onResolve={onResolve} />
))}
</>
);
}
+27
View File
@@ -35,6 +35,19 @@ export const useCallSpeakers = (callEmbed: CallEmbed): Set<string> => {
callEmbed.iframe.contentDocument ?? callEmbed.iframe.contentWindow?.document ?? undefined;
const syncState = (): void => {
// [lotus #2] Prefer the fork's io.lotus.call_state events over scraping
// EC's rendered DOM. Falls back to the DOM path below when the fork hasn't
// sent yet (null) OR sent a spurious empty list (you're always present in
// your own joined call, so [] means "no usable data", not "nobody").
const lotus = callEmbed.getLotusParticipants();
if (lotus !== null && lotus.length > 0) {
const ls = new Set<string>();
lotus.forEach((p) => {
if (p.speaking && isUserId(p.userId)) ls.add(p.userId);
});
setSpeakers(ls);
return;
}
const doc = getDoc();
if (!doc) {
setSpeakers(new Set<string>());
@@ -91,6 +104,8 @@ export const useCallSpeakers = (callEmbed: CallEmbed): Set<string> => {
};
attachObserver();
// [lotus #2] Re-derive whenever the fork pushes new call-state.
const unsubLotus = callEmbed.onLotusCallState(syncState);
// If iframe isn't ready yet, wait for body to be available.
let bodyWatcher: MutationObserver | undefined;
@@ -109,6 +124,7 @@ export const useCallSpeakers = (callEmbed: CallEmbed): Set<string> => {
return () => {
tileObserver?.disconnect();
bodyWatcher?.disconnect();
unsubLotus();
};
}, [callEmbed, callMembers, joined]);
@@ -137,6 +153,14 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
const localUserId = callEmbed.room.client?.getUserId() ?? '';
const syncState = (): void => {
// [lotus #2] Prefer the fork's io.lotus.call_state over DOM scraping;
// ignore a spurious empty list (fall back to DOM).
const lotus = callEmbed.getLotusParticipants();
if (lotus !== null && lotus.length > 0) {
const remote = lotus.filter((p) => p.userId !== localUserId);
setMuted(remote.length > 0 && remote.every((p) => !p.audioEnabled));
return;
}
const doc = getDoc();
if (!doc) {
setMuted(false);
@@ -190,6 +214,8 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
};
attachObserver();
// [lotus #2] Re-derive whenever the fork pushes new call-state.
const unsubLotus = callEmbed.onLotusCallState(syncState);
// If iframe isn't ready yet, wait for body to be available.
let bodyWatcher: MutationObserver | undefined;
@@ -208,6 +234,7 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
return () => {
tileObserver?.disconnect();
bodyWatcher?.disconnect();
unsubLotus();
};
}, [callEmbed]);
+13 -64
View File
@@ -346,71 +346,20 @@ export class CallControl extends EventEmitter implements CallControlState {
* them yet).
*/
public focusCameraParticipant(userId: string): void {
const doc = this.document;
if (!doc) return;
// [lotus #4] Pin the participant via the fork's widget action instead of
// DOM-poking tiles. EC's layout honors it — including surfacing the camera
// alongside a screenshare (A5) — and it's version-stable. The fork always
// acks, so the promise resolves regardless.
void this.call.transport
.send('io.lotus.focus_participant', { userId })
.catch(() => undefined);
}
// EC labels participant tiles inconsistently across versions — the user's
// matrix id may be the full aria-label, a substring of it, or carried on a
// data attribute (and sometimes the visible label is the display name, not
// the id at all). Try several strategies before giving up, then walk up to
// the enclosing video tile.
const findTile = (): HTMLElement | undefined => {
const escaped = CSS.escape(userId);
const el =
doc.querySelector<HTMLElement>(`[aria-label="${escaped}"]`) ??
doc.querySelector<HTMLElement>(`[data-testid="videoTile"][aria-label*="${escaped}"]`) ??
doc.querySelector<HTMLElement>(`[aria-label*="${escaped}"]`) ??
doc.querySelector<HTMLElement>(`[data-member-id="${escaped}"]`) ??
doc.querySelector<HTMLElement>(`[data-id="${escaped}"]`) ??
undefined;
return (
el?.closest<HTMLElement>('[data-testid="videoTile"]') ??
el?.closest<HTMLElement>('[data-video-fit]') ??
el ??
undefined
);
};
const applyFocus = () => {
const tile = findTile();
if (tile) {
tile.click();
} else if (import.meta.env.DEV) {
console.warn(`[CallControl] focusCameraParticipant: no tile matched ${userId}`);
}
};
if (this.spotlight) {
// Already in spotlight — pin immediately.
applyFocus();
return;
}
// Switching to spotlight re-renders EC's layout asynchronously; clicking the
// tile in the same tick would land in the old (grid) DOM. A fixed frame
// delay is unreliable (EC's React commit can exceed it on slow devices), so
// watch the iframe DOM for a spotlight video tile to mount, then focus —
// with a hard timeout so the click is always attempted at least once.
this.spotlightButton?.click();
const tileSelector = '[data-testid="videoTile"]';
let settled = false;
let observer: MutationObserver | undefined;
let timer: ReturnType<typeof setTimeout> | undefined;
const finish = () => {
if (settled) return;
settled = true;
if (timer) clearTimeout(timer);
observer?.disconnect();
applyFocus();
};
observer = new MutationObserver(() => {
if (doc.querySelector(tileSelector)) finish();
});
observer.observe(doc.body, { childList: true, subtree: true });
timer = setTimeout(finish, 600);
// A tile may already be present immediately after toggling spotlight.
if (doc.querySelector(tileSelector)) finish();
/** [lotus #4] Clear any manual spotlight pin and return to speaker-follows. */
public clearFocusParticipant(): void {
void this.call.transport
.send('io.lotus.focus_participant', { userId: null })
.catch(() => undefined);
}
public dispose() {
+58 -6
View File
@@ -36,6 +36,15 @@ const CALL_LOAD_WATCHDOG_MS = 25_000;
export type CallLoadErrorReason = 'timeout' | 'iframe';
/** Payload entry of the fork's io.lotus.call_state widget event (#2). */
export interface LotusCallParticipant {
id: string;
userId: string;
speaking: boolean;
audioEnabled: boolean;
videoEnabled: boolean;
}
export class CallEmbed {
private mx: MatrixClient;
@@ -47,6 +56,13 @@ export class CallEmbed {
public joined = false;
// [lotus #2] Latest per-participant state from io.lotus.call_state, or null
// until the fork sends the first one. When non-null, the speaker/mute hooks
// read it instead of scraping the EC iframe DOM.
private lotusParticipants: LotusCallParticipant[] | null = null;
private lotusCallStateListeners = new Set<() => void>();
public readonly control: CallControl;
private readonly container: HTMLElement;
@@ -148,20 +164,30 @@ export class CallEmbed {
perParticipantE2EE: room.hasEncryptionStateEvent().toString(),
lang: 'en-EN',
theme: themeKind,
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml' we
// disable it here so EC doesn't do its own extra processing, and let the
// Lotus denoise shim (which keeps native NS on) handle the pipeline.
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml'
// we disable it so EC captures a raw mic and the fork's in-source denoise
// TrackProcessor (lotusDenoiseSource) handles the pipeline.
noiseSuppression: (denoiseMode === 'browser').toString(),
audio: initialAudio.toString(),
video: initialVideo.toString(),
header: 'none',
// [lotus] Activate the self-built fork's in-source features (each is a
// no-op on the EC side unless its flag/action is present):
// - call-state stream (speaking/mute events) -> useCallSpeakers
// - transparent background so the room wallpaper shows through natively
lotusCallState: 'true',
lotusTransparent: 'true',
});
if (denoiseMode === 'ml') {
// Signal the Lotus denoise shim to route the mic through the ML processors.
params.append('lotusDenoise', 'ml');
// [lotus] In-source ML denoise: the fork runs RNNoise/Speex/DTLN/DFN as a
// real LiveKit audio TrackProcessor (survives reconnects — fixes A7),
// replacing the old build-time getUserMedia shim. The shim injection was
// removed from vite.config.js; the denoise/ assets are still shipped and
// loaded by the processor. lotusDenoiseSource (not lotusDenoise=ml) gates
// it so the two engines can never both run.
params.append('lotusDenoiseSource', 'true');
params.append('lotusModel', denoiseModel);
params.append('lotusNativeNS', denoiseNativeNS.toString());
params.append('lotusGate', denoiseGate.toString());
params.append('lotusGateThreshold', denoiseGateThreshold.toString());
}
@@ -318,6 +344,18 @@ export class CallEmbed {
this.disposables.push(
this.listenAction(WidgetApiFromWidgetAction.UpdateAlwaysOnScreen, () => {}),
);
// [lotus #2] Consume the fork's per-participant call-state stream. listenAction
// auto-replies {} so the fork's transport doesn't time out. Stored for the
// speaker/mute hooks (which prefer this over DOM scraping).
this.disposables.push(
this.listenAction('io.lotus.call_state', (evt) => {
const data = (evt.detail as { data?: { participants?: unknown } } | undefined)?.data;
this.lotusParticipants = Array.isArray(data?.participants)
? (data!.participants as LotusCallParticipant[])
: [];
this.lotusCallStateListeners.forEach((l) => l());
}),
);
// Populate the map of "read up to" events for this widget with the current event in every room.
// This is a bit inefficient, but should be okay. We do this for all rooms in case the widget
@@ -623,6 +661,20 @@ export class CallEmbed {
}
}
/** [lotus #2] Latest io.lotus.call_state participants, or null if the fork
* hasn't sent any yet (callers then fall back to DOM scraping). */
public getLotusParticipants(): LotusCallParticipant[] | null {
return this.lotusParticipants;
}
/** [lotus #2] Subscribe to io.lotus.call_state updates. Returns an unsubscribe. */
public onLotusCallState(cb: () => void): () => void {
this.lotusCallStateListeners.add(cb);
return () => {
this.lotusCallStateListeners.delete(cb);
};
}
public listenAction<T>(type: string, callback: (event: CustomEvent<T>) => void) {
const wrapped = (ev: CustomEvent<T>) => {
ev.preventDefault();
+12 -30
View File
@@ -16,13 +16,15 @@ const copyFiles = {
// widget URL (/public/element-call/index.html) resolves. v4.x of
// vite-plugin-static-copy preserves the full source path under dest, so
// we strip the 4 leading segments of the source base
// (node_modules/@element-hq/element-call-embedded/dist) — mirroring the
// (node_modules/@lotusguild/element-call-embedded/dist) — mirroring the
// stripBase pattern used by the android/locales targets below. The old
// `rename: 'element-call'` form silently produced
// public/node_modules/.../dist/ under v4.x, 404ing the widget (calls
// broke on cinny-desktop; web only worked because its deployed copy was
// a stale artifact from before the vite-plugin-static-copy v4 bump).
src: 'node_modules/@element-hq/element-call-embedded/dist',
// Source is our self-built fork (LotusGuild/element-call) published to
// the Gitea npm registry; see HANDOFF_ELEMENT_CALL_FORK.md.
src: 'node_modules/@lotusguild/element-call-embedded/dist',
dest: 'public/element-call',
rename: { stripBase: 4 },
},
@@ -159,34 +161,14 @@ function lotusDenoise() {
fs.copyFileSync(s, d);
});
const shimSrc = path.resolve('build/lotus-denoise.js');
if (!fs.existsSync(shimSrc)) {
throw new Error(`[lotus-denoise] Missing shim source ${shimSrc} — build aborted.`);
}
fs.copyFileSync(shimSrc, path.join(ecDir, 'lotus-denoise.js'));
// Inject the shim <script> into Element Call's index.html so it runs
// before EC captures the mic. Verify the injection actually landed —
// if EC's bundle ever drops its deferred module entry the replace would
// no-op and ML would silently never engage, so fail loudly.
const indexPath = path.join(ecDir, 'index.html');
if (fs.existsSync(indexPath)) {
let html = fs.readFileSync(indexPath, 'utf8');
if (!html.includes('lotus-denoise.js')) {
// Classic (non-deferred) script runs before EC's deferred module entry.
html = html.replace(
/<script type="module"/,
'<script src="./lotus-denoise.js"></script><script type="module"',
);
if (!html.includes('lotus-denoise.js')) {
throw new Error(
'[lotus-denoise] Failed to inject shim into Element Call index.html ' +
'(no `<script type="module">` entry found) — build aborted.',
);
}
fs.writeFileSync(indexPath, html);
}
}
// [lotus] DENOISE CUTOVER: the getUserMedia shim is no longer injected.
// Our forked Element Call now runs ML denoise in-source as a real LiveKit
// audio TrackProcessor (activated by lotusDenoiseSource=1 in CallEmbed),
// which survives reconnects (fixes A7). We still copy the denoise/ assets
// above because the in-source processor loads its worklets/wasm from
// ./denoise/ at runtime. To roll back to the shim: restore the
// copy+inject of build/lotus-denoise.js here and swap lotusDenoiseSource
// back to lotusDenoise=ml in CallEmbed.getWidget.
},
};
}