Compare commits
14 Commits
349194e7e5
...
lotus
| Author | SHA1 | Date | |
|---|---|---|---|
| 36343baecc | |||
| 89cf171efc | |||
| 149ec8e4e4 | |||
| d1cd963e4b | |||
| 5ef0a1fd3e | |||
| 6ace96f2cf | |||
| 2d71f2ce30 | |||
| 2c3dba55e6 | |||
| c7a04dcc70 | |||
| 4b14c15518 | |||
| c68ef346bf | |||
| c5d7fcc303 | |||
| 9bf56d5748 | |||
| d5ce56930b |
@@ -1,2 +1,3 @@
|
|||||||
legacy-peer-deps=true
|
legacy-peer-deps=true
|
||||||
save-exact=true
|
save-exact=true
|
||||||
|
@lotusguild:registry=https://code.lotusguild.org/api/packages/LotusGuild/npm/
|
||||||
@@ -0,0 +1,655 @@
|
|||||||
|
# HANDOFF — Forking & Self-Building Element Call ("Lotus Call")
|
||||||
|
|
||||||
|
> **Audience:** a fresh Claude/engineer session with **no prior context** on this
|
||||||
|
> project. Read this top-to-bottom before touching anything. This document is the
|
||||||
|
> single source of truth for the Element Call (EC) fork initiative.
|
||||||
|
>
|
||||||
|
> **Status:** **PHASE 0–2 IMPLEMENTED (build-verified, not yet live-tested)**
|
||||||
|
> (2026-06-30). The fork exists, builds, is published, and cinny consumes it
|
||||||
|
> (Phase 0/1). **All 7 Phase-2 EC features are implemented on the fork's `lotus`
|
||||||
|
> branch**, each additive + flag-gated, build+typecheck-clean, per-feature
|
||||||
|
> reviewed (+ a holistic multi-agent review), and pushed. **None are live-tested
|
||||||
|
> yet** — every one needs the `LOTUS_TESTING.md` §D sweep, and the **cinny host
|
||||||
|
> side must be wired** (set flags / send actions / handle call_state) — see §12.
|
||||||
|
> See **§9** Phase 0/1 results, **§10** cutover, **§11** Phase-2 seams, **§12**
|
||||||
|
> Phase-2 status + cinny integration checklist. Created 2026-06 from `LotusGuild/cinny`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Phase 0 Results (verified 2026-06-29)
|
||||||
|
|
||||||
|
**Decisions taken with the user:** scope = Phase 0 recon; consumption model =
|
||||||
|
**private npm package** (§5 option 1). Recommended registry = **Gitea's built-in
|
||||||
|
npm registry** (`code.lotusguild.org`) — zero new infra.
|
||||||
|
|
||||||
|
### 9.1 Version → tag → commit mapping (LOCKED)
|
||||||
|
|
||||||
|
| Source | Value |
|
||||||
|
| :--------------------------------------------------- | :----------------------------------------- |
|
||||||
|
| cinny `package.json` pin | `@element-hq/element-call-embedded@0.20.1` |
|
||||||
|
| Bundle self-report (`VITE_APP_VERSION`/`appVersion`) | `embedded-v0.20.1` |
|
||||||
|
| npm registry `gitHead` for 0.20.1 | `2d74c48151d9edc01c65a22a91478aac81bf24d0` |
|
||||||
|
| GitHub tag `v0.20.1` → commit | `2d74c48…` ✅ **same commit** |
|
||||||
|
|
||||||
|
→ **Fork from upstream tag `v0.20.1` (commit `2d74c48`).** The embedded package
|
||||||
|
version equals the element-call release tag; repo `package.json` version is
|
||||||
|
`0.0.0` and the real version is stamped at publish time from the tag.
|
||||||
|
|
||||||
|
### 9.2 The shipped npm dist is a CLEAN upstream build
|
||||||
|
|
||||||
|
No `lotus`/`denoise`/`rnnoise` strings anywhere in
|
||||||
|
`node_modules/@element-hq/element-call-embedded/dist`. **All Lotus customization
|
||||||
|
(denoise shim) is injected at cinny build time, not baked into the package** — so
|
||||||
|
swapping the source does not disturb cinny's denoise injection layer. The
|
||||||
|
ringtone/reaction assets (`baduntss`, `cat`, `clap`, `call_declined`, …) are
|
||||||
|
upstream EC's own, not ours.
|
||||||
|
|
||||||
|
### 9.3 Build toolchain & mechanism
|
||||||
|
|
||||||
|
- **Node `24`** (`.node-version`), **pnpm `10.33.0`** (`packageManager` field,
|
||||||
|
via corepack).
|
||||||
|
- Build: **`pnpm run build:embedded`** = `vite build --config
|
||||||
|
vite-embedded.config.ts` with `NODE_OPTIONS=--max-old-space-size=16384`.
|
||||||
|
- Output dir is **repo-root `dist/`**; CI stages it into **`embedded/web/dist`**
|
||||||
|
(the `embedded/web/` dir holds the publish template: `package.json`, README,
|
||||||
|
both LICENSE files).
|
||||||
|
- Publish workflow upstream = `.github/workflows/publish-embedded-packages.yaml`:
|
||||||
|
builds → `npm version <tag> --no-git-tag-version` → `npm publish --provenance
|
||||||
|
--access public` to npmjs as `@element-hq/element-call-embedded`. (Also
|
||||||
|
Android/Maven + iOS/SwiftPM — irrelevant; we are web-only.)
|
||||||
|
|
||||||
|
### 9.4 Build reproduction — PARITY CONFIRMED
|
||||||
|
|
||||||
|
Cloned `element-call@v0.20.1` to `/root/code/element-call` (shallow), built with
|
||||||
|
isolated Node 24 / pnpm 10.33.0 (system Node 20 / cinny untouched). Result vs the
|
||||||
|
shipped npm dist:
|
||||||
|
|
||||||
|
- **137 of 147 files byte-identical** (same Vite content-hash): all CSS, fonts,
|
||||||
|
wasm, audio, JSON locale files, and `IndexedDBWorker`.
|
||||||
|
- **Only 5 JS chunks differ** (`index`, `pako.esm`, `polyfill-force`,
|
||||||
|
`rust-crypto`, `spa`) — **cause isolated to the version define**: our local
|
||||||
|
build baked `appVersion:\`dev\``(because`VITE_APP_VERSION`was unset) vs the
|
||||||
|
npm build's`appVersion:\`embedded-v0.20.1\``. `index.html` is identical modulo
|
||||||
|
the hashed asset filenames. **Benign** — our CI sets the version from the git
|
||||||
|
tag, so a tagged CI build will match.
|
||||||
|
|
||||||
|
### 9.5 Fork CI (drafted)
|
||||||
|
|
||||||
|
`.gitea/workflows/ci.yml` is staged in the clone (models cinny's
|
||||||
|
`.gitea/workflows/ci.yml` + upstream's publish flow). Linux-only (`ubuntu-latest`)
|
||||||
|
— the Windows worker is for cinny-desktop/Tauri, not the EC web bundle. Build job
|
||||||
|
on PR/push to `lotus`; publish job on `v*` tag → `@lotusguild/element-call-embedded`
|
||||||
|
to the Gitea npm registry (needs `secrets.GITEA_NPM_TOKEN`).
|
||||||
|
|
||||||
|
### 9.6 Phase 1 — DONE (2026-06-29)
|
||||||
|
|
||||||
|
1. ✅ **Fork repo live:** `code.lotusguild.org/LotusGuild/element-call` (public,
|
||||||
|
AGPL), default branch `lotus`, full history (7018 commits) + tag `v0.20.1`.
|
||||||
|
Branch `lotus` = `v0.20.1` + 2-file diff (CI workflow + embedded package
|
||||||
|
rename).
|
||||||
|
2. ✅ **Package published:** `@lotusguild/element-call-embedded@0.20.1` on the
|
||||||
|
Gitea npm registry (published manually from the version-faithful build while
|
||||||
|
the admin token was available). **Publicly readable** (unauth `npm install`
|
||||||
|
works → devs/CI need no token to consume; only publishing needs one).
|
||||||
|
3. ✅ **cinny wired & built clean** (Node 24): `.npmrc` scope line +
|
||||||
|
`package.json` dep + `vite.config.js` `viteStaticCopy` src. `npm install`
|
||||||
|
swapped the package (resolved from Gitea), `npm run build` succeeded,
|
||||||
|
`dist/public/element-call/` populated, bundle reports `appVersion:
|
||||||
|
embedded-v0.20.1`, **denoise shim injected + all denoise assets copied**
|
||||||
|
(injection layer unchanged). **These cinny edits are staged in the working
|
||||||
|
tree, NOT committed/pushed** — pushing triggers CI → desktop → deploy, so it's
|
||||||
|
gated on the §D live test (see §10).
|
||||||
|
|
||||||
|
### 9.8 Reproducibility note (important)
|
||||||
|
|
||||||
|
A from-source rebuild is **NOT byte-identical** to upstream's npm tarball.
|
||||||
|
137/147 files match exactly (CSS, fonts, wasm, audio, worker); the 5 JS chunks
|
||||||
|
(`index`, `pako.esm`, `polyfill-force`, `rust-crypto`, `spa`) differ because the
|
||||||
|
rolldown/oxc **minifier mangles export names differently** across build
|
||||||
|
environments (and the version-define is one input). This is normal and benign —
|
||||||
|
the code is functionally equivalent. **Do not chase byte-parity; the §D live call
|
||||||
|
test is the real parity gate.**
|
||||||
|
|
||||||
|
### 9.9 Remaining follow-ups (not blocking the cutover)
|
||||||
|
|
||||||
|
- **CI publishing:** `.gitea/workflows/ci.yml` publishes on a `v*` tag but needs
|
||||||
|
(a) a Gitea Actions runner for `LotusGuild/element-call`, and (b) a **durable**
|
||||||
|
`GITEA_NPM_TOKEN` repo secret with package read/write (the admin token used for
|
||||||
|
the manual publish is being deleted, so it was deliberately NOT baked in). Until
|
||||||
|
then, publishing is manual (`npm version <tag>` in `embedded/web` →
|
||||||
|
`npm publish`).
|
||||||
|
- Decide rebase cadence vs upstream (0.20.2 / 0.20.3 already out — see §9.1).
|
||||||
|
|
||||||
|
### 9.7 Ready-to-apply artifacts (staged 2026-06-29)
|
||||||
|
|
||||||
|
**Fork side — already committed** on branch `lotus` in `/root/code/element-call`
|
||||||
|
(remote `lotus` = `code.lotusguild.org/LotusGuild/element-call.git`, push deferred
|
||||||
|
until the repo exists). Minimal 2-file diff vs tag `v0.20.1`:
|
||||||
|
`.gitea/workflows/ci.yml` (new) + `embedded/web/package.json` (rename to
|
||||||
|
`@lotusguild/element-call-embedded`). Push with:
|
||||||
|
`git push -u lotus lotus && git push lotus v0.20.1` (and tag `v0.20.1` on our side
|
||||||
|
to trigger the first publish, or push our own `v0.20.1` tag).
|
||||||
|
|
||||||
|
**cinny side — NOT yet applied** (applying before the package is published breaks
|
||||||
|
`npm ci`). Exactly 3 edits + a lockfile regen:
|
||||||
|
|
||||||
|
1. `.npmrc` — append the scoped-registry line:
|
||||||
|
```
|
||||||
|
@lotusguild:registry=https://code.lotusguild.org/api/packages/LotusGuild/npm/
|
||||||
|
```
|
||||||
|
(CI/auth: `//code.lotusguild.org/api/packages/LotusGuild/npm/:_authToken=${GITEA_NPM_TOKEN}`
|
||||||
|
— inject via env in CI, do not commit a plaintext token.)
|
||||||
|
2. `package.json:104` —
|
||||||
|
`"@element-hq/element-call-embedded": "0.20.1"` →
|
||||||
|
`"@lotusguild/element-call-embedded": "0.20.1"`.
|
||||||
|
3. `vite.config.js:25` — `viteStaticCopy` src:
|
||||||
|
`node_modules/@element-hq/element-call-embedded/dist` →
|
||||||
|
`node_modules/@lotusguild/element-call-embedded/dist`.
|
||||||
|
**`stripBase: 4` stays unchanged** — `node_modules/@lotusguild/element-call-embedded/dist`
|
||||||
|
is still exactly 4 leading segments. (Update the comment's path reference too.)
|
||||||
|
4. `package-lock.json` — regenerated by `npm install`, not hand-edited (drops the
|
||||||
|
`registry.npmjs.org/@element-hq/...` resolved URL for the Gitea one).
|
||||||
|
|
||||||
|
The denoise injection (`lotusDenoise()` in `vite.config.js`) is **unchanged** — it
|
||||||
|
keys off `dist/public/element-call/index.html`, which our fork's bundle still
|
||||||
|
produces identically (verified: `index.html` byte-identical modulo asset hashes).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. TL;DR / The Goal
|
||||||
|
|
||||||
|
We embed **Element Call** (the Matrix group-VoIP/video app) inside Lotus Chat to
|
||||||
|
power voice/video channels. Today we consume Element's **pre-compiled npm
|
||||||
|
bundle** and can only steer it from the outside (a limited widget API + fragile
|
||||||
|
same-origin DOM hacks). Several in-call problems are **unfixable from outside**
|
||||||
|
because they live in EC's compiled JS.
|
||||||
|
|
||||||
|
**We want true ownership: fork `element-hq/element-call`, build it from source
|
||||||
|
ourselves, host our build, and replace the npm bundle with our fork.** Then
|
||||||
|
every in-call behavior becomes editable code.
|
||||||
|
|
||||||
|
**This requires standing up a brand-new repo and build pipeline for our EC fork.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Why fork? (What we cannot fix today)
|
||||||
|
|
||||||
|
These came out of live testing and are documented in `LOTUS_BUGS.md` →
|
||||||
|
"Known Element Call iframe limitations":
|
||||||
|
|
||||||
|
| Issue | What's wrong | Why outside-fixes fail |
|
||||||
|
| :----------------------------------------------------- | :------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| **A6** — avatar decorations in-call | Our profile-decoration overlays don't appear on in-call video tiles | The video grid is rendered by EC's React app inside the iframe. We can only inject overlay DOM (fragile) — we can't make it a first-class part of the tile. |
|
||||||
|
| **A5** — focus camera / fullscreen during screenshare | Can't reliably spotlight a participant's camera while someone screenshares | EC's **layout logic** (screenshare priority, spotlight) is compiled JS we don't control. We currently DOM-click tiles as a hack. |
|
||||||
|
| **A7** — mic dead after EC's "Reconnect" | After EC's own mid-call reconnect, the local mic isn't re-published | EC's reconnect/track-republish path is internal. (Partly entangled with our denoise shim — see §6.) |
|
||||||
|
| Native theming | EC's UI doesn't match Lotus design; we inject CSS hacks | Real theming needs source-level component/token changes. |
|
||||||
|
| Decorations, custom controls, custom layouts, branding | all blocked | all require source access |
|
||||||
|
|
||||||
|
**Bottom line:** the iframe is **same-origin** (we self-host it), so we can read
|
||||||
|
and even write its DOM — but we **do not own its source**, so we can't change its
|
||||||
|
**behavior/logic**, only poke at its rendered output. Forking removes that wall.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. How EC is integrated TODAY (the current architecture)
|
||||||
|
|
||||||
|
Understand this fully before changing it — the fork must slot into the same
|
||||||
|
integration seams.
|
||||||
|
|
||||||
|
### 2.1 Where the EC bundle comes from
|
||||||
|
|
||||||
|
- npm package: **`@element-hq/element-call-embedded`**, pinned to **`0.20.1`** in
|
||||||
|
`cinny/package.json` (line ~104).
|
||||||
|
- It ships a **pre-built `dist/`**. At cinny build time,
|
||||||
|
`vite-plugin-static-copy` copies that `dist/` flat into
|
||||||
|
**`public/element-call/`** (see `cinny/vite.config.js`, the `copyFiles`
|
||||||
|
target with `rename: { stripBase: 4 }` — note the stripBase gotcha documented
|
||||||
|
there; getting this wrong 404s the widget).
|
||||||
|
- It is **NOT committed** to git (`git ls-files public/element-call` → 0). It's a
|
||||||
|
build artifact materialized from `node_modules`.
|
||||||
|
|
||||||
|
### 2.2 How EC is loaded & controlled
|
||||||
|
|
||||||
|
- The widget iframe `src` is **same-origin**:
|
||||||
|
`${BASE_URL}/public/element-call/index.html?<params>` (see
|
||||||
|
`cinny/src/app/plugins/call/CallEmbed.ts`, `getWidget()` /
|
||||||
|
`getIframe()`). Sandbox: `allow-forms allow-scripts allow-same-origin
|
||||||
|
allow-popups allow-modals allow-downloads`; `allow="microphone; camera;
|
||||||
|
display-capture; autoplay; clipboard-write;"`.
|
||||||
|
- **Control surface #1 — the official widget API** (`matrix-widget-api`):
|
||||||
|
`ClientWidgetApi` + a custom `CallWidgetDriver`. This is the robust,
|
||||||
|
version-stable channel (theme change, hangup, capabilities, timeline events).
|
||||||
|
Files: `plugins/call/CallEmbed.ts`, `plugins/call/CallWidgetDriver.ts`,
|
||||||
|
`plugins/call/utils.ts` (capabilities), `plugins/call/CallControl.ts`.
|
||||||
|
- **Control surface #2 — same-origin DOM poking** (fragile, version-coupled):
|
||||||
|
reading `iframe.contentDocument` to detect speakers/mute state and
|
||||||
|
`.click()`-ing tiles to focus a camera. Files:
|
||||||
|
`hooks/useCallSpeakers.ts` (reads `[data-muted]`, `[data-video-fit]`),
|
||||||
|
`plugins/call/CallControl.ts` (`focusCameraParticipant` — tile selectors).
|
||||||
|
**These selectors break on every EC version bump.** A fork lets us replace
|
||||||
|
these hacks with real APIs/props.
|
||||||
|
- **Control surface #3 — URL params + build-time injection** for our denoise
|
||||||
|
shim (see §6).
|
||||||
|
|
||||||
|
### 2.3 Full file inventory (everything that touches EC in cinny)
|
||||||
|
|
||||||
|
Plugin / core:
|
||||||
|
|
||||||
|
- `src/app/plugins/call/CallEmbed.ts` — iframe creation, widget API wiring, theme sync, hangup, load watchdog/self-heal, denoise URL params.
|
||||||
|
- `src/app/plugins/call/CallControl.ts` — control state + **DOM-poking** (`focusCameraParticipant`, spotlight).
|
||||||
|
- `src/app/plugins/call/CallControl.tsx` _(call-status variant)_ and `features/call-status/CallControl.tsx`.
|
||||||
|
- `src/app/plugins/call/CallWidgetDriver.ts` — widget driver (capabilities, event relay).
|
||||||
|
- `src/app/plugins/call/utils.ts` — widget capabilities set.
|
||||||
|
- `src/app/plugins/call/hooks.ts`, `index.ts` — plugin exports/hooks.
|
||||||
|
- `src/app/state/callEmbed.ts` — jotai atoms for the active embed.
|
||||||
|
|
||||||
|
React / UI:
|
||||||
|
|
||||||
|
- `src/app/components/CallEmbedProvider.tsx` — the big one: incoming-call ring/banner, RTCNotification + **RTCDecline** listeners, PiP, mute badges, fullscreen, ringtones.
|
||||||
|
- `src/app/features/call/CallView.tsx` — prescreen lobby vs joined (the iframe placement target), load-error recovery UI.
|
||||||
|
- `src/app/features/call/CallControls.tsx` — in-call control bar (mic/cam/deafen/screenshare/fullscreen/more/PiP).
|
||||||
|
- `src/app/features/call/CallMemberCard.tsx` — **lobby** participant roster (this is where `AvatarDecoration` works today; in-call grid is EC's).
|
||||||
|
- `src/app/features/call/PrescreenControls.tsx` — join controls.
|
||||||
|
- `src/app/features/call-status/*` — `CallStatus.tsx`, `MemberGlance.tsx` (the "Focus camera" menu lives here), `LiveChip.tsx`.
|
||||||
|
- `src/app/features/room-nav/RoomNavItem.tsx`, `features/room/Room.tsx`, `features/room/RoomViewHeader.tsx`, `pages/client/space/Space.tsx`, `pages/CallStatusRenderer.tsx`, `pages/Router.tsx` — call entry points / status surfacing.
|
||||||
|
|
||||||
|
Hooks:
|
||||||
|
|
||||||
|
- `src/app/hooks/useCallEmbed.ts`, `useCall.ts`, `useCallSpeakers.ts` (DOM-poking), `useCallJoinLeaveSounds.ts`, `useAfkAutoMute.ts`.
|
||||||
|
|
||||||
|
Build:
|
||||||
|
|
||||||
|
- `cinny/vite.config.js` — `copyFiles` (EC dist copy) + `lotusDenoise()` plugin (denoise asset copy + index.html shim injection, in `closeBundle`).
|
||||||
|
|
||||||
|
Utils:
|
||||||
|
|
||||||
|
- `src/app/utils/ringtones.ts`, `utils/denoisePipeline.ts`, `utils/lotusDenoiseUtils.ts`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Hosting / infra context (the OTHER repo)
|
||||||
|
|
||||||
|
There are **two repos**:
|
||||||
|
|
||||||
|
1. **`LotusGuild/cinny`** (`/root/code/cinny`) — this Lotus Chat fork. Consumes EC.
|
||||||
|
2. **`LotusGuild/matrix`** (`/root/code/matrix`) — the **infra/homeserver** repo.
|
||||||
|
Subdirs: `livekit/` (the SFU EC talks to), `deploy/`, `draupnir/`,
|
||||||
|
`hookshot/`, `landing/`, `matrixbot/`, `systemd/`. Gitea remote
|
||||||
|
`code.lotusguild.org/LotusGuild/matrix`, branch `main`.
|
||||||
|
|
||||||
|
EC needs a **LiveKit SFU** + the **livekit-jwt-service**; those live in
|
||||||
|
`matrix/livekit/`. A self-hosted EC build must be configured to point at our
|
||||||
|
homeserver (`matrix.lotusguild.org` / synapse) and our LiveKit. EC's runtime
|
||||||
|
`config.json` (homeserver, livekit URL, feature flags) is part of what we'll own
|
||||||
|
once we build it ourselves.
|
||||||
|
|
||||||
|
Deployment today: `chat.lotusguild.org` (the cinny web build, which embeds EC at
|
||||||
|
`/public/element-call/`). cinny-desktop (`LotusGuild/cinny-desktop`, a Tauri
|
||||||
|
wrapper, bumped by cinny CI) embeds the same.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. The plan (proposed — confirm with the user before executing)
|
||||||
|
|
||||||
|
### Decision: **YES, create a new repo.** `LotusGuild/element-call`
|
||||||
|
|
||||||
|
Rationale: EC is a large standalone app (React + LiveKit client SDK + matrixRTC +
|
||||||
|
its own Vite build + heavy deps). Keep it out of cinny so cinny's build stays
|
||||||
|
clean — cinny keeps consuming a **built EC `dist/`**, exactly as today, just
|
||||||
|
sourced from **our fork** instead of npm.
|
||||||
|
|
||||||
|
### Phase 0 — Recon (no code)
|
||||||
|
|
||||||
|
- Fork `github.com/element-hq/element-call` → `LotusGuild/element-call` on Gitea.
|
||||||
|
- Pin to the upstream tag matching **0.20.1** (`element-call-embedded` 0.20.1's
|
||||||
|
corresponding `element-call` release) so behavior matches what's shipping now.
|
||||||
|
Verify the embedded-package version ↔ element-call repo tag mapping.
|
||||||
|
- Read EC's own build docs: it builds the "embedded" widget bundle (the thing
|
||||||
|
currently published as `@element-hq/element-call-embedded`). Reproduce that
|
||||||
|
build locally and confirm the output matches `public/element-call/` today.
|
||||||
|
- **License:** element-call is **AGPL-3.0**, same as Lotus Chat — compatible.
|
||||||
|
Our fork must remain AGPL and publish source.
|
||||||
|
|
||||||
|
### Phase 1 — Reproduce current behavior from our fork (parity, no features)
|
||||||
|
|
||||||
|
- Build our fork's embedded bundle; wire cinny to consume it instead of the npm
|
||||||
|
package (see §5 for the consumption options). Smoke-test: a call works exactly
|
||||||
|
as today (web + desktop), denoise shim still injects, widget API + theme still
|
||||||
|
work. **No behavior change yet** — this de-risks the swap.
|
||||||
|
|
||||||
|
### Phase 2 — Replace the outside hacks with source-level features
|
||||||
|
|
||||||
|
Tackle the §1 issues in EC's source:
|
||||||
|
|
||||||
|
- **A6:** render avatar decorations as part of the video-tile component
|
||||||
|
(read decoration data we pass in via widget data / URL param / a small bridge).
|
||||||
|
- **A5:** fix focus/spotlight + screenshare-coexistence in EC's layout code;
|
||||||
|
expose a clean widget action so cinny can trigger it (kill the DOM `.click()`).
|
||||||
|
- **A7:** fix mic re-publish on reconnect; reconcile with our denoise shim (§6) —
|
||||||
|
ideally move denoise INTO the fork as a real audio-processing step instead of a
|
||||||
|
`getUserMedia` monkeypatch.
|
||||||
|
- Native Lotus theming/branding at the source (kill the injected-CSS hacks).
|
||||||
|
- Then retire the DOM-poking in `useCallSpeakers.ts` / `CallControl.ts` in favor
|
||||||
|
of real widget messages.
|
||||||
|
|
||||||
|
### Phase 3 — Maintenance posture
|
||||||
|
|
||||||
|
- Decide rebase cadence vs. upstream element-call releases. Keep customizations
|
||||||
|
isolated (feature flags / minimal-diff patches) to ease rebasing.
|
||||||
|
- CI in the new repo builds + publishes the embedded dist as a versioned
|
||||||
|
artifact; cinny CI consumes a pinned version.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. How cinny should consume the fork (pick one — decide with user)
|
||||||
|
|
||||||
|
1. **Private npm package** (mirror the current model): our fork's CI publishes
|
||||||
|
`@lotusguild/element-call-embedded` to a registry; cinny depends on it and
|
||||||
|
`viteStaticCopy` keeps working almost unchanged. _Cleanest swap; needs a
|
||||||
|
registry._
|
||||||
|
2. **Git submodule + build in cinny CI:** add the fork as a submodule, build it
|
||||||
|
during cinny's build, copy its `dist/` to `public/element-call/`. _No
|
||||||
|
registry; heavier cinny CI._
|
||||||
|
3. **CI artifact copy:** fork CI uploads a `dist` tarball; cinny CI downloads a
|
||||||
|
pinned version at build. _Decoupled; needs artifact plumbing._
|
||||||
|
|
||||||
|
**Recommendation: Option 1** — it changes the least in cinny (just swap the
|
||||||
|
package name in `package.json` + the `viteStaticCopy` src path) and preserves the
|
||||||
|
clean cinny/EC separation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. The denoise shim — critical interaction (don't break this)
|
||||||
|
|
||||||
|
Lotus ships ML noise suppression by **injecting a same-origin pre-init shim into
|
||||||
|
EC's `index.html` at build time** (cinny `vite.config.js` → `lotusDenoise()`,
|
||||||
|
`closeBundle`). The shim monkeypatches `getUserMedia` **before EC captures the
|
||||||
|
mic** and routes audio through RNNoise/Speex/DTLN AudioWorklets, then EC/LiveKit
|
||||||
|
publishes the processed track. It's activated via URL params
|
||||||
|
(`lotusDenoise=ml&lotusModel=…&lotusGate=…`) set in `CallEmbed.ts`.
|
||||||
|
|
||||||
|
- Assets copied to `public/element-call/denoise/` at build (sapphi RNNoise/Speex/
|
||||||
|
gate worklets + `@workadventure/noise-suppression` DTLN tree).
|
||||||
|
- Related: `utils/denoisePipeline.ts`, `utils/lotusDenoiseUtils.ts`,
|
||||||
|
`settings/general/DenoiseTester.tsx`, `VoiceMessageRecorder.tsx`.
|
||||||
|
- **Known issues:** denoise quality is still poor (tracked separately); and the
|
||||||
|
mic-after-reconnect bug (A7) is suspected to involve the shim's getUserMedia
|
||||||
|
patch handing back a stale processed stream when EC re-acquires the mic.
|
||||||
|
|
||||||
|
**Once we own the fork, the right move is to make denoise a first-class
|
||||||
|
audio-processing stage inside EC** (not an index.html monkeypatch) — more robust,
|
||||||
|
survives reconnects, and removes the build-time injection hack. Until then, the
|
||||||
|
fork's `index.html` must remain injectable the same way, or the shim must be
|
||||||
|
re-homed into the fork.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Doc-accuracy notes / corrections for the new session
|
||||||
|
|
||||||
|
- `LOTUS_TODO.md` (~line 533) calls EC a **"cross-origin iframe"** — **outdated.**
|
||||||
|
EC is **same-origin** today (self-hosted under our domain;
|
||||||
|
`iframe.sandbox` includes `allow-same-origin`; we read `contentDocument`), and
|
||||||
|
**as of 2026-06-29 we own the fork's source** (`@lotusguild/element-call-embedded`).
|
||||||
|
The _practical_ point it made still holds _until we ship the audio-inject API_:
|
||||||
|
**LiveKit's `LocalAudioTrack` lives in EC's module scope**, not on `window`, so
|
||||||
|
cinny can't reach it even same-origin — which is why the in-call soundboard had
|
||||||
|
to be local-playback-only. **The fork removes this wall:** EC can expose a real
|
||||||
|
`io.lotus.inject_audio` widget action (Phase 2) that mixes into the published
|
||||||
|
track from inside its own module scope.
|
||||||
|
- `LOTUS_FEATURES.md` documents the EC upgrade history (0.16.3 → 0.19.4 →
|
||||||
|
0.20.1), the dark-mode CSS injection, and AFK auto-mute — all relevant prior
|
||||||
|
art for what the fork must preserve.
|
||||||
|
- `LOTUS_TESTING.md` §D is the **EC regression sweep** to re-run after the fork
|
||||||
|
swap (Phase 1 parity check).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. First actions for the new session
|
||||||
|
|
||||||
|
1. Read this file, then skim §2.3's files in `cinny` to internalize the seams.
|
||||||
|
2. Confirm with the user: new repo name, consumption model (§5), rebase cadence.
|
||||||
|
3. Phase 0: fork element-call, map 0.20.1 ↔ element-call tag, reproduce the
|
||||||
|
embedded build locally, diff against `public/element-call/`.
|
||||||
|
4. Phase 1: wire cinny to the fork, run `LOTUS_TESTING.md` §D parity sweep.
|
||||||
|
5. Only then start Phase 2 features (A5/A6/A7, theming, denoise-in-source).
|
||||||
|
|
||||||
|
**Cross-references:** `LOTUS_BUGS.md` (EC limitations + verify queue),
|
||||||
|
`LOTUS_TODO.md` (denoise/soundboard constraints), `LOTUS_FEATURES.md` (EC history),
|
||||||
|
`LOTUS_TESTING.md` §D (regression sweep). Infra: `/root/code/matrix` (`livekit/`,
|
||||||
|
`deploy/`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. Live cutover — the remaining steps (Phase 1 finish)
|
||||||
|
|
||||||
|
The fork is published and cinny builds against it locally (§9.6). What's left to
|
||||||
|
go live:
|
||||||
|
|
||||||
|
1. **Run `LOTUS_TESTING.md` §D** against a local cinny build (`npm run build` is
|
||||||
|
already proven; serve `dist/` or `npm run dev`). Verify a real call: join,
|
||||||
|
mic/cam, screenshare, theme sync, denoise on, widget hangup — web first.
|
||||||
|
2. **Commit the cinny edits** (currently staged, uncommitted in the working tree):
|
||||||
|
`.npmrc`, `package.json`, `package-lock.json`, `vite.config.js`. Suggested
|
||||||
|
message: `chore(call): consume self-built @lotusguild/element-call-embedded`.
|
||||||
|
3. **Push to `lotus`** → cinny CI builds, then `trigger-desktop` bumps
|
||||||
|
cinny-desktop → Tauri release. Re-run §D on **cinny-desktop** (the path where
|
||||||
|
the old `stripBase` bug bit — verify the widget loads, not a 404).
|
||||||
|
4. Only then start **Phase 2** (A5/A6/A7, theming, denoise-in-source).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. Phase 2 — implementation seams (mapped 2026-06-29)
|
||||||
|
|
||||||
|
The exact integration points for each Phase 2 item, found by reading the EC fork
|
||||||
|
|
||||||
|
- cinny source. **All of these are media-path / in-call features that cannot be
|
||||||
|
functionally verified without a live Matrix + LiveKit call** — implement each as
|
||||||
|
a minimal, **feature-flagged, additive** diff (no behavior change unless cinny
|
||||||
|
opts in), build-verify the fork (`pnpm build:embedded`, ~15s) AND cinny
|
||||||
|
(`npm run build`), then gate shipping on `LOTUS_TESTING.md` §D.
|
||||||
|
|
||||||
|
**Shared widget channel (the backbone for #2/#3/#4/#7):**
|
||||||
|
|
||||||
|
- EC→cinny: `widget.api.transport.send("io.lotus.<x>", data)` (see
|
||||||
|
`element-call/src/widget.ts`).
|
||||||
|
- cinny→EC actions: add the action name to the `lazyActions` allow-list in
|
||||||
|
`widget.ts` (the array at ~L101) and handle it in EC; cinny sends via
|
||||||
|
`this.call.transport.send(...)`.
|
||||||
|
- cinny receives EC→cinny actions via the existing `listenAction(type, cb)`
|
||||||
|
helper in `plugins/call/CallEmbed.ts:626` (auto-replies `{}` so the transport
|
||||||
|
doesn't time out — same pattern as `io.element.device_mute`).
|
||||||
|
|
||||||
|
**#2 mute/speaker events** — Source: subscribe to `vm.userMedia$`
|
||||||
|
(`CallViewModel`), per member `speaking$` + `audioEnabled$`
|
||||||
|
(`state/media/UserMediaViewModel.ts:47-48`); aggregate and
|
||||||
|
`transport.send("io.lotus.call_state", {participants:[{id,speaking,audioEnabled}]})`.
|
||||||
|
Mount in `room/InCallView.tsx` via `useEffect` guarded by `widget !== null`.
|
||||||
|
cinny: `listenAction("io.lotus.call_state")` in `CallEmbed.ts`, feed
|
||||||
|
`hooks/useCallSpeakers.ts` → delete its `contentDocument` `[data-muted]` /
|
||||||
|
`[data-video-fit]` scrape. _Additive, low risk._
|
||||||
|
|
||||||
|
**#4 spotlight/focus** — EC: add `io.lotus.focus_participant` to the `lazyActions`
|
||||||
|
list (`widget.ts`), drive `vm`'s spotlight (`spotlightSpeaker$` /
|
||||||
|
`spotlight$` in `CallViewModel.ts:898/1001`) to pin a given identity, coexisting
|
||||||
|
with `hasRemoteScreenShares$` (L1008). cinny: replace
|
||||||
|
`CallControl.ts` `focusCameraParticipant` `.click()` walk with
|
||||||
|
`transport.send("io.lotus.focus_participant", {userId})`. _Additive, low risk._
|
||||||
|
|
||||||
|
**#3 audio-inject** — EC: add `io.lotus.inject_audio` action; mix an
|
||||||
|
`AudioBufferSourceNode` into the published mic track. The local publish path is
|
||||||
|
`state/CallViewModel/localMember/Publisher.ts` + `LocalMember.ts` (LiveKit
|
||||||
|
`localParticipant`); create a `MediaStreamAudioDestinationNode`, mix mic + clip,
|
||||||
|
`replaceTrack`. cinny soundboard calls the action instead of local-only playback.
|
||||||
|
_Medium; touches publish path → live-test carefully._
|
||||||
|
|
||||||
|
**#1 denoise-in-source** — replace the cinny `lotusDenoise()` `getUserMedia`
|
||||||
|
monkeypatch with a real processing stage in EC's mic capture
|
||||||
|
(`Publisher.ts`/`LocalMember.ts`; note EC has a `TrackProcessorContext` +
|
||||||
|
`BlurBackgroundTransformer` precedent in `livekit/`). EC re-runs it on every
|
||||||
|
(re)publish → fixes A7. Remove `vite.config.js` `lotusDenoise()` + URL params in
|
||||||
|
`CallEmbed.ts`; move `denoise/` assets into the fork. _Highest value, highest
|
||||||
|
risk — most live testing._
|
||||||
|
|
||||||
|
**#5 theming** — add a Lotus/TDS theme in EC's theme system (`src/useTheme.ts` +
|
||||||
|
EC theme tokens / CSS); driven by the existing `setTheme()` channel cinny already
|
||||||
|
calls (`CallEmbed.ts:277`). Bake transparent background. Delete cinny's
|
||||||
|
`applyStyles()` injection + `background:none !important`. _Medium._
|
||||||
|
|
||||||
|
**#6 in-call decorations** — render the decoration APNG in EC's tile component
|
||||||
|
(`tile/GridTile.tsx`); pass slugs via widget member data. cinny already has the
|
||||||
|
decoration data + `AvatarDecoration` (lobby `CallMemberCard.tsx`). _Medium-Large._
|
||||||
|
|
||||||
|
**#7 quality controls** — set audio `maxBitrate` via
|
||||||
|
`RTCRtpSender.setParameters` and screenshare `getDisplayMedia` constraints in
|
||||||
|
EC's publish path (`Publisher.ts`); configurable via `config.json` / a widget
|
||||||
|
message. Keep the server `voice-limit-guard` as enforcement. _Medium._
|
||||||
|
|
||||||
|
**Rollback:** revert the 4 cinny files (restores `@element-hq/...@0.20.1` from
|
||||||
|
npmjs). The fork repo/package can stay; nothing else depends on it until pushed.
|
||||||
|
|
||||||
|
### Local repro/build environment (this session, 2026-06-29)
|
||||||
|
|
||||||
|
- Upstream cloned + our `lotus` branch at `/root/code/element-call` (remote
|
||||||
|
`lotus` → Gitea; origin → github upstream, now un-shallowed/full history).
|
||||||
|
- Isolated **Node 24.18.0** lives in the session scratchpad (system Node is 20);
|
||||||
|
cinny's `.node-version` is `24.13.1`, so use Node 24 to build cinny too.
|
||||||
|
- Build the embedded bundle: in `/root/code/element-call`, with Node 24 + pnpm
|
||||||
|
10.33.0 on PATH, `VITE_APP_VERSION=embedded-v0.20.1 pnpm run build:embedded`
|
||||||
|
→ output in `dist/`; stage to `embedded/web/dist` before publishing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. Phase 2 — IMPLEMENTED on the fork (2026-06-30)
|
||||||
|
|
||||||
|
All 7 EC features are on the `lotus` branch of `LotusGuild/element-call`, each
|
||||||
|
**additive + feature-flagged** (a vanilla call with no `lotus*` params / no Lotus
|
||||||
|
actions behaves exactly like upstream), build + `tsc` clean, per-feature reviewed
|
||||||
|
(fixes applied) and holistically reviewed. **Not yet live-tested** — all need the
|
||||||
|
`LOTUS_TESTING.md` §D sweep.
|
||||||
|
|
||||||
|
Fork modules live under `element-call/src/lotus/*`; mounts are `useEffect`s in
|
||||||
|
`src/room/InCallView.tsx`. Custom widget actions are in `src/lotus/lotusActions.ts`
|
||||||
|
(toWidget ones allow-listed in `src/widget.ts`).
|
||||||
|
|
||||||
|
| # | Feature | Enable via | EC module |
|
||||||
|
| :-- | :------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------ | ---------------------------------------------------- |
|
||||||
|
| 2 | Speaker/mute/camera state → host | URL `lotusCallState=1` | `lotusCallState.ts` (sends `io.lotus.call_state`) |
|
||||||
|
| 4 | Focus/spotlight a participant (works during screenshare) | action `io.lotus.focus_participant {userId | null}` | `lotusFocus.ts` + `CallViewModel` spotlight override |
|
||||||
|
| 3 | Soundboard audio-inject (heard by peers) | URL `lotusAudioInject=1` + action `io.lotus.inject_audio {url,volume?}` | `lotusAudioInject.ts` |
|
||||||
|
| 7 | Audio/screenshare quality caps | action `io.lotus.set_quality {audioMaxBitrate?,screenshareMaxBitrate?,screenshareMaxFramerate?}` | `lotusQuality.ts` |
|
||||||
|
| 5 | Transparent bg + Lotus theme | URL `lotusTransparent=1` / `lotusTheme=1` | `useTheme.ts` + `index.css` |
|
||||||
|
| 6 | In-call avatar decorations | action `io.lotus.decorations {decorations:{userId:url}}` | `lotusDecorations.ts` + `MediaView.tsx` |
|
||||||
|
| 1 | ML denoise in-source (fixes A7) | URL **`lotusDenoiseSource=1`** (+`lotusModel`,`lotusGate`,`lotusGateThreshold`,`lotusDenoiseBase`) — deliberately NOT the existing `lotusDenoise=ml` (that drives the host shim; reusing it would double-process) | `lotusDenoise.ts` + `lotusDenoiseProcessor.ts` |
|
||||||
|
|
||||||
|
**Security hardening applied** (holistic audit): `lotusDenoiseBase` forced
|
||||||
|
same-origin before `audioWorklet.addModule` (was an arbitrary-code-load vector
|
||||||
|
via a crafted link); audio-inject gated behind `lotusAudioInject=1`; decoration
|
||||||
|
roster capped. Only `https`/`blob` URLs accepted for inject/decoration assets.
|
||||||
|
|
||||||
|
### 12.1 cinny host integration checklist (REQUIRED to light these up)
|
||||||
|
|
||||||
|
The EC side is additive and dormant until cinny opts in. Host work needed (in
|
||||||
|
`src/app/plugins/call/CallEmbed.ts` unless noted):
|
||||||
|
|
||||||
|
> ⚠️ **CRITICAL TIMING (protocol audit F1):** only send `io.lotus.*` **toWidget**
|
||||||
|
> actions (#3 focus, #6 decorations, #7 quality, audio-inject) **after** the call
|
||||||
|
> is joined (`CallEmbed.onCallJoined` / `this.joined`). Those actions are
|
||||||
|
> allow-listed at EC app-init (so `preventDefault` suppresses the auto-error)
|
||||||
|
> but their handlers only mount with `InCallView` (post-join). Sending earlier
|
||||||
|
> leaves the host's `transport.send` pending until the **10s timeout**. Queue and
|
||||||
|
> flush on join, or no-op before join.
|
||||||
|
>
|
||||||
|
> Also: **F3** — the fork implements only `rnnoise`/`speex`; cinny's `dtln`/
|
||||||
|
> `deepfilternet` selections silently fall back to rnnoise (now logged). Restrict
|
||||||
|
> the embedded-call model picker to rnnoise/speex, or implement the others in
|
||||||
|
> `lotusDenoiseProcessor.ts`. **F4** — cinny sends `lotusNativeNS`, which the
|
||||||
|
> fork ignores; drop it or wire it in. **F7** — no widget _capability_ changes
|
||||||
|
> needed; custom actions bypass capability checks.
|
||||||
|
|
||||||
|
1. **Set the URL flags** on the widget iframe params (the `URLSearchParams` in
|
||||||
|
`CallEmbed`): `lotusCallState=1`, `lotusTransparent=1`/`lotusTheme=1`,
|
||||||
|
`lotusAudioInject=1` as desired. (Denoise already sets `lotusDenoise=ml` etc.)
|
||||||
|
2. **Ack `io.lotus.call_state`**: add `listenAction('io.lotus.call_state', …)` —
|
||||||
|
without a reply the fork's sends time out every 250ms. Feed the payload into
|
||||||
|
`useCallSpeakers` and RETIRE its `contentDocument` DOM scrape.
|
||||||
|
3. **Send actions** via `this.call.transport.send(...)`:
|
||||||
|
`io.lotus.focus_participant` (replace `CallControl.focusCameraParticipant`’s
|
||||||
|
`.click()`), `io.lotus.inject_audio` (from the soundboard), `io.lotus.set_quality`
|
||||||
|
(from quality settings), `io.lotus.decorations` (push the MSC4133 decoration
|
||||||
|
map; resolve mxc→https first).
|
||||||
|
4. **#1 denoise cutover**: once verified, STOP injecting the `lotusDenoise()`
|
||||||
|
shim in `cinny/vite.config.js` and remove the `index.html` injection — the
|
||||||
|
fork now does denoise in-source. Keep shipping the `denoise/` assets (the
|
||||||
|
fork loads `./denoise/…` at runtime) until those move into the fork build.
|
||||||
|
5. Re-run `LOTUS_TESTING.md` §D for each feature; only then ship.
|
||||||
|
|
||||||
|
### 12.2 Holistic multi-agent review — outstanding follow-ups (non-blocking)
|
||||||
|
|
||||||
|
Four aspect-agents reviewed the whole fork. Criticals were fixed in-branch (the
|
||||||
|
denoise restart-silence/A7 bug; the `lotusDenoiseBase` code-load vector;
|
||||||
|
audio-inject opt-in gate; #6 rendering in the wrong component; #7 simulcast cap).
|
||||||
|
Remaining, deliberately deferred:
|
||||||
|
|
||||||
|
- **Denoise H2 (double-processing):** if cinny is set to `lotusDenoise=ml` while
|
||||||
|
ALSO still injecting its build-time `getUserMedia` shim, audio is denoised
|
||||||
|
twice. The #1 cutover MUST remove the cinny-side injection (it currently has
|
||||||
|
none injected into the iframe — keep it that way). Hard requirement, not code.
|
||||||
|
- **Denoise M1 (perf):** in-source uses non-SIMD `rnnoise.wasm`; the reference
|
||||||
|
preferred SIMD with detection. Perf-only; add SIMD detection later.
|
||||||
|
- **dtln/deepfilternet (F3): RESOLVED** — all four models
|
||||||
|
(rnnoise/speex/dtln/deepfilternet) are now implemented in
|
||||||
|
`lotusDenoiseProcessor.ts` (faithful port of cinny's `build/lotus-denoise.js`
|
||||||
|
pipeline). This also fixed a real bug (the gate worklet name was `noiseGate`;
|
||||||
|
correct is the hyphenated `noise-gate`) and added per-model sample rates
|
||||||
|
(DTLN 16 kHz, others 48 kHz), context `resume()`, and SIMD wasm selection.
|
||||||
|
Still needs live §D testing per model, and depends on cinny shipping the
|
||||||
|
DTLN (`denoise/workadventure/`) + DeepFilterNet (`denoise/deepfilternet/`)
|
||||||
|
asset trees (it already does).
|
||||||
|
- **Rebase-fragility (build agent MED):** the `CallViewModel` spotlight override
|
||||||
|
edits hot upstream lines (renamed `spotlightSpeaker$`→`autoSpotlightSpeaker$`).
|
||||||
|
For cheaper future rebases, refactor it into a `src/lotus/lotusSpotlight.ts`
|
||||||
|
wrapper that takes the upstream stream and returns the overridden one, leaving
|
||||||
|
upstream's definition byte-identical (a single import + two token swaps).
|
||||||
|
- **Denoise asset coupling (build agent HIGH):** the fork loads `./denoise/*`
|
||||||
|
shipped by cinny, not by the fork build (documented in the processor). Add an
|
||||||
|
integration smoke-check that `GET …/element-call/denoise/rnnoise.wasm` == 200,
|
||||||
|
and pin the `@sapphi-red/web-noise-suppressor` version both repos expect.
|
||||||
|
- **Unconditional effect registration (build agent LOW):** focus/audio-inject/
|
||||||
|
quality/decorations register widget handlers on every embedded call (true
|
||||||
|
no-ops for a non-Lotus host). Intentional; gate behind a coarse `lotus=1` flag
|
||||||
|
if strict zero-footprint is desired.
|
||||||
|
- **Privacy (security agent):** decoration/inject URLs accept any `https`; ideally
|
||||||
|
restrict to the homeserver media origin host-side. Call-state exposes
|
||||||
|
userId/deviceId/speaking to the (trusted, same-origin) host — documented.
|
||||||
|
|
||||||
|
**Nothing here blocks the §D live test — but every feature still needs it.**
|
||||||
|
|
||||||
|
### 12.3 Safe rollout when prod is the only test environment
|
||||||
|
|
||||||
|
Every Phase-2 feature is now **dormant by default** — with the flags cinny sets
|
||||||
|
today, the fork behaves identically to the parity build (`#1` was decoupled onto
|
||||||
|
`lotusDenoiseSource=1` so it no longer collides with the host's `lotusDenoise=ml`
|
||||||
|
shim). This enables a low-risk incremental rollout even without a staging env:
|
||||||
|
|
||||||
|
1. **Ship dormant first.** Publish the `lotus` branch (e.g. `0.20.1-lotus.1`),
|
||||||
|
bump cinny's pin, deploy. With no Lotus flags set / no Lotus actions sent,
|
||||||
|
this is upstream-equivalent (only inert, holistically-reviewed code runs).
|
||||||
|
"Testing" here = confirm a normal call still works.
|
||||||
|
2. **Enable ONE feature at a time**, each independently revertable:
|
||||||
|
- URL-flag features (#2 `lotusCallState`, #5 `lotusTransparent`/`lotusTheme`,
|
||||||
|
#1 `lotusDenoiseSource`): add the flag in `CallEmbed.getWidget`, deploy,
|
||||||
|
test that one feature, roll back just that flag if needed.
|
||||||
|
- Action features (#3,#4,#6,#7): wire the host send + (for #2) the
|
||||||
|
`listenAction` ack, gated on join (§12.1 F1).
|
||||||
|
3. **#1 denoise cutover is a coordinated 2-step** (do together): set
|
||||||
|
`lotusDenoiseSource=1` AND remove the `lotusDenoise()` shim injection +
|
||||||
|
`lotusDenoise=ml` param in cinny — otherwise audio is denoised twice.
|
||||||
|
Roll back = revert both.
|
||||||
|
4. Baseline is always upstream-equivalent, so any single feature can be disabled
|
||||||
|
by flipping its flag/send off without touching the rest.
|
||||||
|
|
||||||
|
**Blocker to step 1:** publishing the `lotus` branch needs a Gitea npm token
|
||||||
|
(the admin token used for the `0.20.1` parity publish was deleted). Either
|
||||||
|
provide a token for a manual `npm publish`, or stand up the Gitea Actions runner
|
||||||
|
|
||||||
|
- `GITEA_NPM_TOKEN` secret so a `v0.20.1-lotus.1` tag auto-publishes.
|
||||||
+43
-6
@@ -16,17 +16,15 @@ step-by-step checks in [`LOTUS_TESTING.md`](./LOTUS_TESTING.md).
|
|||||||
Implemented and gate-green; confirm each per `LOTUS_TESTING.md`, then delete the row.
|
Implemented and gate-green; confirm each per `LOTUS_TESTING.md`, then delete the row.
|
||||||
|
|
||||||
| ID | Item | File / area | Test |
|
| ID | Item | File / area | Test |
|
||||||
| :--- | :------------------------------------------------------- | :--------------------------------------------------- | :---- |
|
| :--- | :---------------------------------------------------------------------- | :--------------------------------------------------- | :------- |
|
||||||
| #1 | Camera focus during screenshare ("Focus camera" menu) | `CallControl.ts`, `MemberGlance.tsx` | A5 |
|
|
||||||
| #2 | Chat-background animation flicker (`contain:paint`) | `lotus/chatBackground.ts` | F1 |
|
| #2 | Chat-background animation flicker (`contain:paint`) | `lotus/chatBackground.ts` | F1 |
|
||||||
| #3 | Avatar decorations on call tiles | `call/CallMemberCard.tsx` | A6 |
|
| #4 | Ringtone re-fixes: classic loudness + caller decline notice (A2 ✓ live) | `CallEmbedProvider.tsx`, `ringtones.ts` | A1,A3,A4 |
|
||||||
| #4 | DM/group ringtone selection + in-call banner | `CallEmbedProvider.tsx`, `ringtones.ts` | A1–A4 |
|
|
||||||
| #6 | Background vs. seasonal theme mutual exclusion | `state/settings.ts`, `General.tsx` | F2 |
|
| #6 | Background vs. seasonal theme mutual exclusion | `state/settings.ts`, `General.tsx` | F2 |
|
||||||
| #7 | Composer toolbar touch targets (≥44px) | `room/RoomInput.tsx` | E1 |
|
| #7 | Composer toolbar touch targets (≥44px) | `room/RoomInput.tsx` | E1 |
|
||||||
| #8 | Room Settings horizontal overflow (mobile) | `components/page/style.css.ts` | E2 |
|
| #8 | Room Settings horizontal overflow (mobile) | `components/page/style.css.ts` | E2 |
|
||||||
| #9 | Modal fullscreen on mobile (`useModalStyle`) | 22+ modal files | E3 |
|
| #9 | Modal fullscreen on mobile (`useModalStyle`) | 22+ modal files | E3 |
|
||||||
| #10 | Composer not hidden by keyboard (`100dvh`) | `src/index.css` | E4 |
|
| #10 | Composer not hidden by keyboard (`100dvh`) | `src/index.css` | E4 |
|
||||||
| #12 | PiP mute badge attribution (you vs. all-muted) | `CallEmbedProvider.tsx` | G1 |
|
| #12 | PiP "All muted" badge re-fixed (was firing on any single mute) | `hooks/useCallSpeakers.ts` | G1 |
|
||||||
| N96 | Call-recovery overlay single "Back" button | `call/CallView.tsx` | A7 |
|
| N96 | Call-recovery overlay single "Back" button | `call/CallView.tsx` | A7 |
|
||||||
| N95 | AFK-monitor mic released on mute (OS indicator clears) | `hooks/useAfkAutoMute.ts` | L1 |
|
| N95 | AFK-monitor mic released on mute (OS indicator clears) | `hooks/useAfkAutoMute.ts` | L1 |
|
||||||
| N108 | Maskable PWA icons (Android adaptive) | `public/manifest.json` + `res/android/maskable-*` | L2 |
|
| N108 | Maskable PWA icons (Android adaptive) | `public/manifest.json` + `res/android/maskable-*` | L2 |
|
||||||
@@ -34,13 +32,52 @@ Implemented and gate-green; confirm each per `LOTUS_TESTING.md`, then delete the
|
|||||||
| Gal | MediaGallery lazy-decrypt (true virtualization deferred) | `room/MediaGallery.tsx` | H1 |
|
| Gal | MediaGallery lazy-decrypt (true virtualization deferred) | `room/MediaGallery.tsx` | H1 |
|
||||||
| a11y | aria-labels: edit-history / reaction / thread / reply | `message/*` (`FallbackContent`, `Reaction`, `Reply`) | I |
|
| a11y | aria-labels: edit-history / reaction / thread / reply | `message/*` (`FallbackContent`, `Reaction`, `Reply`) | I |
|
||||||
|
|
||||||
|
**Verified working in live testing (2026-06):** A2, B1–B4, C1, C3, D (mic/camera/deafen/screenshare/fullscreen/more-menu/PiP). Denoise quality in D is still poor — tracked under the denoise project, not a regression.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧩 Element Call source-level items — now actionable via the fork
|
||||||
|
|
||||||
|
> 🔱 **[EC-FORK]** **UPDATE 2026-06-29: the fork is live.** We now own and
|
||||||
|
> self-build Element Call (`LotusGuild/element-call` →
|
||||||
|
> `@lotusguild/element-call-embedded`, Phase 1 done & cinny wired). A5/A6/A7
|
||||||
|
> below are **no longer "won't fix"** — they are ordinary source changes. See
|
||||||
|
> [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md) §10 + the Phase
|
||||||
|
> 2 work list. (The iframe is **same-origin** / self-hosted; the old blocker was
|
||||||
|
> that we didn't own EC's compiled source — which we now do.)
|
||||||
|
|
||||||
|
The in-call participant grid is rendered **inside EC's app**. Previously a
|
||||||
|
pre-built npm bundle we could only style/place around; now editable source.
|
||||||
|
Items from testing, with their fork-level fix path:
|
||||||
|
|
||||||
|
- **A5 — "Focus camera":** EC supports native tile-pinning. Our bottom-bar "Focus
|
||||||
|
camera" is a programmatic wrapper that **`.click()`s the tile** today
|
||||||
|
(`CallControl.ts` `focusCameraParticipant`), and during a screenshare EC
|
||||||
|
spotlights the shared screen so a camera pin may not override it. **Fork fix:**
|
||||||
|
add an `io.lotus.focus_participant` widget action that pins a participant in
|
||||||
|
EC's layout (coexisting with / overriding the screenshare spotlight); cinny
|
||||||
|
sends it via the widget API and the DOM-click hack is deleted. _Status: Open —
|
||||||
|
Actionable (Phase 2)._
|
||||||
|
- **A6 — avatar decorations in-call:** decorations render on **our** pre-join
|
||||||
|
lobby roster (`CallMemberCard`) but not on EC's in-call video tiles. **Fork
|
||||||
|
fix:** render the decoration APNG inside EC's participant-tile component, fed
|
||||||
|
decoration slugs via widget member data. _Status: Open — Actionable (Phase 2)._
|
||||||
|
- **A7 — mic dead after EC's "Reconnect":** the mid-call "Connection lost /
|
||||||
|
Reconnect" screen is **EC's own** (our load watchdog only covers an initial
|
||||||
|
hung load). After EC reconnects, the mic isn't re-published through our denoise
|
||||||
|
`getUserMedia` shim until a clean End+rejoin. **Fork fix:** move denoise into
|
||||||
|
EC's mic-capture/publish pipeline as a first-class audio stage — EC re-runs it
|
||||||
|
on every (re)publish, so reconnects keep denoise alive natively, and the
|
||||||
|
build-time `index.html` injection is removed. _Status: Open — Actionable
|
||||||
|
(Phase 2); root cause is the `getUserMedia` monkeypatch, not EC itself._
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔴 Open — Actionable
|
## 🔴 Open — Actionable
|
||||||
|
|
||||||
### Calls / Audio
|
### Calls / Audio
|
||||||
|
|
||||||
- **N127 — ML denoise shim is never injected in `vite dev`.** The `lotusDenoise` plugin injects only on `closeBundle` (build), so ML noise suppression is silently inactive during local dev. Add a dev-mode injection (`configureServer` / `transformIndexHtml`). Dev-only impact.
|
- **N127 — ML denoise shim is never injected in `vite dev`.** The `lotusDenoise` plugin injects only on `closeBundle` (build), so ML noise suppression is silently inactive during local dev. Add a dev-mode injection (`configureServer` / `transformIndexHtml`). Dev-only impact. _Note: this **dissolves entirely** once denoise moves in-source in the fork (A7 fix) — there is then no build-time injection to be missing in dev._
|
||||||
|
|
||||||
### Security & Privacy
|
### Security & Privacy
|
||||||
|
|
||||||
|
|||||||
@@ -322,6 +322,11 @@ Users can set a custom background color for `@mention` chips that highlight thei
|
|||||||
|
|
||||||
## Voice / Video Call Improvements
|
## Voice / Video Call Improvements
|
||||||
|
|
||||||
|
> 🔱 **[EC-FORK]** Element Call is embedded as a **pre-built npm bundle** today.
|
||||||
|
> The plan to fork & self-build it from source for true ownership — and which of
|
||||||
|
> the items below would move into our EC source — is in
|
||||||
|
> [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md).
|
||||||
|
|
||||||
### Element Call Upgrade
|
### Element Call Upgrade
|
||||||
|
|
||||||
Upgraded embedded Element Call widget from **0.16.3** to **0.19.4**.
|
Upgraded embedded Element Call widget from **0.16.3** to **0.19.4**.
|
||||||
|
|||||||
@@ -264,6 +264,7 @@ Features:
|
|||||||
|
|
||||||
**What:** Grid of short audio clips playable into the call audio stream via Web Audio API (AudioBufferSourceNode → MediaStreamDestinationNode → mixed with mic). Built-in clips + user-uploadable custom clips (stored as mxc://). Accessible from call controls bar.
|
**What:** Grid of short audio clips playable into the call audio stream via Web Audio API (AudioBufferSourceNode → MediaStreamDestinationNode → mixed with mic). Built-in clips + user-uploadable custom clips (stored as mxc://). Accessible from call controls bar.
|
||||||
**[AUDIT REQUIRED]** Verify the Element Call integration exposes the mic MediaStream for mixing. This is the highest-risk part of this feature.
|
**[AUDIT REQUIRED]** Verify the Element Call integration exposes the mic MediaStream for mixing. This is the highest-risk part of this feature.
|
||||||
|
**🔱 [EC-FORK]** Owning the EC source (see [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md)) would unblock real audio-injection — a proper soundboard mixed into the call — which is impossible against the prebuilt bundle today.
|
||||||
**Complexity:** High.
|
**Complexity:** High.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -281,6 +282,7 @@ Features:
|
|||||||
|
|
||||||
**What:** High-end background noise cancellation using a pre-trained ML model (RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
|
**What:** High-end background noise cancellation using a pre-trained ML model (RNNoise) running in the browser. Removes dogs, fans, and keyboard clicks from the mic stream.
|
||||||
**Shipped:** 3-tier setting (Off / Browser-native / ML) in Settings → General → Calls. ML tier injects a same-origin pre-init shim into the vendored Element Call `index.html` that monkeypatches `getUserMedia` and routes the captured mic through an RNNoise `AudioWorklet` before LiveKit publishes — no EC fork required. See LOTUS_FEATURES.md → "Noise Suppression (Advanced Multi-Tier)".
|
**Shipped:** 3-tier setting (Off / Browser-native / ML) in Settings → General → Calls. ML tier injects a same-origin pre-init shim into the vendored Element Call `index.html` that monkeypatches `getUserMedia` and routes the captured mic through an RNNoise `AudioWorklet` before LiveKit publishes — no EC fork required. See LOTUS_FEATURES.md → "Noise Suppression (Advanced Multi-Tier)".
|
||||||
|
**🔱 [EC-FORK]** Once we own the EC source (see [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md)), denoise should become a first-class audio stage **inside** EC instead of an `index.html` getUserMedia monkeypatch — more robust, survives reconnects (fixes the A7 mic-after-reconnect bug), and removes the build-time injection hack.
|
||||||
**Key decision:** LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU); EC's own RNNoise PR #3892 is unmerged. The shim is the same post-capture pipeline #3892 uses, executed from the realm we control, so it survives EC version bumps.
|
**Key decision:** LiveKit's Krisp filter is LiveKit-Cloud-only (we self-host the SFU); EC's own RNNoise PR #3892 is unmerged. The shim is the same post-capture pipeline #3892 uses, executed from the realm we control, so it survives EC version bumps.
|
||||||
**AEC note (resolved-as-accepted):** WebAudio capture routing can weaken browser AEC — same tradeoff as EC's upstream feature; mitigated by keeping `echoCancellation`/`autoGainControl` on the raw capture and labeling the tier "beta".
|
**AEC note (resolved-as-accepted):** WebAudio capture routing can weaken browser AEC — same tradeoff as EC's upstream feature; mitigated by keeping `echoCancellation`/`autoGainControl` on the raw capture and labeling the tier "beta".
|
||||||
|
|
||||||
@@ -531,6 +533,8 @@ Exhaustive, low-level implementation details for backlog items. Follow these pat
|
|||||||
- Pass the destination's `.stream` to the call bridge.
|
- Pass the destination's `.stream` to the call bridge.
|
||||||
|
|
||||||
> ⚠️ **[Gemini_Found — CORRECTED]** Gemini originally suggested using LiveKit's `LocalAudioTrack.replaceTrack()` to mix audio into the call stream. This is **not possible** from Lotus Chat's realm: Element Call runs in a **cross-origin iframe** controlled via `matrix-widget-api` (postMessage). LiveKit's JS SDK and its `LocalAudioTrack` live inside EC's sandboxed context — inaccessible from our code. This directly contradicts the confirmed constraint already listed in the Server Capabilities table: _"Cindy CANNOT inject audio into EC call stream — In-call soundboard must be redesigned as local-only."_ The soundboard must be a local-playback-only feature (output through the user's speakers, not mixed into the call audio stream).
|
> ⚠️ **[Gemini_Found — CORRECTED]** Gemini originally suggested using LiveKit's `LocalAudioTrack.replaceTrack()` to mix audio into the call stream. This is **not possible** from Lotus Chat's realm: Element Call runs in a **cross-origin iframe** controlled via `matrix-widget-api` (postMessage). LiveKit's JS SDK and its `LocalAudioTrack` live inside EC's sandboxed context — inaccessible from our code. This directly contradicts the confirmed constraint already listed in the Server Capabilities table: _"Cindy CANNOT inject audio into EC call stream — In-call soundboard must be redesigned as local-only."_ The soundboard must be a local-playback-only feature (output through the user's speakers, not mixed into the call audio stream).
|
||||||
|
>
|
||||||
|
> 🔱 **[EC-FORK — partial correction]** The "cross-origin" claim above is **outdated**: EC is now **same-origin** / self-hosted (`iframe.sandbox` has `allow-same-origin`; we read `contentDocument`). The _practical_ blocker still holds — LiveKit's `LocalAudioTrack` lives in EC's **module scope** (not on `window`), so it's unreachable from cinny even same-origin. **Owning the EC source** (see [`HANDOFF_ELEMENT_CALL_FORK.md`](./HANDOFF_ELEMENT_CALL_FORK.md)) is the path to a real call-audio-inject API, which would unblock a true in-call soundboard.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -144,6 +144,23 @@ The source code lives in `/root/code/cinny`. All changes should be made on the `
|
|||||||
|
|
||||||
See [LOTUS_FEATURES.md](LOTUS_FEATURES.md) for the full feature changelog and [LOTUS_TODO.md](LOTUS_TODO.md) for the work backlog.
|
See [LOTUS_FEATURES.md](LOTUS_FEATURES.md) for the full feature changelog and [LOTUS_TODO.md](LOTUS_TODO.md) for the work backlog.
|
||||||
|
|
||||||
|
### 🔱 Planned: Element Call fork ("Lotus Call")
|
||||||
|
|
||||||
|
Voice/video channels embed **Element Call**. Today it's a **pre-built npm bundle**
|
||||||
|
(`@element-hq/element-call-embedded` 0.20.1) copied to `public/element-call/` and
|
||||||
|
served same-origin; we steer it via the `matrix-widget-api` plus fragile DOM
|
||||||
|
hacks. Because we don't own its compiled source, several in-call issues (avatar
|
||||||
|
decorations on tiles, camera focus/fullscreen during screenshare, mic recovery
|
||||||
|
after reconnect, native theming, real call-audio injection) are unfixable from
|
||||||
|
outside.
|
||||||
|
|
||||||
|
**The plan is to fork `element-hq/element-call` into a new `LotusGuild/element-call`
|
||||||
|
repo, build it from source, and host our own build** for true ownership. The full
|
||||||
|
self-contained plan and integration map — written for a fresh session with no
|
||||||
|
prior context — is in **[`HANDOFF_ELEMENT_CALL_FORK.md`](HANDOFF_ELEMENT_CALL_FORK.md)**.
|
||||||
|
Infra/hosting notes also live in the `LotusGuild/matrix` repo README. Search the
|
||||||
|
docs for the **`[EC-FORK]`** tag to find every related note.
|
||||||
|
|
||||||
### Build
|
### Build
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
Generated
+7
-7
@@ -77,7 +77,7 @@
|
|||||||
"ua-parser-js": "2.0.10"
|
"ua-parser-js": "2.0.10"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@element-hq/element-call-embedded": "0.20.1",
|
"@lotusguild/element-call-embedded": "0.20.1-lotus.1",
|
||||||
"@rollup/plugin-inject": "5.0.5",
|
"@rollup/plugin-inject": "5.0.5",
|
||||||
"@rollup/plugin-wasm": "6.2.2",
|
"@rollup/plugin-wasm": "6.2.2",
|
||||||
"@types/chroma-js": "3.1.2",
|
"@types/chroma-js": "3.1.2",
|
||||||
@@ -1788,12 +1788,6 @@
|
|||||||
"node": ">=v18"
|
"node": ">=v18"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"node_modules/@element-hq/element-call-embedded": {
|
|
||||||
"version": "0.20.1",
|
|
||||||
"resolved": "https://registry.npmjs.org/@element-hq/element-call-embedded/-/element-call-embedded-0.20.1.tgz",
|
|
||||||
"integrity": "sha512-ODg2r7UmR8UjRpapLKbn6v1PS8fu/r58zdbvXMYaAlUEAC2f6L/9Moc9S4noG1+ARgWxY+m2vLmNDK9G9uFZYQ==",
|
|
||||||
"dev": true
|
|
||||||
},
|
|
||||||
"node_modules/@emnapi/core": {
|
"node_modules/@emnapi/core": {
|
||||||
"version": "1.10.0",
|
"version": "1.10.0",
|
||||||
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz",
|
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz",
|
||||||
@@ -2693,6 +2687,12 @@
|
|||||||
"resolved": "https://registry.npmjs.org/@juggle/resize-observer/-/resize-observer-3.4.0.tgz",
|
"resolved": "https://registry.npmjs.org/@juggle/resize-observer/-/resize-observer-3.4.0.tgz",
|
||||||
"integrity": "sha512-dfLbk+PwWvFzSxwk3n5ySL0hfBog779o8h68wK/7/APo/7cgyWp5jcXockbxdk5kFRkbeXWm4Fbi9FrdN381sA=="
|
"integrity": "sha512-dfLbk+PwWvFzSxwk3n5ySL0hfBog779o8h68wK/7/APo/7cgyWp5jcXockbxdk5kFRkbeXWm4Fbi9FrdN381sA=="
|
||||||
},
|
},
|
||||||
|
"node_modules/@lotusguild/element-call-embedded": {
|
||||||
|
"version": "0.20.1-lotus.1",
|
||||||
|
"resolved": "https://code.lotusguild.org/api/packages/LotusGuild/npm/%40lotusguild%2Felement-call-embedded/-/0.20.1-lotus.1/element-call-embedded-0.20.1-lotus.1.tgz",
|
||||||
|
"integrity": "sha512-hy1KEnFw4MuwvlactUFPPvvtPZh1y56JMK/ehnficUmJNwdJsOhSwThaYp35RZ/ar6RCuiW86yQqlQBOSpZJVQ==",
|
||||||
|
"dev": true
|
||||||
|
},
|
||||||
"node_modules/@matrix-org/matrix-sdk-crypto-wasm": {
|
"node_modules/@matrix-org/matrix-sdk-crypto-wasm": {
|
||||||
"version": "18.3.0",
|
"version": "18.3.0",
|
||||||
"resolved": "https://registry.npmjs.org/@matrix-org/matrix-sdk-crypto-wasm/-/matrix-sdk-crypto-wasm-18.3.0.tgz",
|
"resolved": "https://registry.npmjs.org/@matrix-org/matrix-sdk-crypto-wasm/-/matrix-sdk-crypto-wasm-18.3.0.tgz",
|
||||||
|
|||||||
+1
-1
@@ -101,7 +101,7 @@
|
|||||||
"ua-parser-js": "2.0.10"
|
"ua-parser-js": "2.0.10"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@element-hq/element-call-embedded": "0.20.1",
|
"@lotusguild/element-call-embedded": "0.20.1-lotus.1",
|
||||||
"@rollup/plugin-inject": "5.0.5",
|
"@rollup/plugin-inject": "5.0.5",
|
||||||
"@rollup/plugin-wasm": "6.2.2",
|
"@rollup/plugin-wasm": "6.2.2",
|
||||||
"@types/chroma-js": "3.1.2",
|
"@types/chroma-js": "3.1.2",
|
||||||
|
|||||||
@@ -37,6 +37,7 @@ import {
|
|||||||
useCallStart,
|
useCallStart,
|
||||||
} from '../hooks/useCallEmbed';
|
} from '../hooks/useCallEmbed';
|
||||||
import { callChatAtom, callEmbedAtom } from '../state/callEmbed';
|
import { callChatAtom, callEmbedAtom } from '../state/callEmbed';
|
||||||
|
import { toastQueueAtom } from '../state/toast';
|
||||||
import { CallEmbed, useCallControlState } from '../plugins/call';
|
import { CallEmbed, useCallControlState } from '../plugins/call';
|
||||||
import { useSelectedRoom } from '../hooks/router/useSelectedRoom';
|
import { useSelectedRoom } from '../hooks/router/useSelectedRoom';
|
||||||
import { ScreenSize, useScreenSizeContext } from '../hooks/useScreenSize';
|
import { ScreenSize, useScreenSizeContext } from '../hooks/useScreenSize';
|
||||||
@@ -64,6 +65,7 @@ import { getRoomPermissionsAPI } from '../hooks/useRoomPermissions';
|
|||||||
import { useLivekitSupport } from '../hooks/useLivekitSupport';
|
import { useLivekitSupport } from '../hooks/useLivekitSupport';
|
||||||
import { CallAvatarAnimation } from '../styles/Animations.css';
|
import { CallAvatarAnimation } from '../styles/Animations.css';
|
||||||
import { webRTCSupported } from '../utils/rtc';
|
import { webRTCSupported } from '../utils/rtc';
|
||||||
|
import { zIndices } from '../styles/zIndex';
|
||||||
|
|
||||||
const PIP_MIN_W = 200;
|
const PIP_MIN_W = 200;
|
||||||
const PIP_MIN_H = 112;
|
const PIP_MIN_H = 112;
|
||||||
@@ -323,7 +325,7 @@ function IncomingCallBanner({ dm, info, onIgnore, onAnswer, onReject }: Incoming
|
|||||||
position: 'fixed',
|
position: 'fixed',
|
||||||
top: config.space.S400,
|
top: config.space.S400,
|
||||||
right: config.space.S400,
|
right: config.space.S400,
|
||||||
zIndex: 9990,
|
zIndex: zIndices.inCallBanner,
|
||||||
width: toRem(300),
|
width: toRem(300),
|
||||||
maxWidth: `calc(100vw - 2 * ${config.space.S400})`,
|
maxWidth: `calc(100vw - 2 * ${config.space.S400})`,
|
||||||
padding: config.space.S300,
|
padding: config.space.S300,
|
||||||
@@ -404,6 +406,7 @@ function IncomingCallListener({ callEmbed, joined }: IncomingCallListenerProps)
|
|||||||
const mx = useMatrixClient();
|
const mx = useMatrixClient();
|
||||||
const directs = useAtomValue(mDirectAtom);
|
const directs = useAtomValue(mDirectAtom);
|
||||||
const { navigateRoom } = useRoomNavigate();
|
const { navigateRoom } = useRoomNavigate();
|
||||||
|
const setToast = useSetAtom(toastQueueAtom);
|
||||||
|
|
||||||
const [callInfo, setCallInfo] = useState<IncomingCallInfo>();
|
const [callInfo, setCallInfo] = useState<IncomingCallInfo>();
|
||||||
const dm = callInfo ? directs.has(callInfo.room.roomId) : false;
|
const dm = callInfo ? directs.has(callInfo.room.roomId) : false;
|
||||||
@@ -423,6 +426,31 @@ function IncomingCallListener({ callEmbed, joined }: IncomingCallListenerProps)
|
|||||||
await event.getDecryptionPromise();
|
await event.getDecryptionPromise();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Caller-side: a participant declined a call we're hosting in this room.
|
||||||
|
// Without this the caller's UI keeps "ringing" until the notification
|
||||||
|
// lifetime expires, with no indication the callee said no.
|
||||||
|
if (event.getType() === EventType.RTCDecline) {
|
||||||
|
const decliner = event.getSender();
|
||||||
|
if (
|
||||||
|
data.liveEvent &&
|
||||||
|
room &&
|
||||||
|
decliner &&
|
||||||
|
decliner !== mx.getSafeUserId() &&
|
||||||
|
callEmbed?.roomId === room.roomId
|
||||||
|
) {
|
||||||
|
const declinerName =
|
||||||
|
getMemberDisplayName(room, decliner) ?? getMxIdLocalPart(decliner) ?? decliner;
|
||||||
|
setToast({
|
||||||
|
id: `rtc-decline-${event.getId() ?? decliner}`,
|
||||||
|
displayName: declinerName,
|
||||||
|
body: 'Declined your call',
|
||||||
|
roomName: room.name,
|
||||||
|
roomId: room.roomId,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
if (
|
if (
|
||||||
!room ||
|
!room ||
|
||||||
event.getType() !== EventType.RTCNotification ||
|
event.getType() !== EventType.RTCNotification ||
|
||||||
@@ -485,7 +513,7 @@ function IncomingCallListener({ callEmbed, joined }: IncomingCallListenerProps)
|
|||||||
|
|
||||||
setCallInfo(info);
|
setCallInfo(info);
|
||||||
},
|
},
|
||||||
[mx, directs],
|
[mx, directs, callEmbed, setToast],
|
||||||
);
|
);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
/* eslint-disable @typescript-eslint/no-explicit-any */
|
/* eslint-disable @typescript-eslint/no-explicit-any */
|
||||||
import React, { useCallback, useEffect, useState } from 'react';
|
import React, { useCallback, useEffect, useState } from 'react';
|
||||||
import { Box, color, config, Text, toRem } from 'folds';
|
import { Box, color, config, Icon, Icons, Text, toRem } from 'folds';
|
||||||
import { RelationsEvent } from 'matrix-js-sdk/lib/models/relations';
|
import { RelationsEvent } from 'matrix-js-sdk/lib/models/relations';
|
||||||
import { RoomEvent } from 'matrix-js-sdk';
|
import { RoomEvent } from 'matrix-js-sdk';
|
||||||
import { useMatrixClient } from '../../../hooks/useMatrixClient';
|
import { useMatrixClient } from '../../../hooks/useMatrixClient';
|
||||||
@@ -339,11 +339,7 @@ export function PollContent({
|
|||||||
transition: 'all 0.15s',
|
transition: 'all 0.15s',
|
||||||
}}
|
}}
|
||||||
>
|
>
|
||||||
{selected && isMultiple ? (
|
{selected && isMultiple ? <Icon size="50" src={Icons.Check} /> : null}
|
||||||
<Text as="span" size="T200" style={{ lineHeight: 1 }}>
|
|
||||||
✓
|
|
||||||
</Text>
|
|
||||||
) : null}
|
|
||||||
</span>
|
</span>
|
||||||
<Text as="span" size="T300" style={{ flexGrow: 1 }}>
|
<Text as="span" size="T300" style={{ flexGrow: 1 }}>
|
||||||
{text}
|
{text}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import React, { useMemo } from 'react';
|
import React, { useMemo } from 'react';
|
||||||
import { useAtomValue } from 'jotai';
|
import { useAtomValue } from 'jotai';
|
||||||
import { settingsAtom } from '../../state/settings';
|
import { settingsAtom } from '../../state/settings';
|
||||||
|
import { zIndices } from '../../styles/zIndex';
|
||||||
import {
|
import {
|
||||||
animSeasonFall,
|
animSeasonFall,
|
||||||
animLeafFall,
|
animLeafFall,
|
||||||
@@ -758,7 +759,7 @@ function SeasonalOverlay({ theme, reduced }: { theme: SeasonTheme; reduced: bool
|
|||||||
pointerEvents: 'none',
|
pointerEvents: 'none',
|
||||||
// Below the Night Light overlay (9998) so seasonal particles are tinted
|
// Below the Night Light overlay (9998) so seasonal particles are tinted
|
||||||
// by it, and below modals (9999) so dialogs are never obscured.
|
// by it, and below modals (9999) so dialogs are never obscured.
|
||||||
zIndex: 9997,
|
zIndex: zIndices.seasonalEffect,
|
||||||
overflow: 'hidden',
|
overflow: 'hidden',
|
||||||
}}
|
}}
|
||||||
>
|
>
|
||||||
|
|||||||
@@ -0,0 +1,98 @@
|
|||||||
|
import React, { MouseEventHandler, useState } from 'react';
|
||||||
|
import { Box, Button, config, Icon, Icons, Menu, MenuItem, PopOut, RectCords, Text } from 'folds';
|
||||||
|
import FocusTrap from 'focus-trap-react';
|
||||||
|
import { stopPropagation } from '../../utils/keyboard';
|
||||||
|
|
||||||
|
export type SettingsSelectOption<T extends string> = {
|
||||||
|
value: T;
|
||||||
|
label: string;
|
||||||
|
disabled?: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A folds-native dropdown (Button + PopOut + Menu) matching Cinny's select
|
||||||
|
* pattern — used instead of a raw `<select>`, which renders OS-styled and
|
||||||
|
* breaks under non-default themes.
|
||||||
|
*/
|
||||||
|
export function SettingsSelect<T extends string>({
|
||||||
|
value,
|
||||||
|
options,
|
||||||
|
onChange,
|
||||||
|
disabled,
|
||||||
|
'aria-label': ariaLabel,
|
||||||
|
}: {
|
||||||
|
value: T;
|
||||||
|
options: SettingsSelectOption<T>[];
|
||||||
|
onChange: (v: T) => void;
|
||||||
|
disabled?: boolean;
|
||||||
|
'aria-label'?: string;
|
||||||
|
}) {
|
||||||
|
const [menuCords, setMenuCords] = useState<RectCords>();
|
||||||
|
const selectedLabel = options.find((o) => o.value === value)?.label ?? value;
|
||||||
|
|
||||||
|
const handleMenu: MouseEventHandler<HTMLButtonElement> = (evt) => {
|
||||||
|
setMenuCords(evt.currentTarget.getBoundingClientRect());
|
||||||
|
};
|
||||||
|
|
||||||
|
const handleSelect = (v: T) => {
|
||||||
|
onChange(v);
|
||||||
|
setMenuCords(undefined);
|
||||||
|
};
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Button
|
||||||
|
size="300"
|
||||||
|
variant="Secondary"
|
||||||
|
outlined
|
||||||
|
fill="Soft"
|
||||||
|
radii="300"
|
||||||
|
after={<Icon size="300" src={Icons.ChevronBottom} />}
|
||||||
|
onClick={handleMenu}
|
||||||
|
disabled={disabled}
|
||||||
|
aria-label={ariaLabel}
|
||||||
|
aria-haspopup="menu"
|
||||||
|
aria-expanded={!!menuCords}
|
||||||
|
>
|
||||||
|
<Text size="T300">{selectedLabel}</Text>
|
||||||
|
</Button>
|
||||||
|
<PopOut
|
||||||
|
anchor={menuCords}
|
||||||
|
offset={5}
|
||||||
|
position="Bottom"
|
||||||
|
align="End"
|
||||||
|
content={
|
||||||
|
<FocusTrap
|
||||||
|
focusTrapOptions={{
|
||||||
|
initialFocus: false,
|
||||||
|
onDeactivate: () => setMenuCords(undefined),
|
||||||
|
clickOutsideDeactivates: true,
|
||||||
|
isKeyForward: (evt: KeyboardEvent) =>
|
||||||
|
evt.key === 'ArrowDown' || evt.key === 'ArrowRight',
|
||||||
|
isKeyBackward: (evt: KeyboardEvent) =>
|
||||||
|
evt.key === 'ArrowUp' || evt.key === 'ArrowLeft',
|
||||||
|
escapeDeactivates: stopPropagation,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
<Menu>
|
||||||
|
<Box direction="Column" gap="100" style={{ padding: config.space.S100 }}>
|
||||||
|
{options.map((opt) => (
|
||||||
|
<MenuItem
|
||||||
|
key={opt.value}
|
||||||
|
size="300"
|
||||||
|
variant={opt.value === value ? 'Primary' : 'Surface'}
|
||||||
|
radii="300"
|
||||||
|
disabled={opt.disabled}
|
||||||
|
onClick={() => !opt.disabled && handleSelect(opt.value)}
|
||||||
|
>
|
||||||
|
<Text size="T300">{opt.label}</Text>
|
||||||
|
</MenuItem>
|
||||||
|
))}
|
||||||
|
</Box>
|
||||||
|
</Menu>
|
||||||
|
</FocusTrap>
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -17,6 +17,7 @@ import { useRoomPermissions } from '../../hooks/useRoomPermissions';
|
|||||||
import { useMatrixClient } from '../../hooks/useMatrixClient';
|
import { useMatrixClient } from '../../hooks/useMatrixClient';
|
||||||
import { StateEvent } from '../../../types/matrix/room';
|
import { StateEvent } from '../../../types/matrix/room';
|
||||||
import { useCallMembers, useCallSession } from '../../hooks/useCall';
|
import { useCallMembers, useCallSession } from '../../hooks/useCall';
|
||||||
|
import { LotusDecorationPusher } from '../lotus/LotusDecorationPusher';
|
||||||
import { useStateEvent } from '../../hooks/useStateEvent';
|
import { useStateEvent } from '../../hooks/useStateEvent';
|
||||||
import { VoiceLimitContent } from '../common-settings/general/RoomVoiceLimit';
|
import { VoiceLimitContent } from '../common-settings/general/RoomVoiceLimit';
|
||||||
import { CallMemberRenderer } from './CallMemberCard';
|
import { CallMemberRenderer } from './CallMemberCard';
|
||||||
@@ -199,6 +200,8 @@ function CallJoined({ joined, containerRef }: CallJoinedProps) {
|
|||||||
<Box grow="Yes" direction="Column">
|
<Box grow="Yes" direction="Column">
|
||||||
<Box grow="Yes" ref={containerRef} />
|
<Box grow="Yes" ref={containerRef} />
|
||||||
{callEmbed && joined && <CallControls callEmbed={callEmbed} />}
|
{callEmbed && joined && <CallControls callEmbed={callEmbed} />}
|
||||||
|
{/* [lotus #6] push avatar decorations to EC's in-call tiles (post-join) */}
|
||||||
|
{callEmbed && joined && <LotusDecorationPusher callEmbed={callEmbed} />}
|
||||||
</Box>
|
</Box>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
import React, { useCallback, useEffect, useMemo, useRef, type ReactElement } from 'react';
|
||||||
|
import { type CallEmbed } from '../../plugins/call';
|
||||||
|
import { useCallMembers, useCallSession } from '../../hooks/useCall';
|
||||||
|
import { useAvatarDecoration } from '../../hooks/useAvatarDecoration';
|
||||||
|
import { decorationUrl } from './avatarDecorations';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* [lotus #6] Pushes each call participant's avatar-decoration image URL to the
|
||||||
|
* forked Element Call (`io.lotus.decorations`), which renders it on the in-call
|
||||||
|
* video-tile avatars. Mounted only while joined, so the EC-side handler exists.
|
||||||
|
*
|
||||||
|
* The decoration roster is per-user slugs resolved via `useAvatarDecoration`;
|
||||||
|
* we render one invisible probe per member to reuse that hook + its cache, then
|
||||||
|
* debounce-send the aggregated `{ userId: url }` map whenever it changes.
|
||||||
|
*/
|
||||||
|
function DecorationProbe({
|
||||||
|
userId,
|
||||||
|
onResolve,
|
||||||
|
}: {
|
||||||
|
userId: string;
|
||||||
|
onResolve: (userId: string, url: string | null) => void;
|
||||||
|
}): null {
|
||||||
|
const slug = useAvatarDecoration(userId);
|
||||||
|
useEffect(() => {
|
||||||
|
onResolve(userId, slug ? decorationUrl(slug) : null);
|
||||||
|
}, [userId, slug, onResolve]);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function LotusDecorationPusher({ callEmbed }: { callEmbed: CallEmbed }): ReactElement {
|
||||||
|
const session = useCallSession(callEmbed.room);
|
||||||
|
const members = useCallMembers(session);
|
||||||
|
const map = useRef<Map<string, string>>(new Map());
|
||||||
|
const pushTimer = useRef<ReturnType<typeof setTimeout> | undefined>(undefined);
|
||||||
|
|
||||||
|
const userIds = useMemo(
|
||||||
|
() => Array.from(new Set(members.map((m) => m.userId).filter((u): u is string => !!u))),
|
||||||
|
[members],
|
||||||
|
);
|
||||||
|
|
||||||
|
const push = useCallback(() => {
|
||||||
|
const decorations: Record<string, string> = {};
|
||||||
|
map.current.forEach((url, userId) => {
|
||||||
|
decorations[userId] = url;
|
||||||
|
});
|
||||||
|
callEmbed.call.transport.send('io.lotus.decorations', { decorations }).catch(() => undefined);
|
||||||
|
}, [callEmbed]);
|
||||||
|
|
||||||
|
const schedulePush = useCallback(() => {
|
||||||
|
if (pushTimer.current) clearTimeout(pushTimer.current);
|
||||||
|
pushTimer.current = setTimeout(push, 300);
|
||||||
|
}, [push]);
|
||||||
|
|
||||||
|
const onResolve = useCallback(
|
||||||
|
(userId: string, url: string | null) => {
|
||||||
|
const prev = map.current.get(userId);
|
||||||
|
if (url) {
|
||||||
|
if (prev !== url) {
|
||||||
|
map.current.set(userId, url);
|
||||||
|
schedulePush();
|
||||||
|
}
|
||||||
|
} else if (prev !== undefined) {
|
||||||
|
map.current.delete(userId);
|
||||||
|
schedulePush();
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[schedulePush],
|
||||||
|
);
|
||||||
|
|
||||||
|
// Drop decorations for participants who left the call.
|
||||||
|
useEffect(() => {
|
||||||
|
const present = new Set(userIds);
|
||||||
|
let changed = false;
|
||||||
|
map.current.forEach((_url, userId) => {
|
||||||
|
if (!present.has(userId)) {
|
||||||
|
map.current.delete(userId);
|
||||||
|
changed = true;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
if (changed) schedulePush();
|
||||||
|
}, [userIds, schedulePush]);
|
||||||
|
|
||||||
|
useEffect(
|
||||||
|
() => () => {
|
||||||
|
if (pushTimer.current) clearTimeout(pushTimer.current);
|
||||||
|
},
|
||||||
|
[],
|
||||||
|
);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
{userIds.map((userId) => (
|
||||||
|
<DecorationProbe key={userId} userId={userId} onResolve={onResolve} />
|
||||||
|
))}
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -133,7 +133,7 @@ function EncryptedRoomCachePanel({ roomIds, onLoaded }: EncryptedRoomCachePanelP
|
|||||||
<Text size="T300" truncate>
|
<Text size="T300" truncate>
|
||||||
{room.name}
|
{room.name}
|
||||||
</Text>
|
</Text>
|
||||||
<Text size="T200" style={{ opacity: 0.55 }}>
|
<Text size="T200" priority="300">
|
||||||
{msgEvents.length > 0
|
{msgEvents.length > 0
|
||||||
? `${msgEvents.length} messages cached · oldest: ${new Date(oldest!.getTs()).toLocaleDateString()}`
|
? `${msgEvents.length} messages cached · oldest: ${new Date(oldest!.getTs()).toLocaleDateString()}`
|
||||||
: 'No messages cached yet'}
|
: 'No messages cached yet'}
|
||||||
@@ -153,7 +153,7 @@ function EncryptedRoomCachePanel({ roomIds, onLoaded }: EncryptedRoomCachePanelP
|
|||||||
</Button>
|
</Button>
|
||||||
)}
|
)}
|
||||||
{!canLoadMore && events.length > 0 && (
|
{!canLoadMore && events.length > 0 && (
|
||||||
<Text size="T200" style={{ opacity: 0.5, flexShrink: 0 }}>
|
<Text size="T200" priority="300" style={{ flexShrink: 0 }}>
|
||||||
Fully cached
|
Fully cached
|
||||||
</Text>
|
</Text>
|
||||||
)}
|
)}
|
||||||
@@ -656,7 +656,7 @@ export function MessageSearch({
|
|||||||
<Icon size="200" src={senderOnlyMode ? Icons.User : Icons.Lock} />
|
<Icon size="200" src={senderOnlyMode ? Icons.User : Icons.Lock} />
|
||||||
<Text size="H5">{senderOnlyMode ? 'Messages from user' : 'Encrypted Rooms'}</Text>
|
<Text size="H5">{senderOnlyMode ? 'Messages from user' : 'Encrypted Rooms'}</Text>
|
||||||
{!senderOnlyMode && (
|
{!senderOnlyMode && (
|
||||||
<Text size="T200" style={{ opacity: 0.55 }}>
|
<Text size="T200" priority="300">
|
||||||
{`${localResult.searchedRoomsCount} / ${localResult.encryptedRoomsCount} cached`}
|
{`${localResult.searchedRoomsCount} / ${localResult.encryptedRoomsCount} cached`}
|
||||||
</Text>
|
</Text>
|
||||||
)}
|
)}
|
||||||
|
|||||||
@@ -280,7 +280,8 @@ export function SearchInput({
|
|||||||
<Text
|
<Text
|
||||||
size="T200"
|
size="T200"
|
||||||
truncate
|
truncate
|
||||||
style={{ opacity: 0.6, fontFamily: 'monospace', fontSize: '0.75em' }}
|
priority="300"
|
||||||
|
style={{ fontFamily: 'monospace', fontSize: '0.75em' }}
|
||||||
>
|
>
|
||||||
{user.userId}
|
{user.userId}
|
||||||
</Text>
|
</Text>
|
||||||
|
|||||||
@@ -6,6 +6,8 @@ import {
|
|||||||
Icon,
|
Icon,
|
||||||
IconButton,
|
IconButton,
|
||||||
Icons,
|
Icons,
|
||||||
|
Overlay,
|
||||||
|
OverlayBackdrop,
|
||||||
Scroll,
|
Scroll,
|
||||||
Spinner,
|
Spinner,
|
||||||
Text,
|
Text,
|
||||||
@@ -15,6 +17,7 @@ import {
|
|||||||
config,
|
config,
|
||||||
} from 'folds';
|
} from 'folds';
|
||||||
import { EventType, MatrixClient, MatrixEvent, MsgType, Room } from 'matrix-js-sdk';
|
import { EventType, MatrixClient, MatrixEvent, MsgType, Room } from 'matrix-js-sdk';
|
||||||
|
import FocusTrap from 'focus-trap-react';
|
||||||
import classNames from 'classnames';
|
import classNames from 'classnames';
|
||||||
import { useNearViewport } from '../../hooks/useNearViewport';
|
import { useNearViewport } from '../../hooks/useNearViewport';
|
||||||
import { IEncryptedFile, IImageInfo, IThumbnailContent } from '../../../types/matrix/common';
|
import { IEncryptedFile, IImageInfo, IThumbnailContent } from '../../../types/matrix/common';
|
||||||
@@ -250,7 +253,15 @@ function Lightbox({
|
|||||||
});
|
});
|
||||||
|
|
||||||
return (
|
return (
|
||||||
// eslint-disable-next-line jsx-a11y/no-static-element-interactions
|
<Overlay open backdrop={<OverlayBackdrop />}>
|
||||||
|
<FocusTrap
|
||||||
|
focusTrapOptions={{
|
||||||
|
initialFocus: false,
|
||||||
|
clickOutsideDeactivates: false,
|
||||||
|
escapeDeactivates: false,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{/* eslint-disable-next-line jsx-a11y/no-static-element-interactions */}
|
||||||
<div
|
<div
|
||||||
role="dialog"
|
role="dialog"
|
||||||
aria-modal
|
aria-modal
|
||||||
@@ -346,6 +357,8 @@ function Lightbox({
|
|||||||
)}
|
)}
|
||||||
</Box>
|
</Box>
|
||||||
</div>
|
</div>
|
||||||
|
</FocusTrap>
|
||||||
|
</Overlay>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -241,7 +241,7 @@ export function ScheduleMessageModal({
|
|||||||
<Text size="L400">Send at</Text>
|
<Text size="L400">Send at</Text>
|
||||||
<Box gap="200">
|
<Box gap="200">
|
||||||
<Box direction="Column" gap="100" style={{ flex: 1 }}>
|
<Box direction="Column" gap="100" style={{ flex: 1 }}>
|
||||||
<Text as="label" htmlFor="schedule-date" size="T200" style={{ opacity: 0.7 }}>
|
<Text as="label" htmlFor="schedule-date" size="T200" priority="400">
|
||||||
Date
|
Date
|
||||||
</Text>
|
</Text>
|
||||||
<input
|
<input
|
||||||
@@ -253,7 +253,7 @@ export function ScheduleMessageModal({
|
|||||||
/>
|
/>
|
||||||
</Box>
|
</Box>
|
||||||
<Box direction="Column" gap="100" style={{ flex: 1 }}>
|
<Box direction="Column" gap="100" style={{ flex: 1 }}>
|
||||||
<Text as="label" htmlFor="schedule-time" size="T200" style={{ opacity: 0.7 }}>
|
<Text as="label" htmlFor="schedule-time" size="T200" priority="400">
|
||||||
Time
|
Time
|
||||||
</Text>
|
</Text>
|
||||||
<input
|
<input
|
||||||
|
|||||||
@@ -140,17 +140,17 @@ export function ScheduledMessagesTray({ roomId }: ScheduledMessagesTrayProps) {
|
|||||||
>
|
>
|
||||||
<Text
|
<Text
|
||||||
size="T200"
|
size="T200"
|
||||||
|
priority="400"
|
||||||
style={{
|
style={{
|
||||||
flex: 1,
|
flex: 1,
|
||||||
overflow: 'hidden',
|
overflow: 'hidden',
|
||||||
textOverflow: 'ellipsis',
|
textOverflow: 'ellipsis',
|
||||||
whiteSpace: 'nowrap',
|
whiteSpace: 'nowrap',
|
||||||
opacity: 0.8,
|
|
||||||
}}
|
}}
|
||||||
>
|
>
|
||||||
{typeof msg.content.body === 'string' ? (msg.content.body as string) : '(message)'}
|
{typeof msg.content.body === 'string' ? (msg.content.body as string) : '(message)'}
|
||||||
</Text>
|
</Text>
|
||||||
<Text size="T200" style={{ opacity: 0.6, whiteSpace: 'nowrap', flexShrink: 0 }}>
|
<Text size="T200" priority="300" style={{ whiteSpace: 'nowrap', flexShrink: 0 }}>
|
||||||
{formatSendAt(msg.sendAt)}
|
{formatSendAt(msg.sendAt)}
|
||||||
</Text>
|
</Text>
|
||||||
<IconButton
|
<IconButton
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ import {
|
|||||||
} from 'folds';
|
} from 'folds';
|
||||||
import { Method } from 'matrix-js-sdk';
|
import { Method } from 'matrix-js-sdk';
|
||||||
import FocusTrap from 'focus-trap-react';
|
import FocusTrap from 'focus-trap-react';
|
||||||
|
import { SettingsSelect } from '../../../components/settings-select/SettingsSelect';
|
||||||
import { SequenceCard } from '../../../components/sequence-card';
|
import { SequenceCard } from '../../../components/sequence-card';
|
||||||
import { SequenceCardStyle } from '../styles.css';
|
import { SequenceCardStyle } from '../styles.css';
|
||||||
import { SettingTile } from '../../../components/setting-tile';
|
import { SettingTile } from '../../../components/setting-tile';
|
||||||
@@ -541,38 +542,15 @@ function ProfileStatus() {
|
|||||||
</Text>
|
</Text>
|
||||||
)}
|
)}
|
||||||
<Box alignItems="Center" gap="200">
|
<Box alignItems="Center" gap="200">
|
||||||
<Text size="T200" style={{ opacity: 0.6, whiteSpace: 'nowrap', flexShrink: 0 }}>
|
<Text size="T200" priority="300" style={{ whiteSpace: 'nowrap', flexShrink: 0 }}>
|
||||||
Auto-clear after:
|
Auto-clear after:
|
||||||
</Text>
|
</Text>
|
||||||
<select
|
<SettingsSelect
|
||||||
value={clearAfter}
|
value={clearAfter}
|
||||||
onChange={(e) => setClearAfter(e.target.value)}
|
options={CLEAR_AFTER_OPTIONS}
|
||||||
|
onChange={setClearAfter}
|
||||||
aria-label="Auto-clear status after"
|
aria-label="Auto-clear status after"
|
||||||
style={{
|
/>
|
||||||
background: color.SurfaceVariant.Container,
|
|
||||||
border: `1px solid ${color.SurfaceVariant.ContainerLine}`,
|
|
||||||
borderRadius: config.radii.R300,
|
|
||||||
color: color.SurfaceVariant.OnContainer,
|
|
||||||
colorScheme: 'dark',
|
|
||||||
fontSize: '0.82rem',
|
|
||||||
padding: `${config.space.S100} ${config.space.S200}`,
|
|
||||||
cursor: 'pointer',
|
|
||||||
outline: 'none',
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{CLEAR_AFTER_OPTIONS.map((opt) => (
|
|
||||||
<option
|
|
||||||
key={opt.value}
|
|
||||||
value={opt.value}
|
|
||||||
style={{
|
|
||||||
background: color.SurfaceVariant.Container,
|
|
||||||
color: color.SurfaceVariant.OnContainer,
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{opt.label}
|
|
||||||
</option>
|
|
||||||
))}
|
|
||||||
</select>
|
|
||||||
</Box>
|
</Box>
|
||||||
{(presence?.status || statusMsg) && (
|
{(presence?.status || statusMsg) && (
|
||||||
<Button
|
<Button
|
||||||
@@ -781,10 +759,6 @@ function ProfileTimezone() {
|
|||||||
);
|
);
|
||||||
const saving = saveState.status === AsyncStatus.Loading;
|
const saving = saveState.status === AsyncStatus.Loading;
|
||||||
|
|
||||||
const handleSelectChange = (evt: React.ChangeEvent<HTMLSelectElement>) => {
|
|
||||||
setTimezone(evt.currentTarget.value);
|
|
||||||
};
|
|
||||||
|
|
||||||
const handleReset = () => {
|
const handleReset = () => {
|
||||||
setTimezone(savedTimezone);
|
setTimezone(savedTimezone);
|
||||||
};
|
};
|
||||||
@@ -813,39 +787,16 @@ function ProfileTimezone() {
|
|||||||
<Box direction="Column" grow="Yes" gap="100">
|
<Box direction="Column" grow="Yes" gap="100">
|
||||||
<Box as="form" onSubmit={handleSubmit} gap="200" alignItems="Center" aria-disabled={saving}>
|
<Box as="form" onSubmit={handleSubmit} gap="200" alignItems="Center" aria-disabled={saving}>
|
||||||
<Box grow="Yes" direction="Column">
|
<Box grow="Yes" direction="Column">
|
||||||
<select
|
<SettingsSelect
|
||||||
name="timezoneInput"
|
|
||||||
aria-label="Timezone"
|
|
||||||
value={timezone}
|
value={timezone}
|
||||||
onChange={handleSelectChange}
|
options={[
|
||||||
|
{ value: '', label: '— select timezone —' },
|
||||||
|
...COMMON_TIMEZONES.map((tz) => ({ value: tz, label: tz })),
|
||||||
|
]}
|
||||||
|
onChange={setTimezone}
|
||||||
disabled={saving}
|
disabled={saving}
|
||||||
style={{
|
aria-label="Timezone"
|
||||||
background: color.SurfaceVariant.Container,
|
/>
|
||||||
border: `1px solid ${color.SurfaceVariant.ContainerLine}`,
|
|
||||||
borderRadius: config.radii.R300,
|
|
||||||
color: color.SurfaceVariant.OnContainer,
|
|
||||||
colorScheme: 'dark',
|
|
||||||
fontSize: '0.875rem',
|
|
||||||
padding: `${config.space.S200} ${config.space.S300}`,
|
|
||||||
width: '100%',
|
|
||||||
cursor: 'pointer',
|
|
||||||
outline: 'none',
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<option value="">— select timezone —</option>
|
|
||||||
{COMMON_TIMEZONES.map((tz) => (
|
|
||||||
<option
|
|
||||||
key={tz}
|
|
||||||
value={tz}
|
|
||||||
style={{
|
|
||||||
background: color.SurfaceVariant.Container,
|
|
||||||
color: color.SurfaceVariant.OnContainer,
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{tz}
|
|
||||||
</option>
|
|
||||||
))}
|
|
||||||
</select>
|
|
||||||
</Box>
|
</Box>
|
||||||
{hasChanges && !saving && (
|
{hasChanges && !saving && (
|
||||||
<IconButton
|
<IconButton
|
||||||
|
|||||||
@@ -218,7 +218,7 @@ export function ProfileDecoration() {
|
|||||||
>
|
>
|
||||||
{DECORATION_CATEGORIES.map((category) => (
|
{DECORATION_CATEGORIES.map((category) => (
|
||||||
<div key={category.id} style={{ display: 'flex', flexDirection: 'column', gap: 10 }}>
|
<div key={category.id} style={{ display: 'flex', flexDirection: 'column', gap: 10 }}>
|
||||||
<Text size="L400" style={{ opacity: 0.7 }}>
|
<Text size="L400" priority="400">
|
||||||
{category.label}
|
{category.label}
|
||||||
</Text>
|
</Text>
|
||||||
<div
|
<div
|
||||||
|
|||||||
@@ -81,6 +81,7 @@ import { useDateFormatItems } from '../../../hooks/useDateFormat';
|
|||||||
import { playCallJoinSound } from '../../../utils/callSounds';
|
import { playCallJoinSound } from '../../../utils/callSounds';
|
||||||
import { previewRingtone, RINGTONE_OPTIONS } from '../../../utils/ringtones';
|
import { previewRingtone, RINGTONE_OPTIONS } from '../../../utils/ringtones';
|
||||||
import { DenoiseTester } from './DenoiseTester';
|
import { DenoiseTester } from './DenoiseTester';
|
||||||
|
import { SettingsSelect } from '../../../components/settings-select/SettingsSelect';
|
||||||
|
|
||||||
type ThemeSelectorProps = {
|
type ThemeSelectorProps = {
|
||||||
themeNames: Record<string, string>;
|
themeNames: Record<string, string>;
|
||||||
@@ -169,83 +170,6 @@ function SelectTheme({ disabled }: { disabled?: boolean }) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
type SettingsSelectOption<T extends string> = { value: T; label: string; disabled?: boolean };
|
|
||||||
|
|
||||||
function SettingsSelect<T extends string>({
|
|
||||||
value,
|
|
||||||
options,
|
|
||||||
onChange,
|
|
||||||
}: {
|
|
||||||
value: T;
|
|
||||||
options: SettingsSelectOption<T>[];
|
|
||||||
onChange: (v: T) => void;
|
|
||||||
}) {
|
|
||||||
const [menuCords, setMenuCords] = useState<RectCords>();
|
|
||||||
const selectedLabel = options.find((o) => o.value === value)?.label ?? value;
|
|
||||||
|
|
||||||
const handleMenu: MouseEventHandler<HTMLButtonElement> = (evt) => {
|
|
||||||
setMenuCords(evt.currentTarget.getBoundingClientRect());
|
|
||||||
};
|
|
||||||
|
|
||||||
const handleSelect = (v: T) => {
|
|
||||||
onChange(v);
|
|
||||||
setMenuCords(undefined);
|
|
||||||
};
|
|
||||||
|
|
||||||
return (
|
|
||||||
<>
|
|
||||||
<Button
|
|
||||||
size="300"
|
|
||||||
variant="Secondary"
|
|
||||||
outlined
|
|
||||||
fill="Soft"
|
|
||||||
radii="300"
|
|
||||||
after={<Icon size="300" src={Icons.ChevronBottom} />}
|
|
||||||
onClick={handleMenu}
|
|
||||||
>
|
|
||||||
<Text size="T300">{selectedLabel}</Text>
|
|
||||||
</Button>
|
|
||||||
<PopOut
|
|
||||||
anchor={menuCords}
|
|
||||||
offset={5}
|
|
||||||
position="Bottom"
|
|
||||||
align="End"
|
|
||||||
content={
|
|
||||||
<FocusTrap
|
|
||||||
focusTrapOptions={{
|
|
||||||
initialFocus: false,
|
|
||||||
onDeactivate: () => setMenuCords(undefined),
|
|
||||||
clickOutsideDeactivates: true,
|
|
||||||
isKeyForward: (evt: KeyboardEvent) =>
|
|
||||||
evt.key === 'ArrowDown' || evt.key === 'ArrowRight',
|
|
||||||
isKeyBackward: (evt: KeyboardEvent) =>
|
|
||||||
evt.key === 'ArrowUp' || evt.key === 'ArrowLeft',
|
|
||||||
escapeDeactivates: stopPropagation,
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
<Menu>
|
|
||||||
<Box direction="Column" gap="100" style={{ padding: config.space.S100 }}>
|
|
||||||
{options.map((opt) => (
|
|
||||||
<MenuItem
|
|
||||||
key={opt.value}
|
|
||||||
size="300"
|
|
||||||
variant={opt.value === value ? 'Primary' : 'Surface'}
|
|
||||||
radii="300"
|
|
||||||
disabled={opt.disabled}
|
|
||||||
onClick={() => !opt.disabled && handleSelect(opt.value)}
|
|
||||||
>
|
|
||||||
<Text size="T300">{opt.label}</Text>
|
|
||||||
</MenuItem>
|
|
||||||
))}
|
|
||||||
</Box>
|
|
||||||
</Menu>
|
|
||||||
</FocusTrap>
|
|
||||||
}
|
|
||||||
/>
|
|
||||||
</>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function SystemThemePreferences() {
|
function SystemThemePreferences() {
|
||||||
const themeKind = useSystemThemeKind();
|
const themeKind = useSystemThemeKind();
|
||||||
const themeNames = useThemeNames();
|
const themeNames = useThemeNames();
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import React, { ChangeEventHandler, FormEventHandler, useCallback, useMemo, useState } from 'react';
|
import React, { ChangeEventHandler, FormEventHandler, useCallback, useMemo, useState } from 'react';
|
||||||
import { IPushRule, IPushRules, PushRuleKind } from 'matrix-js-sdk';
|
import { IPushRule, IPushRules, PushRuleKind } from 'matrix-js-sdk';
|
||||||
import { Box, Text, Button, Input, config, IconButton, Icons, Icon, Spinner, Switch } from 'folds';
|
import { Box, Text, Button, Input, config, IconButton, Icons, Icon, Spinner, Switch } from 'folds';
|
||||||
|
import { SettingsSelect } from '../../../components/settings-select/SettingsSelect';
|
||||||
import { useAccountData } from '../../../hooks/useAccountData';
|
import { useAccountData } from '../../../hooks/useAccountData';
|
||||||
import { AccountDataEvent } from '../../../../types/matrix/accountData';
|
import { AccountDataEvent } from '../../../../types/matrix/accountData';
|
||||||
import { SequenceCard } from '../../../components/sequence-card';
|
import { SequenceCard } from '../../../components/sequence-card';
|
||||||
@@ -193,10 +194,6 @@ function AddRuleForm({ kind, placeholder, label }: AddRuleFormProps) {
|
|||||||
setRuleId(evt.currentTarget.value);
|
setRuleId(evt.currentTarget.value);
|
||||||
};
|
};
|
||||||
|
|
||||||
const handleModeChange: ChangeEventHandler<HTMLSelectElement> = (evt) => {
|
|
||||||
setMode(evt.target.value as NotificationMode);
|
|
||||||
};
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<Box as="form" onSubmit={handleSubmit} direction="Column" gap="200">
|
<Box as="form" onSubmit={handleSubmit} direction="Column" gap="200">
|
||||||
<Text size="T200" priority="300">
|
<Text size="T200" priority="300">
|
||||||
@@ -217,24 +214,12 @@ function AddRuleForm({ kind, placeholder, label }: AddRuleFormProps) {
|
|||||||
/>
|
/>
|
||||||
</Box>
|
</Box>
|
||||||
<Box shrink="No">
|
<Box shrink="No">
|
||||||
<select
|
<SettingsSelect
|
||||||
value={mode}
|
value={mode}
|
||||||
onChange={handleModeChange}
|
options={ADD_MODES.map((m) => ({ value: m, label: MODE_LABELS[m] }))}
|
||||||
style={{
|
onChange={setMode}
|
||||||
background: 'transparent',
|
aria-label="Notification mode"
|
||||||
border: '1px solid currentColor',
|
/>
|
||||||
borderRadius: config.radii.R300,
|
|
||||||
padding: `${config.space.S100} ${config.space.S200}`,
|
|
||||||
color: 'inherit',
|
|
||||||
fontSize: 'inherit',
|
|
||||||
}}
|
|
||||||
>
|
|
||||||
{ADD_MODES.map((m) => (
|
|
||||||
<option key={m} value={m}>
|
|
||||||
{MODE_LABELS[m]}
|
|
||||||
</option>
|
|
||||||
))}
|
|
||||||
</select>
|
|
||||||
</Box>
|
</Box>
|
||||||
<Button
|
<Button
|
||||||
size="400"
|
size="400"
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import { color, config, Icon, IconButton, Icons } from 'folds';
|
|||||||
import { toastQueueAtom, dismissToastAtom, ToastNotif } from '../../state/toast';
|
import { toastQueueAtom, dismissToastAtom, ToastNotif } from '../../state/toast';
|
||||||
import { useSetting } from '../../state/hooks/settings';
|
import { useSetting } from '../../state/hooks/settings';
|
||||||
import { settingsAtom } from '../../state/settings';
|
import { settingsAtom } from '../../state/settings';
|
||||||
|
import { zIndices } from '../../styles/zIndex';
|
||||||
|
|
||||||
// Inject the keyframe animation once
|
// Inject the keyframe animation once
|
||||||
const STYLE_ID = 'lotus-toast-keyframes';
|
const STYLE_ID = 'lotus-toast-keyframes';
|
||||||
@@ -214,7 +215,7 @@ export function LotusToastContainer() {
|
|||||||
position: 'fixed',
|
position: 'fixed',
|
||||||
bottom: '1.5rem',
|
bottom: '1.5rem',
|
||||||
right: '1.5rem',
|
right: '1.5rem',
|
||||||
zIndex: 10001,
|
zIndex: zIndices.toast,
|
||||||
display: 'flex',
|
display: 'flex',
|
||||||
flexDirection: 'column',
|
flexDirection: 'column',
|
||||||
gap: config.space.S200,
|
gap: config.space.S200,
|
||||||
|
|||||||
@@ -35,6 +35,19 @@ export const useCallSpeakers = (callEmbed: CallEmbed): Set<string> => {
|
|||||||
callEmbed.iframe.contentDocument ?? callEmbed.iframe.contentWindow?.document ?? undefined;
|
callEmbed.iframe.contentDocument ?? callEmbed.iframe.contentWindow?.document ?? undefined;
|
||||||
|
|
||||||
const syncState = (): void => {
|
const syncState = (): void => {
|
||||||
|
// [lotus #2] Prefer the fork's io.lotus.call_state events over scraping
|
||||||
|
// EC's rendered DOM. Falls back to the DOM path below when the fork hasn't
|
||||||
|
// sent yet (null) OR sent a spurious empty list (you're always present in
|
||||||
|
// your own joined call, so [] means "no usable data", not "nobody").
|
||||||
|
const lotus = callEmbed.getLotusParticipants();
|
||||||
|
if (lotus !== null && lotus.length > 0) {
|
||||||
|
const ls = new Set<string>();
|
||||||
|
lotus.forEach((p) => {
|
||||||
|
if (p.speaking && isUserId(p.userId)) ls.add(p.userId);
|
||||||
|
});
|
||||||
|
setSpeakers(ls);
|
||||||
|
return;
|
||||||
|
}
|
||||||
const doc = getDoc();
|
const doc = getDoc();
|
||||||
if (!doc) {
|
if (!doc) {
|
||||||
setSpeakers(new Set<string>());
|
setSpeakers(new Set<string>());
|
||||||
@@ -91,6 +104,8 @@ export const useCallSpeakers = (callEmbed: CallEmbed): Set<string> => {
|
|||||||
};
|
};
|
||||||
|
|
||||||
attachObserver();
|
attachObserver();
|
||||||
|
// [lotus #2] Re-derive whenever the fork pushes new call-state.
|
||||||
|
const unsubLotus = callEmbed.onLotusCallState(syncState);
|
||||||
|
|
||||||
// If iframe isn't ready yet, wait for body to be available.
|
// If iframe isn't ready yet, wait for body to be available.
|
||||||
let bodyWatcher: MutationObserver | undefined;
|
let bodyWatcher: MutationObserver | undefined;
|
||||||
@@ -109,6 +124,7 @@ export const useCallSpeakers = (callEmbed: CallEmbed): Set<string> => {
|
|||||||
return () => {
|
return () => {
|
||||||
tileObserver?.disconnect();
|
tileObserver?.disconnect();
|
||||||
bodyWatcher?.disconnect();
|
bodyWatcher?.disconnect();
|
||||||
|
unsubLotus();
|
||||||
};
|
};
|
||||||
}, [callEmbed, callMembers, joined]);
|
}, [callEmbed, callMembers, joined]);
|
||||||
|
|
||||||
@@ -137,6 +153,14 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
|
|||||||
const localUserId = callEmbed.room.client?.getUserId() ?? '';
|
const localUserId = callEmbed.room.client?.getUserId() ?? '';
|
||||||
|
|
||||||
const syncState = (): void => {
|
const syncState = (): void => {
|
||||||
|
// [lotus #2] Prefer the fork's io.lotus.call_state over DOM scraping;
|
||||||
|
// ignore a spurious empty list (fall back to DOM).
|
||||||
|
const lotus = callEmbed.getLotusParticipants();
|
||||||
|
if (lotus !== null && lotus.length > 0) {
|
||||||
|
const remote = lotus.filter((p) => p.userId !== localUserId);
|
||||||
|
setMuted(remote.length > 0 && remote.every((p) => !p.audioEnabled));
|
||||||
|
return;
|
||||||
|
}
|
||||||
const doc = getDoc();
|
const doc = getDoc();
|
||||||
if (!doc) {
|
if (!doc) {
|
||||||
setMuted(false);
|
setMuted(false);
|
||||||
@@ -145,13 +169,17 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
|
|||||||
// Each participant's mute icon has data-muted="true"|"false" and
|
// Each participant's mute icon has data-muted="true"|"false" and
|
||||||
// aria-label set to their Matrix user ID.
|
// aria-label set to their Matrix user ID.
|
||||||
const muteIcons = doc.querySelectorAll<HTMLElement>('[data-muted]');
|
const muteIcons = doc.querySelectorAll<HTMLElement>('[data-muted]');
|
||||||
let anyRemoteMuted = false;
|
let remoteCount = 0;
|
||||||
|
let remoteMutedCount = 0;
|
||||||
muteIcons.forEach((el) => {
|
muteIcons.forEach((el) => {
|
||||||
const userId = el.getAttribute('aria-label') ?? '';
|
const userId = el.getAttribute('aria-label') ?? '';
|
||||||
if (userId === localUserId) return;
|
if (userId === localUserId) return;
|
||||||
if (el.getAttribute('data-muted') === 'true') anyRemoteMuted = true;
|
remoteCount += 1;
|
||||||
|
if (el.getAttribute('data-muted') === 'true') remoteMutedCount += 1;
|
||||||
});
|
});
|
||||||
setMuted(anyRemoteMuted);
|
// "All muted" badge: true only when there is at least one remote
|
||||||
|
// participant and every one of them is muted (not merely any single one).
|
||||||
|
setMuted(remoteCount > 0 && remoteMutedCount === remoteCount);
|
||||||
};
|
};
|
||||||
|
|
||||||
let tileObserver: MutationObserver | undefined;
|
let tileObserver: MutationObserver | undefined;
|
||||||
@@ -186,6 +214,8 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
|
|||||||
};
|
};
|
||||||
|
|
||||||
attachObserver();
|
attachObserver();
|
||||||
|
// [lotus #2] Re-derive whenever the fork pushes new call-state.
|
||||||
|
const unsubLotus = callEmbed.onLotusCallState(syncState);
|
||||||
|
|
||||||
// If iframe isn't ready yet, wait for body to be available.
|
// If iframe isn't ready yet, wait for body to be available.
|
||||||
let bodyWatcher: MutationObserver | undefined;
|
let bodyWatcher: MutationObserver | undefined;
|
||||||
@@ -204,6 +234,7 @@ export const useRemoteAllMuted = (callEmbed: CallEmbed | undefined): boolean =>
|
|||||||
return () => {
|
return () => {
|
||||||
tileObserver?.disconnect();
|
tileObserver?.disconnect();
|
||||||
bodyWatcher?.disconnect();
|
bodyWatcher?.disconnect();
|
||||||
|
unsubLotus();
|
||||||
};
|
};
|
||||||
}, [callEmbed]);
|
}, [callEmbed]);
|
||||||
|
|
||||||
|
|||||||
@@ -27,6 +27,7 @@ import { LotusToastContainer } from '../features/toast/LotusToastContainer';
|
|||||||
import { useTauriNotificationBadge } from '../hooks/useTauriNotificationBadge';
|
import { useTauriNotificationBadge } from '../hooks/useTauriNotificationBadge';
|
||||||
import { SeasonalEffect } from '../components/seasonal/SeasonalEffect';
|
import { SeasonalEffect } from '../components/seasonal/SeasonalEffect';
|
||||||
import { applyCustomAccent, removeCustomAccent } from '../utils/accentColor';
|
import { applyCustomAccent, removeCustomAccent } from '../utils/accentColor';
|
||||||
|
import { zIndices } from '../styles/zIndex';
|
||||||
|
|
||||||
const FONT_MAP: Record<string, string> = {
|
const FONT_MAP: Record<string, string> = {
|
||||||
system: "system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif",
|
system: "system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', sans-serif",
|
||||||
@@ -95,7 +96,7 @@ function NightLightOverlay() {
|
|||||||
position: 'fixed',
|
position: 'fixed',
|
||||||
inset: 0,
|
inset: 0,
|
||||||
pointerEvents: 'none',
|
pointerEvents: 'none',
|
||||||
zIndex: 9998,
|
zIndex: zIndices.nightLight,
|
||||||
backgroundColor: `rgba(255, 140, 0, ${(settings.nightLightOpacity ?? 30) / 100})`,
|
backgroundColor: `rgba(255, 140, 0, ${(settings.nightLightOpacity ?? 30) / 100})`,
|
||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -346,71 +346,16 @@ export class CallControl extends EventEmitter implements CallControlState {
|
|||||||
* them yet).
|
* them yet).
|
||||||
*/
|
*/
|
||||||
public focusCameraParticipant(userId: string): void {
|
public focusCameraParticipant(userId: string): void {
|
||||||
const doc = this.document;
|
// [lotus #4] Pin the participant via the fork's widget action instead of
|
||||||
if (!doc) return;
|
// DOM-poking tiles. EC's layout honors it — including surfacing the camera
|
||||||
|
// alongside a screenshare (A5) — and it's version-stable. The fork always
|
||||||
// EC labels participant tiles inconsistently across versions — the user's
|
// acks, so the promise resolves regardless.
|
||||||
// matrix id may be the full aria-label, a substring of it, or carried on a
|
this.call.transport.send('io.lotus.focus_participant', { userId }).catch(() => undefined);
|
||||||
// data attribute (and sometimes the visible label is the display name, not
|
|
||||||
// the id at all). Try several strategies before giving up, then walk up to
|
|
||||||
// the enclosing video tile.
|
|
||||||
const findTile = (): HTMLElement | undefined => {
|
|
||||||
const escaped = CSS.escape(userId);
|
|
||||||
const el =
|
|
||||||
doc.querySelector<HTMLElement>(`[aria-label="${escaped}"]`) ??
|
|
||||||
doc.querySelector<HTMLElement>(`[data-testid="videoTile"][aria-label*="${escaped}"]`) ??
|
|
||||||
doc.querySelector<HTMLElement>(`[aria-label*="${escaped}"]`) ??
|
|
||||||
doc.querySelector<HTMLElement>(`[data-member-id="${escaped}"]`) ??
|
|
||||||
doc.querySelector<HTMLElement>(`[data-id="${escaped}"]`) ??
|
|
||||||
undefined;
|
|
||||||
return (
|
|
||||||
el?.closest<HTMLElement>('[data-testid="videoTile"]') ??
|
|
||||||
el?.closest<HTMLElement>('[data-video-fit]') ??
|
|
||||||
el ??
|
|
||||||
undefined
|
|
||||||
);
|
|
||||||
};
|
|
||||||
|
|
||||||
const applyFocus = () => {
|
|
||||||
const tile = findTile();
|
|
||||||
if (tile) {
|
|
||||||
tile.click();
|
|
||||||
} else if (import.meta.env.DEV) {
|
|
||||||
console.warn(`[CallControl] focusCameraParticipant: no tile matched ${userId}`);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
if (this.spotlight) {
|
|
||||||
// Already in spotlight — pin immediately.
|
|
||||||
applyFocus();
|
|
||||||
return;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Switching to spotlight re-renders EC's layout asynchronously; clicking the
|
/** [lotus #4] Clear any manual spotlight pin and return to speaker-follows. */
|
||||||
// tile in the same tick would land in the old (grid) DOM. A fixed frame
|
public clearFocusParticipant(): void {
|
||||||
// delay is unreliable (EC's React commit can exceed it on slow devices), so
|
this.call.transport.send('io.lotus.focus_participant', { userId: null }).catch(() => undefined);
|
||||||
// watch the iframe DOM for a spotlight video tile to mount, then focus —
|
|
||||||
// with a hard timeout so the click is always attempted at least once.
|
|
||||||
this.spotlightButton?.click();
|
|
||||||
|
|
||||||
const tileSelector = '[data-testid="videoTile"]';
|
|
||||||
let settled = false;
|
|
||||||
let observer: MutationObserver | undefined;
|
|
||||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
|
||||||
const finish = () => {
|
|
||||||
if (settled) return;
|
|
||||||
settled = true;
|
|
||||||
if (timer) clearTimeout(timer);
|
|
||||||
observer?.disconnect();
|
|
||||||
applyFocus();
|
|
||||||
};
|
|
||||||
observer = new MutationObserver(() => {
|
|
||||||
if (doc.querySelector(tileSelector)) finish();
|
|
||||||
});
|
|
||||||
observer.observe(doc.body, { childList: true, subtree: true });
|
|
||||||
timer = setTimeout(finish, 600);
|
|
||||||
// A tile may already be present immediately after toggling spotlight.
|
|
||||||
if (doc.querySelector(tileSelector)) finish();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public dispose() {
|
public dispose() {
|
||||||
|
|||||||
@@ -36,6 +36,15 @@ const CALL_LOAD_WATCHDOG_MS = 25_000;
|
|||||||
|
|
||||||
export type CallLoadErrorReason = 'timeout' | 'iframe';
|
export type CallLoadErrorReason = 'timeout' | 'iframe';
|
||||||
|
|
||||||
|
/** Payload entry of the fork's io.lotus.call_state widget event (#2). */
|
||||||
|
export interface LotusCallParticipant {
|
||||||
|
id: string;
|
||||||
|
userId: string;
|
||||||
|
speaking: boolean;
|
||||||
|
audioEnabled: boolean;
|
||||||
|
videoEnabled: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
export class CallEmbed {
|
export class CallEmbed {
|
||||||
private mx: MatrixClient;
|
private mx: MatrixClient;
|
||||||
|
|
||||||
@@ -47,6 +56,13 @@ export class CallEmbed {
|
|||||||
|
|
||||||
public joined = false;
|
public joined = false;
|
||||||
|
|
||||||
|
// [lotus #2] Latest per-participant state from io.lotus.call_state, or null
|
||||||
|
// until the fork sends the first one. When non-null, the speaker/mute hooks
|
||||||
|
// read it instead of scraping the EC iframe DOM.
|
||||||
|
private lotusParticipants: LotusCallParticipant[] | null = null;
|
||||||
|
|
||||||
|
private lotusCallStateListeners = new Set<() => void>();
|
||||||
|
|
||||||
public readonly control: CallControl;
|
public readonly control: CallControl;
|
||||||
|
|
||||||
private readonly container: HTMLElement;
|
private readonly container: HTMLElement;
|
||||||
@@ -122,7 +138,9 @@ export class CallEmbed {
|
|||||||
themeKind: ElementCallThemeKind,
|
themeKind: ElementCallThemeKind,
|
||||||
denoiseMode: NoiseSuppressionMode = 'browser',
|
denoiseMode: NoiseSuppressionMode = 'browser',
|
||||||
denoiseModel: string = 'rnnoise',
|
denoiseModel: string = 'rnnoise',
|
||||||
denoiseNativeNS: boolean = true,
|
// [lotus] no longer used by the in-source denoise path; kept positionally
|
||||||
|
// for callers. Prefixed with _ to satisfy no-unused-vars.
|
||||||
|
_denoiseNativeNS: boolean = true,
|
||||||
denoiseGate: boolean = false,
|
denoiseGate: boolean = false,
|
||||||
denoiseGateThreshold: number = -45,
|
denoiseGateThreshold: number = -45,
|
||||||
initialAudio = true,
|
initialAudio = true,
|
||||||
@@ -148,20 +166,30 @@ export class CallEmbed {
|
|||||||
perParticipantE2EE: room.hasEncryptionStateEvent().toString(),
|
perParticipantE2EE: room.hasEncryptionStateEvent().toString(),
|
||||||
lang: 'en-EN',
|
lang: 'en-EN',
|
||||||
theme: themeKind,
|
theme: themeKind,
|
||||||
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml' we
|
// EC's built-in WebRTC suppressor: on only for 'browser' tier. For 'ml'
|
||||||
// disable it here so EC doesn't do its own extra processing, and let the
|
// we disable it so EC captures a raw mic and the fork's in-source denoise
|
||||||
// Lotus denoise shim (which keeps native NS on) handle the pipeline.
|
// TrackProcessor (lotusDenoiseSource) handles the pipeline.
|
||||||
noiseSuppression: (denoiseMode === 'browser').toString(),
|
noiseSuppression: (denoiseMode === 'browser').toString(),
|
||||||
audio: initialAudio.toString(),
|
audio: initialAudio.toString(),
|
||||||
video: initialVideo.toString(),
|
video: initialVideo.toString(),
|
||||||
header: 'none',
|
header: 'none',
|
||||||
|
// [lotus] Activate the self-built fork's in-source features (each is a
|
||||||
|
// no-op on the EC side unless its flag/action is present):
|
||||||
|
// - call-state stream (speaking/mute events) -> useCallSpeakers
|
||||||
|
// - transparent background so the room wallpaper shows through natively
|
||||||
|
lotusCallState: 'true',
|
||||||
|
lotusTransparent: 'true',
|
||||||
});
|
});
|
||||||
|
|
||||||
if (denoiseMode === 'ml') {
|
if (denoiseMode === 'ml') {
|
||||||
// Signal the Lotus denoise shim to route the mic through the ML processors.
|
// [lotus] In-source ML denoise: the fork runs RNNoise/Speex/DTLN/DFN as a
|
||||||
params.append('lotusDenoise', 'ml');
|
// real LiveKit audio TrackProcessor (survives reconnects — fixes A7),
|
||||||
|
// replacing the old build-time getUserMedia shim. The shim injection was
|
||||||
|
// removed from vite.config.js; the denoise/ assets are still shipped and
|
||||||
|
// loaded by the processor. lotusDenoiseSource (not lotusDenoise=ml) gates
|
||||||
|
// it so the two engines can never both run.
|
||||||
|
params.append('lotusDenoiseSource', 'true');
|
||||||
params.append('lotusModel', denoiseModel);
|
params.append('lotusModel', denoiseModel);
|
||||||
params.append('lotusNativeNS', denoiseNativeNS.toString());
|
|
||||||
params.append('lotusGate', denoiseGate.toString());
|
params.append('lotusGate', denoiseGate.toString());
|
||||||
params.append('lotusGateThreshold', denoiseGateThreshold.toString());
|
params.append('lotusGateThreshold', denoiseGateThreshold.toString());
|
||||||
}
|
}
|
||||||
@@ -318,6 +346,18 @@ export class CallEmbed {
|
|||||||
this.disposables.push(
|
this.disposables.push(
|
||||||
this.listenAction(WidgetApiFromWidgetAction.UpdateAlwaysOnScreen, () => {}),
|
this.listenAction(WidgetApiFromWidgetAction.UpdateAlwaysOnScreen, () => {}),
|
||||||
);
|
);
|
||||||
|
// [lotus #2] Consume the fork's per-participant call-state stream. listenAction
|
||||||
|
// auto-replies {} so the fork's transport doesn't time out. Stored for the
|
||||||
|
// speaker/mute hooks (which prefer this over DOM scraping).
|
||||||
|
this.disposables.push(
|
||||||
|
this.listenAction('io.lotus.call_state', (evt) => {
|
||||||
|
const data = (evt.detail as { data?: { participants?: unknown } } | undefined)?.data;
|
||||||
|
this.lotusParticipants = Array.isArray(data?.participants)
|
||||||
|
? (data!.participants as LotusCallParticipant[])
|
||||||
|
: [];
|
||||||
|
this.lotusCallStateListeners.forEach((l) => l());
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
|
||||||
// Populate the map of "read up to" events for this widget with the current event in every room.
|
// Populate the map of "read up to" events for this widget with the current event in every room.
|
||||||
// This is a bit inefficient, but should be okay. We do this for all rooms in case the widget
|
// This is a bit inefficient, but should be okay. We do this for all rooms in case the widget
|
||||||
@@ -623,6 +663,20 @@ export class CallEmbed {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** [lotus #2] Latest io.lotus.call_state participants, or null if the fork
|
||||||
|
* hasn't sent any yet (callers then fall back to DOM scraping). */
|
||||||
|
public getLotusParticipants(): LotusCallParticipant[] | null {
|
||||||
|
return this.lotusParticipants;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** [lotus #2] Subscribe to io.lotus.call_state updates. Returns an unsubscribe. */
|
||||||
|
public onLotusCallState(cb: () => void): () => void {
|
||||||
|
this.lotusCallStateListeners.add(cb);
|
||||||
|
return () => {
|
||||||
|
this.lotusCallStateListeners.delete(cb);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
public listenAction<T>(type: string, callback: (event: CustomEvent<T>) => void) {
|
public listenAction<T>(type: string, callback: (event: CustomEvent<T>) => void) {
|
||||||
const wrapped = (ev: CustomEvent<T>) => {
|
const wrapped = (ev: CustomEvent<T>) => {
|
||||||
ev.preventDefault();
|
ev.preventDefault();
|
||||||
|
|||||||
@@ -0,0 +1,16 @@
|
|||||||
|
/**
|
||||||
|
* Global overlay stacking layers, centralized so floating Lotus UI doesn't
|
||||||
|
* collide. (folds `Overlay`/`Dialog` modals resolve to 9999, which sits between
|
||||||
|
* `nightLight` and `toast`.) Component-internal stacking uses small local
|
||||||
|
* z-index values and is intentionally not listed here.
|
||||||
|
*/
|
||||||
|
export const zIndices = {
|
||||||
|
/** In-call incoming-call banner — below seasonal/night-light/modals. */
|
||||||
|
inCallBanner: 9990,
|
||||||
|
/** Seasonal particle effect — below the night-light tint so particles tint. */
|
||||||
|
seasonalEffect: 9997,
|
||||||
|
/** Night Light tint overlay — above effects, below modals. */
|
||||||
|
nightLight: 9998,
|
||||||
|
/** Toasts — above everything, including modals. */
|
||||||
|
toast: 10001,
|
||||||
|
} as const;
|
||||||
@@ -102,12 +102,17 @@ const playPhrase = (style: SynthStyle, volume: number, destination: AudioNode):
|
|||||||
});
|
});
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// The bundled call.ogg is mastered near full scale, so at equal `volume` it is
|
||||||
|
// perceptibly much louder than the synthesized styles (which peak at ~0.12–0.3).
|
||||||
|
// Attenuate it so all ringtones sit at a comparable loudness.
|
||||||
|
const CLASSIC_GAIN = 0.45;
|
||||||
|
|
||||||
const startClassic = (volume: number, loop: boolean): (() => void) => {
|
const startClassic = (volume: number, loop: boolean): (() => void) => {
|
||||||
let audio: HTMLAudioElement | undefined;
|
let audio: HTMLAudioElement | undefined;
|
||||||
try {
|
try {
|
||||||
audio = new Audio(CallSound);
|
audio = new Audio(CallSound);
|
||||||
audio.loop = loop;
|
audio.loop = loop;
|
||||||
audio.volume = clamp01(volume);
|
audio.volume = clamp01(volume) * CLASSIC_GAIN;
|
||||||
audio.play().catch(() => undefined);
|
audio.play().catch(() => undefined);
|
||||||
} catch {
|
} catch {
|
||||||
audio = undefined;
|
audio = undefined;
|
||||||
|
|||||||
+12
-30
@@ -16,13 +16,15 @@ const copyFiles = {
|
|||||||
// widget URL (/public/element-call/index.html) resolves. v4.x of
|
// widget URL (/public/element-call/index.html) resolves. v4.x of
|
||||||
// vite-plugin-static-copy preserves the full source path under dest, so
|
// vite-plugin-static-copy preserves the full source path under dest, so
|
||||||
// we strip the 4 leading segments of the source base
|
// we strip the 4 leading segments of the source base
|
||||||
// (node_modules/@element-hq/element-call-embedded/dist) — mirroring the
|
// (node_modules/@lotusguild/element-call-embedded/dist) — mirroring the
|
||||||
// stripBase pattern used by the android/locales targets below. The old
|
// stripBase pattern used by the android/locales targets below. The old
|
||||||
// `rename: 'element-call'` form silently produced
|
// `rename: 'element-call'` form silently produced
|
||||||
// public/node_modules/.../dist/ under v4.x, 404ing the widget (calls
|
// public/node_modules/.../dist/ under v4.x, 404ing the widget (calls
|
||||||
// broke on cinny-desktop; web only worked because its deployed copy was
|
// broke on cinny-desktop; web only worked because its deployed copy was
|
||||||
// a stale artifact from before the vite-plugin-static-copy v4 bump).
|
// a stale artifact from before the vite-plugin-static-copy v4 bump).
|
||||||
src: 'node_modules/@element-hq/element-call-embedded/dist',
|
// Source is our self-built fork (LotusGuild/element-call) published to
|
||||||
|
// the Gitea npm registry; see HANDOFF_ELEMENT_CALL_FORK.md.
|
||||||
|
src: 'node_modules/@lotusguild/element-call-embedded/dist',
|
||||||
dest: 'public/element-call',
|
dest: 'public/element-call',
|
||||||
rename: { stripBase: 4 },
|
rename: { stripBase: 4 },
|
||||||
},
|
},
|
||||||
@@ -159,34 +161,14 @@ function lotusDenoise() {
|
|||||||
fs.copyFileSync(s, d);
|
fs.copyFileSync(s, d);
|
||||||
});
|
});
|
||||||
|
|
||||||
const shimSrc = path.resolve('build/lotus-denoise.js');
|
// [lotus] DENOISE CUTOVER: the getUserMedia shim is no longer injected.
|
||||||
if (!fs.existsSync(shimSrc)) {
|
// Our forked Element Call now runs ML denoise in-source as a real LiveKit
|
||||||
throw new Error(`[lotus-denoise] Missing shim source ${shimSrc} — build aborted.`);
|
// audio TrackProcessor (activated by lotusDenoiseSource=1 in CallEmbed),
|
||||||
}
|
// which survives reconnects (fixes A7). We still copy the denoise/ assets
|
||||||
fs.copyFileSync(shimSrc, path.join(ecDir, 'lotus-denoise.js'));
|
// above because the in-source processor loads its worklets/wasm from
|
||||||
|
// ./denoise/ at runtime. To roll back to the shim: restore the
|
||||||
// Inject the shim <script> into Element Call's index.html so it runs
|
// copy+inject of build/lotus-denoise.js here and swap lotusDenoiseSource
|
||||||
// before EC captures the mic. Verify the injection actually landed —
|
// back to lotusDenoise=ml in CallEmbed.getWidget.
|
||||||
// if EC's bundle ever drops its deferred module entry the replace would
|
|
||||||
// no-op and ML would silently never engage, so fail loudly.
|
|
||||||
const indexPath = path.join(ecDir, 'index.html');
|
|
||||||
if (fs.existsSync(indexPath)) {
|
|
||||||
let html = fs.readFileSync(indexPath, 'utf8');
|
|
||||||
if (!html.includes('lotus-denoise.js')) {
|
|
||||||
// Classic (non-deferred) script runs before EC's deferred module entry.
|
|
||||||
html = html.replace(
|
|
||||||
/<script type="module"/,
|
|
||||||
'<script src="./lotus-denoise.js"></script><script type="module"',
|
|
||||||
);
|
|
||||||
if (!html.includes('lotus-denoise.js')) {
|
|
||||||
throw new Error(
|
|
||||||
'[lotus-denoise] Failed to inject shim into Element Call index.html ' +
|
|
||||||
'(no `<script type="module">` entry found) — build aborted.',
|
|
||||||
);
|
|
||||||
}
|
|
||||||
fs.writeFileSync(indexPath, html);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user