diff --git a/HANDOFF_ELEMENT_CALL_FORK.md b/HANDOFF_ELEMENT_CALL_FORK.md index aa998c712..bfee889f6 100644 --- a/HANDOFF_ELEMENT_CALL_FORK.md +++ b/HANDOFF_ELEMENT_CALL_FORK.md @@ -548,8 +548,19 @@ roster capped. Only `https`/`blob` URLs accepted for inject/decoration assets. ### 12.1 cinny host integration checklist (REQUIRED to light these up) -The EC side is additive and dormant until cinny opts in. Host work needed (in -`src/app/plugins/call/CallEmbed.ts` unless noted): +> ✅ **STATUS (2026-06): COMPLETE.** All items below are shipped. call_state, +> focus_participant, decorations, and transparent background are active; the +> in-source denoise cutover is done (flag `lotusDenoiseSource=1`, **all four** +> models in-source); and the two formerly-dormant capabilities now have cinny +> UI — **soundboard** (`io.lotus.inject_audio`, P5-15) and **quality controls + +> room permissions** (`io.lotus.set_quality` + `io.lotus.room_quality`, P5-31, +> with server-side enforcement in `LotusGuild/matrix`). See `LOTUS_FEATURES.md` +> → "Element Call — Self-Built Fork". The checklist is kept below as the record +> of what was wired. (One open denoise item tracked separately: the "Series +> Suppression" native-NS toggle is not wired to the real call path.) + +The EC side is additive and dormant until cinny opts in. Host work (in +`src/app/plugins/call/CallEmbed.ts` unless noted) — **done**: > ⚠️ **CRITICAL TIMING (protocol audit F1):** only send `io.lotus.*` **toWidget** > actions (#3 focus, #6 decorations, #7 quality, audio-inject) **after** the call @@ -559,16 +570,16 @@ The EC side is additive and dormant until cinny opts in. Host work needed (in > leaves the host's `transport.send` pending until the **10s timeout**. Queue and > flush on join, or no-op before join. > -> Also: **F3** — the fork implements only `rnnoise`/`speex`; cinny's `dtln`/ -> `deepfilternet` selections silently fall back to rnnoise (now logged). Restrict -> the embedded-call model picker to rnnoise/speex, or implement the others in -> `lotusDenoiseProcessor.ts`. **F4** — cinny sends `lotusNativeNS`, which the -> fork ignores; drop it or wire it in. **F7** — no widget _capability_ changes -> needed; custom actions bypass capability checks. +> Also: **F3 (RESOLVED)** — all four models (`rnnoise`/`speex`/`dtln`/ +> `deepfilternet`) are now implemented in-source in `lotusDenoiseProcessor.ts`; +> the picker offers all four. **F4** — cinny no longer forwards a native-NS flag +> in the `ml` branch (the "Series Suppression" toggle is currently a no-op in +> real calls — open item). **F7** — no widget _capability_ changes needed; +> custom actions bypass capability checks. 1. **Set the URL flags** on the widget iframe params (the `URLSearchParams` in `CallEmbed`): `lotusCallState=1`, `lotusTransparent=1`/`lotusTheme=1`, - `lotusAudioInject=1` as desired. (Denoise already sets `lotusDenoise=ml` etc.) + `lotusAudioInject=1` as desired. (Denoise sets `lotusDenoiseSource=1` + `lotusModel`/`lotusGate`/`lotusGateThreshold` in the `ml` tier.) 2. **Ack `io.lotus.call_state`**: add `listenAction('io.lotus.call_state', …)` — without a reply the fork's sends time out every 250ms. Feed the payload into `useCallSpeakers` and RETIRE its `contentDocument` DOM scrape. diff --git a/LOTUS_TODO.md b/LOTUS_TODO.md index d5b8cf71d..a7f45c848 100644 --- a/LOTUS_TODO.md +++ b/LOTUS_TODO.md @@ -560,7 +560,7 @@ Exhaustive, low-level implementation details for backlog items. Follow these pat > ⚠️ **[Gemini_Found — CORRECTED]** Gemini originally suggested using LiveKit's `LocalAudioTrack.replaceTrack()` to mix audio into the call stream. This is **not possible** from Lotus Chat's realm: Element Call runs in a **cross-origin iframe** controlled via `matrix-widget-api` (postMessage). LiveKit's JS SDK and its `LocalAudioTrack` live inside EC's sandboxed context — inaccessible from our code. This directly contradicts the confirmed constraint already listed in the Server Capabilities table: _"Cindy CANNOT inject audio into EC call stream — In-call soundboard must be redesigned as local-only."_ The soundboard must be a local-playback-only feature (output through the user's speakers, not mixed into the call audio stream). > -> 🔱 **[EC-FORK — RESOLVED]** Both the original claim and the earlier "practical blocker still holds" correction are now **outdated**. EC is same-origin **and** we own the source, so we no longer reach into EC's module scope from cinny — instead the fork **exposes the inject point itself**: the `io.lotus.inject_audio` widget action (`LotusWidgetActions.InjectAudio`) publishes a clip as a separate LiveKit track from inside EC. A **real** in-call soundboard (mixed into the call, not local-only) is therefore unblocked; only the cinny-side UI remains (see P5-15 above). The capability ships dormant today. +> 🔱 **[EC-FORK — RESOLVED]** Both the original claim and the earlier "practical blocker still holds" correction are now **outdated**. EC is same-origin **and** we own the source, so we no longer reach into EC's module scope from cinny — instead the fork **exposes the inject point itself**: the `io.lotus.inject_audio` widget action (`LotusWidgetActions.InjectAudio`) publishes a clip as a separate LiveKit track from inside EC. A **real** in-call soundboard (mixed into the call, not local-only) is therefore unblocked, and the cinny-side soundboard UI is now **built** (P5-15 above): uploadable clips played into the call via this action, stored in `io.lotus.soundboard` account data. --- diff --git a/README.md b/README.md index d4faea1ff..339479d49 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,9 @@ The Lotus Chat logo (`public/res/Lotus.png`) is a derivative work based on the o - AFK auto-mute: mic is automatically silenced after a configurable idle timeout (1–30 min); a toast confirms the action - Voice channel user limit: admins can cap how many people can be in a room's call — enforced server-side for every Matrix client (not just Lotus Chat); others see "Channel Full" until a spot opens - Custom join/leave sound effects when someone enters or leaves your call — choose Chime, Soft, Retro, or off +- Soundboard: upload your own short audio clips (like custom emojis — they sync across your devices) and play them into a call so everyone hears them +- Call quality settings: cap your microphone bitrate, screenshare bitrate, and screenshare framerate — handy on a slow connection (Settings → Calls) +- Room call permissions: admins can turn off screen sharing or make a room audio-only (no cameras) — enforced server-side for every Matrix client, and it stops an in-progress share within seconds of being switched off ### Customization & Appearance