Compare commits

...
Author SHA1 Message Date
Lotus CIandClaude Opus 5.5 9e4ddb3eb6 chore(lotus): 0.25.0-lotus.21
CI / Build embedded bundle (push) Successful in 4m16s
CI / Publish to Gitea npm registry (push) Successful in 1m10s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-27 01:30:40 -04:00
jared 9d57d104b4 Merge pull request #41: work when served from its own origin (cinny #43)
CI / Build embedded bundle (push) Canceled after 6s
CI / Publish to Gitea npm registry (push) Canceled after 0s
2026-09-27 01:30:31 -04:00
Lotus CIandClaude Opus 5.5 a93bd9d7d8 feat(lotus): work when served from its own origin (cinny #43)
CI / Build embedded bundle (pull_request) Successful in 3m43s
CI / Publish to Gitea npm registry (pull_request) Skipped
Two things tied EC to the host's origin:

- Widget message check: matrix-widget-api's strictOriginCheck compares
  ev.origin with THIS frame's origin, so on call.chat.lotusguild.org every
  message from chat.lotusguild.org would be dropped and calls would not
  start. restrictToHost() instead requires ev.source === window.parent and
  ev.origin === parentUrl's origin. Same-origin deployments keep working
  (the host origin is our own origin there), and it is stricter than
  before: the sender must also be our parent window.
- Soundboard: the host's blob: clip URL is origin-bound. io.lotus.inject_audio
  now accepts the clip's bytes (`audio`, ArrayBuffer, <= 8 MiB) and prefers
  them over `url`; hosts that only send `url` are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 23:24:32 -04:00
Lotus CIandClaude Opus 5.5 43a5e93c37 chore(lotus): 0.25.0-lotus.20
CI / Build embedded bundle (push) Successful in 4m17s
CI / Publish to Gitea npm registry (push) Successful in 52s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 21:48:22 -04:00
Lotus CIandClaude Opus 5.5 0f93368a7e feat(lotus): in-frame screenshare button where the host can't delegate (cinny #43)
Firefox, Safari and WebKitGTK have no Capability Delegation, so the host
can't start a share from its call bar: getDisplayMedia needs the click in
this frame. Until now the host clicked EC's hidden footer button through
the DOM, which only works while the frame is same-origin.

With lotusHostControls + lotusFrameScreenshare the fork now shows EC's own
screenshare button (floating bottom-right, footer still hidden), so the
click is native. controls_state reports frameScreenshare so the host hides
its own button. New toWidget io.lotus.set_frame_screenshare { visible }
lets the host hide it while the room's call policy forbids screensharing
(never while a share is live, so it can still be stopped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 21:48:22 -04:00
Lotus CIandClaude Opus 5.5 dfb72bd9a5 chore(lotus): 0.25.0-lotus.19
CI / Build embedded bundle (push) Successful in 3m27s
CI / Publish to Gitea npm registry (push) Successful in 1m16s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 13:59:34 -04:00
Lotus CIandClaude Opus 5.5 e0eecc5271 feat(lotus): report host call hotkeys from inside the frame (cinny #43)
The host used to add PTT/deafen key listeners to this frame's window,
which needs same-origin access. Now it sends `io.lotus.set_hotkeys
{ codes }` and the fork reports those keys as `io.lotus.hotkey`
(keydown/keyup with modifiers + whether the target is a text field or a
button), plus window focus changes so a held PTT is released. The
default action is cancelled here with the host's rules (not while typing,
not on a focused button). `controls_state` carries `hotkeys: true`.

Fixes a live bug: EC's own shortcuts (M = toggle mic, Space = PTT) also
fired for the host's keys, so with focus in the call and the default
deafen key M, the first press turned the MIC ON instead of deafening
(even in push-to-talk mode). EC's shortcuts now ignore keys the host owns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 13:59:34 -04:00
Lotus CIandClaude Opus 5.5 c4f6193888 chore(lotus): 0.25.0-lotus.18
CI / Build embedded bundle (push) Successful in 3m9s
CI / Publish to Gitea npm registry (push) Successful in 1m17s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 13:39:18 -04:00
Lotus CIandClaude Opus 5.5 7855a0c22f feat(lotus): hide the footer + root color-scheme natively (cinny #43)
- `lotusHostControls=1`: body.lotus-host-controls hides EC's footer
  (position absolute + visibility hidden, still in the DOM so a host on an
  engine without Capability Delegation can click its screenshare button).
- With lotusTheme/lotusTransparent, `:root` color-scheme follows the
  theme, so the transparent frame gets no opaque backdrop.
Together these replace the CSS the host injected into this frame.

Also fixes unreadable tile name tags in the light theme: the Lotus
near-black canvas override now applies to dark themes only (it sat behind
light-theme dark text).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 13:39:18 -04:00
Lotus CIandClaude Opus 5.5 aeb72c22b9 chore(lotus): 0.25.0-lotus.17
CI / Build embedded bundle (push) Successful in 2m45s
CI / Publish to Gitea npm registry (push) Successful in 1m17s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 12:49:58 -04:00
Lotus CIandClaude Opus 5.5 6fa34911ef feat(lotus): io.lotus.set_screenshare for the host call bar (cinny #43)
toWidget `io.lotus.set_screenshare { on?: boolean }` (omit to toggle)
starts/stops sharing via the view model instead of the host clicking EC's
hidden button. The host sends it with Capability Delegation
(`delegate: "display-capture"`) where supported, so getDisplayMedia keeps
the user's click on engines that require it. `controls_state` now carries
`screenshareAction: true` so the host only uses it on a fork that has it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-26 12:49:58 -04:00
Lotus CIandClaude Opus 5.5 35bc7a1e98 chore(lotus): 0.25.0-lotus.16
CI / Build embedded bundle (push) Successful in 2m4s
CI / Publish to Gitea npm registry (push) Successful in 56s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-25 12:08:57 -04:00
Lotus CIandClaude Opus 5.5 cee6bab9a6 feat(lotus): report the local mic level to the host (cinny #146)
The host had only a boolean `speaking` for yourself, throttled to 500 ms, so
nothing on the call bar showed that the mic was actually picking you up.

- lotusMicLevel.ts: one shared local mic sampler (a clone of the published
  track, ~10 Hz RMS, while a mic is published, muted or not), shared per
  call view model so there is one AudioContext.
- MicLevelQuantizer: 0–3 bars (≈ −46/−36/−26 dBFS), rising at once and
  falling one bar per sample so it doesn't flicker between words.
- fromWidget io.lotus.mic_level { bars } only when the value changes; 0 while
  muted or with no mic. Opt-in with the host state stream (lotusCallState=1).
- "Talking while muted" (#37) now reads the same sampler instead of running
  its own; same gate and behaviour.

Verified in a local call with a fake-tone mic: the host receives 0–3 as the
tone rises/falls (about 4 messages/s while it changes), 0 on mute and levels
again on unmute; muted + tone still raises speakingWhileMuted and the host's
"You're muted" notice. Lotus/room suites pass on Node 22 (148 tests).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-25 12:08:57 -04:00
Lotus CIandClaude Opus 5.5 44023a4c84 style(lotus): oxfmt the volume memory change (#36)
CI / Build embedded bundle (push) Successful in 1m51s
CI / Publish to Gitea npm registry (push) Successful in 1m5s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-25 10:03:49 -04:00
Lotus CIandClaude Opus 5.5 3086edc64a chore(lotus): 0.25.0-lotus.15
CI / Build embedded bundle (push) Failing after 48s
CI / Publish to Gitea npm registry (push) Skipped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-25 09:57:55 -04:00
Lotus CIandClaude Opus 5.5 2b6bb20104 feat(lotus): remember each person's volume across calls (#36)
The per-participant volume slider lived only in the tile's view model, so it
reset to 100 % on the next call, a reload, or when that person reconnected.

- lotus/lotusVolumeMemory.ts: per-Matrix-user volume in localStorage
  (`lotus-per-user-volume`), keyed by user id so it follows them across
  devices; 1 (100 %) forgets the entry; at most 50 people, least recently set
  dropped first; junk in storage ignored. Local only, never synced.
- VolumeControls take an optional initialVolume and onCommit; remote user
  media seeds from and saves to the memory. Screenshare audio stays per-share.
- commitPlaybackVolume now takes the slider's committed value. With the
  keyboard the slider commits before its last change arrives, so the
  committed (and remembered) volume was one step behind, and returning to
  100 % with End never registered.

Verified in a two-person local call: Alice sets Bob to 70 % with the
keyboard → audio element 0.70, stored 0.7; leave + rejoin → slider and audio
0.70; End → entry removed. Unit tests: memory module (LRU, reset, junk),
remembered volume restore + commit-with-value; state/tile/room/lotus suites
all pass on Node 22 (410 tests).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-25 09:57:55 -04:00
28 changed files with 1223 additions and 164 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lotusguild/element-call-embedded",
"version": "0.25.0-lotus.14",
"version": "0.25.0-lotus.21",
"files": [
"README.md",
"LICENSE-AGPL-3.0",
+14 -1
View File
@@ -90,16 +90,29 @@ body.lotus-transparent[data-background="gradient"]::before {
display: none !important;
}
/* [cinny #43] lotusHostControls=1: the host renders its own call bar, so EC's
footer is hidden (kept in the DOM but out of the layout flow). */
body.lotus-host-controls [data-testid="footer-container"] {
position: absolute !important;
visibility: hidden !important;
}
/* [lotus] Native Lotus/TDS theme, applied when lotusTheme=1, instead of the
host injecting CSS into the iframe after load. Overrides Compound design tokens
with Lotus values at the source so theming is complete and flash-free. Extend
this block with the full Lotus token map from the design system; the canvas
override below is a safe starting point that matches the Lotus dark surface. */
body.lotus-theme {
--cpd-color-bg-canvas-default: #0c0d10;
--video-tile-background: var(--cpd-color-bg-subtle-secondary);
}
/* Dark themes only: in the light theme this near-black canvas sat behind
light-theme (dark) text, e.g. the tile name tags were unreadable. */
body.lotus-theme.cpd-theme-dark,
body.lotus-theme.cpd-theme-dark-hc {
--cpd-color-bg-canvas-default: #0c0d10;
}
/* [lotus #21] Subtle contrast guard for elements that sit directly on the
transparent canvas (no opaque tile background behind them) when
lotusTransparent is set: the host's real wallpaper is unknown to us, so a
@@ -0,0 +1,8 @@
/* [cinny #43] EC's screenshare button, floating in the frame's bottom-right
corner while the host draws the rest of the call bar. */
.floating {
position: fixed;
right: var(--cpd-space-4x);
bottom: calc(env(safe-area-inset-bottom) + var(--cpd-space-4x));
z-index: 1;
}
+88
View File
@@ -0,0 +1,88 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it, vi } from "vitest";
import { act, render, screen } from "@testing-library/react";
import userEvent from "@testing-library/user-event";
import { TooltipProvider } from "@vector-im/compound-web";
import { BehaviorSubject } from "rxjs";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { LotusFrameScreenshare } from "./LotusFrameScreenshare";
import {
lotusFrameScreenshareEnabled,
lotusFrameScreenshareVisible$,
parseFrameScreensharePayload,
} from "./lotusControls";
// lotusFlag memoizes the URL params on first read, so set them once here.
window.history.pushState(
{},
"",
"/?lotusHostControls=1&lotusFrameScreenshare=1",
);
const mockVm = (
toggle: (() => void) | null,
sharing = new BehaviorSubject(false),
): CallViewModel =>
({
sharingScreen$: sharing,
toggleScreenSharing: toggle,
}) as unknown as CallViewModel;
const renderButton = (vm: CallViewModel): void => {
render(
<TooltipProvider>
<LotusFrameScreenshare vm={vm} />
</TooltipProvider>,
);
};
describe("LotusFrameScreenshare", () => {
it("is enabled by lotusHostControls + lotusFrameScreenshare", () => {
expect(lotusFrameScreenshareEnabled()).toBe(true);
});
it("toggles the share from a click inside the frame", async () => {
const toggle = vi.fn();
renderButton(mockVm(toggle));
await userEvent.click(screen.getByTestId("lotus_frame_screenshare"));
expect(toggle).toHaveBeenCalledTimes(1);
});
it("follows the sharing state", () => {
const sharing = new BehaviorSubject(false);
renderButton(mockVm(vi.fn(), sharing));
const button = screen.getByTestId("lotus_frame_screenshare");
expect(button).toHaveAttribute("aria-checked", "false");
act(() => sharing.next(true));
expect(button).toHaveAttribute("aria-checked", "true");
});
it("hides while the host's policy forbids sharing, unless a share is live", () => {
const sharing = new BehaviorSubject(false);
act(() => lotusFrameScreenshareVisible$.next(false));
renderButton(mockVm(vi.fn(), sharing));
expect(screen.queryByTestId("lotus_frame_screenshare")).toBeNull();
act(() => sharing.next(true));
expect(screen.getByTestId("lotus_frame_screenshare")).toBeInTheDocument();
act(() => lotusFrameScreenshareVisible$.next(true));
});
it("parses the set_frame_screenshare payload", () => {
expect(parseFrameScreensharePayload({ visible: false })).toBe(false);
expect(parseFrameScreensharePayload({ visible: true })).toBe(true);
expect(parseFrameScreensharePayload({})).toBeUndefined();
expect(parseFrameScreensharePayload(null)).toBeUndefined();
});
it("renders nothing when screensharing is unavailable", () => {
renderButton(mockVm(null));
expect(screen.queryByTestId("lotus_frame_screenshare")).toBeNull();
});
});
+41
View File
@@ -0,0 +1,41 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type FC } from "react";
import { ShareScreenButton } from "../button";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { useBehavior } from "../useBehavior";
import {
lotusFrameScreenshareEnabled,
lotusFrameScreenshareVisible$,
} from "./lotusControls";
import styles from "./LotusFrameScreenshare.module.css";
/**
* [cinny #43] EC's screenshare button, shown inside the frame when the host
* can't start a share for us (`lotusFrameScreenshare`). The rest of EC's
* footer stays hidden behind the host's call bar.
*/
export const LotusFrameScreenshare: FC<{ vm: CallViewModel }> = ({ vm }) => {
const sharing = useBehavior(vm.sharingScreen$);
const visible = useBehavior(lotusFrameScreenshareVisible$);
if (!lotusFrameScreenshareEnabled() || !vm.toggleScreenSharing) return null;
// Hidden by the host's call policy, but never while a share is live: the
// user must still be able to stop it.
if (!visible && !sharing) return null;
return (
<div className={styles.floating}>
<ShareScreenButton
size="md"
enabled={sharing}
onClick={vm.toggleScreenSharing}
data-testid="lotus_frame_screenshare"
/>
</div>
);
};
+5
View File
@@ -30,6 +30,9 @@ describe("LotusWidgetActions", () => {
LotusWidgetActions.SetLayout,
LotusWidgetActions.OpenSettings,
LotusWidgetActions.ToggleReactions,
LotusWidgetActions.SetScreenshare,
LotusWidgetActions.SetHotkeys,
LotusWidgetActions.SetFrameScreenshare,
];
expect(new Set(LOTUS_TO_WIDGET_ACTIONS)).toEqual(new Set(expectedToWidget));
@@ -50,5 +53,7 @@ describe("LotusWidgetActions", () => {
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(
LotusWidgetActions.ControlsState,
);
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.MicLevel);
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.Hotkey);
});
});
+33 -1
View File
@@ -75,11 +75,40 @@ export enum LotusWidgetActions {
/** toWidget: toggle the reactions / raise-hand menu (cinny #43). */
ToggleReactions = "io.lotus.toggle_reactions",
/**
* fromWidget: `{ screensharing: boolean, layout: "grid" | "spotlight" | null }`
* toWidget: start/stop sharing `{ on?: boolean }` (omit to toggle; cinny #43).
* Starting calls getDisplayMedia, which needs the user's click: the host
* sends this with Capability Delegation (`delegate: "display-capture"`), and
* without it the browser rejects the share.
*/
SetScreenshare = "io.lotus.set_screenshare",
/**
* toWidget: `{ codes: string[] }` — KeyboardEvent.code values (PTT, deafen)
* to report while focus is inside this frame (cinny #43). `[]` stops.
*/
SetHotkeys = "io.lotus.set_hotkeys",
/**
* toWidget: `{ visible: boolean }` — whether the in-frame screenshare button
* (`lotusFrameScreenshare`) may show, e.g. false while the room's call
* policy forbids screensharing (cinny #43). Defaults to visible.
*/
SetFrameScreenshare = "io.lotus.set_frame_screenshare",
/**
* fromWidget: a watched key went down/up here, or this window's focus
* changed; see `LotusHotkeyReport` (cinny #43).
*/
Hotkey = "io.lotus.hotkey",
/**
* fromWidget: `{ screensharing: boolean, layout: "grid" | "spotlight" | null,
* screenshareAction: true, hotkeys: true, frameScreenshare: boolean }`
* whenever either changes (cinny #43), so the host stops reading EC's DOM for
* them. Its arrival also tells the host this fork supports the actions above.
*/
ControlsState = "io.lotus.controls_state",
/**
* fromWidget: local mic level `{ bars: 0 | 1 | 2 | 3 }` (cinny #146), sent
* only when it changes (≤ 10 Hz); 0 while muted or with no mic.
*/
MicLevel = "io.lotus.mic_level",
}
/** toWidget Lotus actions that must be allow-listed in `initializeWidget`. */
@@ -93,4 +122,7 @@ export const LOTUS_TO_WIDGET_ACTIONS: LotusWidgetActions[] = [
LotusWidgetActions.SetLayout,
LotusWidgetActions.OpenSettings,
LotusWidgetActions.ToggleReactions,
LotusWidgetActions.SetScreenshare,
LotusWidgetActions.SetHotkeys,
LotusWidgetActions.SetFrameScreenshare,
];
+36 -1
View File
@@ -10,7 +10,11 @@ import { afterEach, beforeEach, expect, test, vi } from "vitest";
import { of } from "rxjs";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { startLotusAudioInject } from "./lotusAudioInject";
import {
MAX_INJECT_BYTES,
parseInjectSource,
startLotusAudioInject,
} from "./lotusAudioInject";
import { LotusWidgetActions } from "./lotusActions";
const lazyActions = new EventEmitter();
@@ -223,3 +227,34 @@ test("#14: the shared context stays open while another instance is still active"
stopB();
expect(ctx.close).toHaveBeenCalledTimes(1);
});
// [cinny #43] Cross-origin hosts send the clip bytes instead of a blob: URL.
test("parseInjectSource prefers the clip bytes over the url", () => {
const audio = new ArrayBuffer(16);
expect(parseInjectSource({ audio, url: "https://x.example/a.ogg" })).toBe(
audio,
);
});
test("parseInjectSource falls back to a safe url", () => {
expect(parseInjectSource({ url: "https://x.example/a.ogg" })).toBe(
"https://x.example/a.ogg",
);
expect(
parseInjectSource({
audio: new ArrayBuffer(0),
url: "https://x.example/a.ogg",
}),
).toBe("https://x.example/a.ogg");
expect(parseInjectSource({ url: "javascript" + ":alert(1)" })).toBeNull();
expect(parseInjectSource({})).toBeNull();
expect(parseInjectSource(undefined)).toBeNull();
});
test("parseInjectSource rejects oversized or non-buffer audio", () => {
expect(
parseInjectSource({ audio: new ArrayBuffer(MAX_INJECT_BYTES + 1) }),
).toBeNull();
expect(parseInjectSource({ audio: "not bytes" })).toBeNull();
expect(parseInjectSource({ audio: new Uint8Array(4) })).toBeNull();
});
+53 -25
View File
@@ -52,8 +52,10 @@ function acquireSharedAudio(): {
* that was impossible against the prebuilt EC bundle (LiveKit's
* LocalParticipant lived in EC's module scope).
*
* Action data: `{ url: string, volume?: number }`. `url` must be an https/blob
* URL (the host resolves mxc → media URL).
* Action data: `{ url?: string, audio?: ArrayBuffer, volume?: number }`.
* `audio` is the clip's bytes (cinny #43: a host `blob:` URL can't be fetched
* once EC is served from its own origin, so the host sends the bytes and they
* win); otherwise `url` must be an https/blob URL (older hosts).
*
* No effect unless the host sends the action. Returns a teardown function that
* also aborts any clip still playing.
@@ -88,12 +90,12 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
return;
}
const data = ev.detail.data as
| { url?: unknown; volume?: unknown }
| { url?: unknown; audio?: unknown; volume?: unknown }
| undefined;
const url = typeof data?.url === "string" ? safeMediaUrl(data.url) : null;
if (!url) {
const clip = parseInjectSource(data);
if (!clip) {
w.api.transport.reply(ev.detail, {});
logger.warn("[lotus] inject_audio: missing/invalid url");
logger.warn("[lotus] inject_audio: missing/invalid url or audio");
return;
}
const volume =
@@ -115,7 +117,7 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
}
w.api.transport.reply(ev.detail, {});
void playInjectedClip(url, volume, rooms, activeClips).catch((e) =>
void playInjectedClip(clip, volume, rooms, activeClips).catch((e) =>
logger.warn("[lotus] inject_audio failed", e),
);
};
@@ -144,6 +146,27 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
};
}
/** Largest clip accepted as bytes (the host's soundboard clips are small). */
export const MAX_INJECT_BYTES = 8 * 1024 * 1024;
/**
* The clip to play: the bytes the host sent (`audio`), or else a fetchable
* `url`. Exported for tests.
*/
export function parseInjectSource(
data: { url?: unknown; audio?: unknown } | undefined,
): string | ArrayBuffer | null {
const audio = data?.audio;
if (
audio instanceof ArrayBuffer &&
audio.byteLength > 0 &&
audio.byteLength <= MAX_INJECT_BYTES
) {
return audio;
}
return typeof data?.url === "string" ? safeMediaUrl(data.url) : null;
}
/** Only allow fetchable media URLs; never same-origin credentialed GETs etc. */
function safeMediaUrl(raw: string): string | null {
try {
@@ -155,7 +178,7 @@ function safeMediaUrl(raw: string): string | null {
}
async function playInjectedClip(
url: string,
clip: string | ArrayBuffer,
volume: number,
rooms: LivekitRoom[],
activeClips: Set<() => void>,
@@ -188,25 +211,30 @@ async function playInjectedClip(
};
activeClips.add(placeholder);
let resp: Response;
try {
resp = await fetch(url, {
credentials: "omit",
mode: "cors",
signal: controller.signal,
});
} catch (e) {
// Superseded by a newer clip mid-fetch — expected, not a failure.
let arrayBuffer: ArrayBuffer;
if (typeof clip !== "string") {
arrayBuffer = clip;
} else {
let resp: Response;
try {
resp = await fetch(clip, {
credentials: "omit",
mode: "cors",
signal: controller.signal,
});
} catch (e) {
// Superseded by a newer clip mid-fetch — expected, not a failure.
if (aborted) return;
throw e;
}
if (aborted) return;
if (!resp.ok) {
activeClips.delete(placeholder);
throw new Error(`fetch ${clip} -> ${resp.status}`);
}
arrayBuffer = await resp.arrayBuffer();
if (aborted) return;
throw e;
}
if (aborted) return;
if (!resp.ok) {
activeClips.delete(placeholder);
throw new Error(`fetch ${url} -> ${resp.status}`);
}
const arrayBuffer = await resp.arrayBuffer();
if (aborted) return;
// [lotus] Reuse the shared module-level context/destination (#14) rather
// than `new AudioContext()` per clip — see the declaration above.
+19 -1
View File
@@ -7,7 +7,11 @@ Please see LICENSE in the repository root for full details.
import { describe, expect, it } from "vitest";
import { parseLayoutPayload, parseSettingsPayload } from "./lotusControls";
import {
parseLayoutPayload,
parseSettingsPayload,
shouldToggleScreenshare,
} from "./lotusControls";
describe("parseLayoutPayload", () => {
it("accepts grid and spotlight", () => {
@@ -33,3 +37,17 @@ describe("parseSettingsPayload", () => {
expect(parseSettingsPayload({ open: "yes" }, true)).toBe(false);
});
});
describe("shouldToggleScreenshare", () => {
it("toggles only when an explicit on differs from the current state", () => {
expect(shouldToggleScreenshare({ on: true }, false)).toBe(true);
expect(shouldToggleScreenshare({ on: true }, true)).toBe(false);
expect(shouldToggleScreenshare({ on: false }, true)).toBe(true);
expect(shouldToggleScreenshare({ on: false }, false)).toBe(false);
});
it("toggles when on is missing or not a boolean", () => {
expect(shouldToggleScreenshare({}, false)).toBe(true);
expect(shouldToggleScreenshare({ on: "yes" }, true)).toBe(true);
expect(shouldToggleScreenshare(null, true)).toBe(true);
});
});
+83 -5
View File
@@ -6,13 +6,13 @@ Please see LICENSE in the repository root for full details.
*/
import { type IWidgetApiRequest } from "matrix-widget-api";
import { combineLatest, of, type Subscription } from "rxjs";
import { BehaviorSubject, combineLatest, of, type Subscription } from "rxjs";
import { distinctUntilChanged, map, switchMap } from "rxjs/operators";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { type LayoutMode } from "../state/LayoutSwitchViewModel";
import { widget } from "../widget";
import { LotusWidgetActions, lotusSendToHost } from "./lotusWidget";
import { LotusWidgetActions, lotusFlag, lotusSendToHost } from "./lotusWidget";
/** Window event the reactions button listens for (see ReactionToggleButton). */
export const LOTUS_TOGGLE_REACTIONS_EVENT = "lotus:toggle-reactions";
@@ -21,6 +21,42 @@ export interface LotusControlsState {
screensharing: boolean;
/** Null while EC offers no layout switch (e.g. PiP or a 1:1 layout). */
layout: LayoutMode | null;
/** This fork handles `io.lotus.set_screenshare` (always true). */
screenshareAction: true;
/** This fork handles `io.lotus.set_hotkeys` (always true). */
hotkeys: true;
/**
* EC's own screenshare button is shown inside the frame (see
* `lotusFrameScreenshare`), so the host should hide its own.
*/
frameScreenshare: boolean;
}
/**
* [cinny #43] `lotusFrameScreenshare=1` (with `lotusHostControls`): the host
* can't hand the user's click into this frame (no Capability Delegation:
* Firefox, Safari, WebKitGTK), and `getDisplayMedia` needs that click here.
* So EC's own screenshare button stays visible in the frame, where the click
* is native, instead of the host clicking it through the DOM.
*/
export function lotusFrameScreenshareEnabled(): boolean {
return lotusFlag("lotusHostControls") && lotusFlag("lotusFrameScreenshare");
}
/**
* Host-controlled visibility of the in-frame screenshare button
* (`io.lotus.set_frame_screenshare`): the host hides it while the room's
* call policy forbids screensharing, as it does its own call-bar button.
*/
export const lotusFrameScreenshareVisible$ = new BehaviorSubject(true);
/** `{ visible }` payload → the visibility to apply, or undefined. Exported for tests. */
export function parseFrameScreensharePayload(
data: unknown,
): boolean | undefined {
if (typeof data !== "object" || data === null) return undefined;
const { visible } = data as { visible?: unknown };
return typeof visible === "boolean" ? visible : undefined;
}
/** `{ layout }` payload → a layout mode, or undefined if invalid. Exported for tests. */
@@ -39,13 +75,28 @@ export function parseSettingsPayload(data: unknown, current: boolean): boolean {
return !current;
}
/**
* `{ on? }` payload + current sharing state → whether to call the toggle.
* Exported for tests.
*/
export function shouldToggleScreenshare(
data: unknown,
sharing: boolean,
): boolean {
if (typeof data === "object" && data !== null && "on" in data) {
const { on } = data as { on?: unknown };
if (typeof on === "boolean") return on !== sharing;
}
return true;
}
/**
* [cinny #43] Widget-API replacements for the host's DOM access to EC's
* controls: layout switch, settings modal and reactions menu, plus a
* `controls_state` report (screensharing + layout) so the host no longer reads
* EC's DOM for them. Screensharing itself stays host-DOM driven for now:
* `getDisplayMedia` needs the user's click to reach this frame (Capability
* Delegation), which a plain widget message doesn't carry.
* EC's DOM for them. Screenshare start/stop too: `getDisplayMedia` needs the
* user's click in this frame, so the host sends `set_screenshare` with
* Capability Delegation (Chromium). Elsewhere it keeps clicking EC's button.
*
* No effect unless the host sends the actions; registering is safe whenever
* we're a widget. Returns a teardown function.
@@ -69,10 +120,29 @@ export function startLotusControls(vm: CallViewModel): () => void {
w.api.transport.reply(ev.detail, {});
window.dispatchEvent(new Event(LOTUS_TOGGLE_REACTIONS_EVENT));
};
const onSetScreenshare = (ev: CustomEvent<IWidgetApiRequest>): void => {
w.api.transport.reply(ev.detail, {});
// Call synchronously: the delegated activation is only good for a few
// seconds after the message arrived.
if (shouldToggleScreenshare(ev.detail.data, vm.sharingScreen$.value)) {
vm.toggleScreenSharing?.();
}
};
w.lazyActions.on(LotusWidgetActions.SetLayout, onSetLayout);
w.lazyActions.on(LotusWidgetActions.OpenSettings, onOpenSettings);
w.lazyActions.on(LotusWidgetActions.ToggleReactions, onToggleReactions);
const onSetFrameScreenshare = (ev: CustomEvent<IWidgetApiRequest>): void => {
w.api.transport.reply(ev.detail, {});
const visible = parseFrameScreensharePayload(ev.detail.data);
if (visible !== undefined) lotusFrameScreenshareVisible$.next(visible);
};
w.lazyActions.on(LotusWidgetActions.SetScreenshare, onSetScreenshare);
w.lazyActions.on(
LotusWidgetActions.SetFrameScreenshare,
onSetFrameScreenshare,
);
const sub: Subscription = combineLatest([
vm.sharingScreen$,
@@ -85,6 +155,9 @@ export function startLotusControls(vm: CallViewModel): () => void {
([screensharing, layout]): LotusControlsState => ({
screensharing,
layout,
screenshareAction: true,
hotkeys: true,
frameScreenshare: lotusFrameScreenshareEnabled(),
}),
),
distinctUntilChanged(
@@ -100,5 +173,10 @@ export function startLotusControls(vm: CallViewModel): () => void {
w.lazyActions.off(LotusWidgetActions.SetLayout, onSetLayout);
w.lazyActions.off(LotusWidgetActions.OpenSettings, onOpenSettings);
w.lazyActions.off(LotusWidgetActions.ToggleReactions, onToggleReactions);
w.lazyActions.off(LotusWidgetActions.SetScreenshare, onSetScreenshare);
w.lazyActions.off(
LotusWidgetActions.SetFrameScreenshare,
onSetFrameScreenshare,
);
};
}
+50
View File
@@ -0,0 +1,50 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import {
isEditableTarget,
isInteractiveTarget,
parseHotkeyCodes,
} from "./lotusHotkeys";
describe("parseHotkeyCodes", () => {
it("keeps KeyboardEvent.code-like strings", () => {
expect(parseHotkeyCodes({ codes: ["Space", "KeyM", "F13"] })).toEqual([
"Space",
"KeyM",
"F13",
]);
});
it("drops junk and caps the list", () => {
expect(parseHotkeyCodes({ codes: ["Space", 3, "", "a b", "<x>"] })).toEqual(
["Space"],
);
expect(parseHotkeyCodes({ codes: Array(20).fill("KeyA") })).toHaveLength(8);
expect(parseHotkeyCodes({})).toEqual([]);
expect(parseHotkeyCodes(null)).toEqual([]);
});
});
describe("target checks", () => {
it("treats inputs and contenteditable as editable", () => {
document.body.innerHTML =
'<input id="i"><div contenteditable="true"><span id="s">x</span></div><div id="d"></div>';
expect(isEditableTarget(document.getElementById("i"))).toBe(true);
expect(isEditableTarget(document.getElementById("s"))).toBe(true);
expect(isEditableTarget(document.getElementById("d"))).toBe(false);
expect(isEditableTarget(null)).toBe(false);
});
it("treats buttons, links and button roles as interactive", () => {
document.body.innerHTML =
'<button id="b">b</button><div role="menuitem"><span id="m">m</span></div><p id="p"></p>';
expect(isInteractiveTarget(document.getElementById("b"))).toBe(true);
expect(isInteractiveTarget(document.getElementById("m"))).toBe(true);
expect(isInteractiveTarget(document.getElementById("p"))).toBe(false);
});
});
+159
View File
@@ -0,0 +1,159 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type IWidgetApiRequest } from "matrix-widget-api";
import { widget } from "../widget";
import { LotusWidgetActions, lotusSendToHost } from "./lotusWidget";
/** fromWidget `io.lotus.hotkey` payload. */
export type LotusHotkeyReport =
| {
type: "keydown" | "keyup";
code: string;
repeat: boolean;
ctrlKey: boolean;
altKey: boolean;
metaKey: boolean;
shiftKey: boolean;
/** The key went to a text field (typing, not a hotkey). */
editable: boolean;
/** The key went to a button/link/etc. (Space activates it). */
interactive: boolean;
}
/** This window gained or lost focus: the host releases a held PTT. */
| { type: "focus" };
const MAX_CODES = 8;
// Codes the host currently owns. Module-level so EC's own shortcuts
// (useCallViewKeyboardShortcuts) can stand down for them.
let hostCodes = new Set<string>();
/**
* Whether `code` is one of the host's call hotkeys (its PTT or deafen key).
* EC's built-in shortcuts ignore such keys: with the defaults, M would
* otherwise both toggle EC's mic and the host's deafen, and Space would open
* EC's own push-to-talk alongside the host's.
*/
export function isLotusHostHotkey(code: string): boolean {
return hostCodes.has(code);
}
/** `{ codes }` payload → the KeyboardEvent.code values to watch. Exported for tests. */
export function parseHotkeyCodes(data: unknown): string[] {
if (typeof data !== "object" || data === null) return [];
const { codes } = data as { codes?: unknown };
if (!Array.isArray(codes)) return [];
return codes
.filter(
(c): c is string =>
typeof c === "string" && /^[A-Za-z0-9]{1,32}$/.test(c),
)
.slice(0, MAX_CODES);
}
/** Same rules as the host's useCallHotkeys. Exported for tests. */
export function isEditableTarget(el: Element | null): boolean {
if (!(el instanceof HTMLElement)) return false;
const tag = el.tagName;
if (tag === "INPUT" || tag === "TEXTAREA" || tag === "SELECT") return true;
let node: HTMLElement | null = el;
while (node && node !== el.ownerDocument.body) {
const ce = node.getAttribute("contenteditable");
if (ce === "" || ce === "true" || ce === "plaintext-only") return true;
if (ce === "false") return false;
node = node.parentElement;
}
return false;
}
/** Exported for tests. */
export function isInteractiveTarget(el: Element | null): boolean {
if (!(el instanceof HTMLElement)) return false;
const tag = el.tagName;
if (tag === "BUTTON" || tag === "A" || tag === "SELECT") return true;
let node: HTMLElement | null = el;
while (node && node !== el.ownerDocument.body) {
const role = node.getAttribute("role");
if (
role === "button" ||
role === "link" ||
role === "menuitem" ||
role === "tab"
)
return true;
node = node.parentElement;
}
return false;
}
/**
* [cinny #43] Call hotkeys (push-to-talk, deafen) while focus is inside this
* frame. The host used to add key listeners to this window directly, which
* needs same-origin access. Now it sends `io.lotus.set_hotkeys { codes }` and
* we report those keys back as `io.lotus.hotkey`. The default action is
* cancelled here, synchronously (it can't be across frames), with the host's
* rules: never while typing in a field, and not on a focused button/link so
* keyboard users can still activate it. Returns a teardown function.
*/
export function startLotusHotkeys(): () => void {
const w = widget;
if (!w) return (): void => undefined;
hostCodes = new Set<string>();
const onSetHotkeys = (ev: CustomEvent<IWidgetApiRequest>): void => {
w.api.transport.reply(ev.detail, {});
hostCodes = new Set(parseHotkeyCodes(ev.detail.data));
};
const onKey = (e: KeyboardEvent): void => {
if (!hostCodes.has(e.code)) return;
const target = e.target instanceof Element ? e.target : null;
const editable = isEditableTarget(target);
const interactive = isInteractiveTarget(target);
if (
e.type === "keydown" &&
!editable &&
!interactive &&
!e.ctrlKey &&
!e.altKey &&
!e.metaKey
) {
e.preventDefault();
}
const report: LotusHotkeyReport = {
type: e.type === "keyup" ? "keyup" : "keydown",
code: e.code,
repeat: e.repeat,
ctrlKey: e.ctrlKey,
altKey: e.altKey,
metaKey: e.metaKey,
shiftKey: e.shiftKey,
editable,
interactive,
};
lotusSendToHost(LotusWidgetActions.Hotkey, report);
};
const onFocusChange = (): void => {
if (hostCodes.size > 0)
lotusSendToHost(LotusWidgetActions.Hotkey, { type: "focus" });
};
w.lazyActions.on(LotusWidgetActions.SetHotkeys, onSetHotkeys);
window.addEventListener("keydown", onKey, true);
window.addEventListener("keyup", onKey, true);
window.addEventListener("blur", onFocusChange);
window.addEventListener("focus", onFocusChange);
return (): void => {
w.lazyActions.off(LotusWidgetActions.SetHotkeys, onSetHotkeys);
window.removeEventListener("keydown", onKey, true);
window.removeEventListener("keyup", onKey, true);
window.removeEventListener("blur", onFocusChange);
window.removeEventListener("focus", onFocusChange);
hostCodes = new Set<string>();
};
}
+37
View File
@@ -0,0 +1,37 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import { MicLevelQuantizer } from "./lotusMicLevel";
describe("MicLevelQuantizer", () => {
it("maps RMS to 0–3 bars", () => {
expect(new MicLevelQuantizer().push(0.001)).toBe(0);
expect(new MicLevelQuantizer().push(0.008)).toBe(1);
expect(new MicLevelQuantizer().push(0.02)).toBe(2);
expect(new MicLevelQuantizer().push(0.2)).toBe(3);
});
it("rises at once and falls one bar per sample", () => {
const q = new MicLevelQuantizer();
expect(q.push(0.2)).toBe(3);
expect(q.push(0)).toBe(2);
expect(q.push(0)).toBe(1);
expect(q.push(0.02)).toBe(2);
expect(q.push(0)).toBe(1);
expect(q.push(0)).toBe(0);
expect(q.push(0)).toBe(0);
});
it("reset drops straight to 0", () => {
const q = new MicLevelQuantizer();
q.push(0.2);
q.reset();
expect(q.push(0)).toBe(0);
});
});
+192
View File
@@ -0,0 +1,192 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type LocalTrackPublication,
type Room as LivekitRoom,
RoomEvent,
Track,
} from "livekit-client";
import { Observable, type Subscription, share, switchMap } from "rxjs";
import { distinctUntilChanged, map } from "rxjs/operators";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { widget } from "../widget";
import { LotusWidgetActions, lotusFlag, lotusSendToHost } from "./lotusWidget";
/**
* [lotus #146] One local mic sampler shared by the host's mic level meter
* (`io.lotus.mic_level`) and "talking while muted" (#37, lotusMutedSpeech).
* It taps a CLONE of the published mic track (the post-processor track when
* the in-source denoiser is active, so what's measured is what's sent) and
* reads RMS at ~10 Hz while a mic track is published, muted or not.
* Local only: nothing here reaches other participants.
*/
const SAMPLE_MS = 100;
export interface LocalMicSample {
rms: number;
/** The mic is published but muted (the clone still hears it). */
muted: boolean;
}
const micPublication = (
room: LivekitRoom,
): { track: MediaStreamTrack; muted: boolean } | null => {
const pub: LocalTrackPublication | undefined =
room.localParticipant.getTrackPublication(Track.Source.Microphone);
const track = pub?.track?.mediaStreamTrack;
return track && track.readyState === "live"
? { track, muted: pub?.isMuted ?? false }
: null;
};
const samplers = new WeakMap<
CallViewModel,
Observable<LocalMicSample | null>
>();
/**
* RMS samples of the local mic, or `null` while no mic track is published.
* Shared per call view model, so the meter and the muted-speech detector use
* one AudioContext between them.
*/
export function observeLocalMicSample$(
vm: CallViewModel,
): Observable<LocalMicSample | null> {
const cached = samplers.get(vm);
if (cached) return cached;
const sampler$ = vm.allConnections$.pipe(
switchMap(
(data) =>
new Observable<LocalMicSample | null>((subscriber) => {
const rooms = data.getConnections().map((c) => c.livekitRoom);
let ctx: AudioContext | null = null;
let clone: MediaStreamTrack | null = null;
let timer: ReturnType<typeof setInterval> | undefined;
let tapped: MediaStreamTrack | null = null;
let muted = false;
const stopTap = (): void => {
if (timer !== undefined) clearInterval(timer);
timer = undefined;
clone?.stop();
clone = null;
void ctx?.close().catch(() => undefined);
ctx = null;
tapped = null;
subscriber.next(null);
};
const startTap = (source: MediaStreamTrack): void => {
try {
clone = source.clone();
// Muting disables the published track; the clone must still hear.
clone.enabled = true;
ctx = new AudioContext();
const analyser = ctx.createAnalyser();
analyser.fftSize = 1024;
ctx
.createMediaStreamSource(new MediaStream([clone]))
.connect(analyser);
const buf = new Float32Array(analyser.fftSize);
tapped = source;
timer = setInterval(() => {
analyser.getFloatTimeDomainData(buf);
let sum = 0;
for (let i = 0; i < buf.length; i += 1) sum += buf[i] * buf[i];
subscriber.next({ rms: Math.sqrt(sum / buf.length), muted });
}, SAMPLE_MS);
} catch {
stopTap();
}
};
const reconcile = (): void => {
const pub =
rooms.map(micPublication).find((p) => p !== null) ?? null;
muted = pub?.muted ?? false;
const track = pub?.track ?? null;
if (track === tapped) return;
if (tapped) stopTap();
if (track) startTap(track);
};
const events = [
RoomEvent.TrackMuted,
RoomEvent.TrackUnmuted,
RoomEvent.LocalTrackPublished,
RoomEvent.LocalTrackUnpublished,
RoomEvent.Reconnected,
RoomEvent.Disconnected,
] as const;
rooms.forEach((room) =>
events.forEach((ev) => room.on(ev, reconcile)),
);
subscriber.next(null);
reconcile();
return (): void => {
rooms.forEach((room) =>
events.forEach((ev) => room.off(ev, reconcile)),
);
if (tapped) stopTap();
};
}),
),
share(),
);
samplers.set(vm, sampler$);
return sampler$;
}
/** RMS at which each bar lights: ≈ −46, −36 and −26 dBFS. */
export const BAR_THRESHOLDS = [0.005, 0.015, 0.05] as const;
/**
* Quantise RMS to 0–3 bars with a little hysteresis: rises at once, falls one
* bar per sample, so the meter doesn't flicker between words. Unit-tested.
*/
export class MicLevelQuantizer {
private bars = 0;
public reset(): void {
this.bars = 0;
}
public push(rms: number): number {
const target = BAR_THRESHOLDS.filter((t) => rms >= t).length;
this.bars = target >= this.bars ? target : this.bars - 1;
return this.bars;
}
}
/**
* Send the host `io.lotus.mic_level { bars }` (0–3) whenever the quantised
* level changes; 0 while muted or with no mic. At most one message per sample
* (10 Hz) and none during steady silence. Opt-in with the rest of the host
* state stream (`lotusCallState=1`). Returns a teardown function.
*/
export function startLotusMicLevel(vm: CallViewModel): () => void {
if (!lotusFlag("lotusCallState") || !widget) return (): void => undefined;
const quantizer = new MicLevelQuantizer();
const sub: Subscription = observeLocalMicSample$(vm)
.pipe(
map((sample) => {
if (!sample || sample.muted) {
quantizer.reset();
return 0;
}
return quantizer.push(sample.rms);
}),
distinctUntilChanged(),
)
.subscribe((bars) => {
lotusSendToHost(LotusWidgetActions.MicLevel, { bars });
});
return (): void => sub.unsubscribe();
}
+19 -89
View File
@@ -6,14 +6,15 @@ Please see LICENSE in the repository root for full details.
*/
import {
type LocalTrackPublication,
type Room as LivekitRoom,
RoomEvent,
Track,
} from "livekit-client";
import { Observable, distinctUntilChanged, switchMap } from "rxjs";
type Observable,
distinctUntilChanged,
map,
scan,
startWith,
} from "rxjs";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { type LocalMicSample, observeLocalMicSample$ } from "./lotusMicLevel";
/**
* [lotus #37] "Talking while muted" detection for the LOCAL participant.
@@ -23,13 +24,12 @@ import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
* tells the user they are talking into a muted mic. This taps a CLONE of the
* published track (the post-processor track when the in-source denoiser is
* active, so keyboard noise doesn't count), samples RMS at ~10 Hz and emits a
* debounced boolean. Zero cost when unmuted (tap torn down), local-only —
* debounced boolean. Local-only —
* the flag rides `io.lotus.call_state` to the host and never reaches other
* participants.
*/
export const MUTED_SPEECH_RMS = 0.015; // ≈ −36 dBFS; normal speech into a headset is 0.05–0.3
const SAMPLE_MS = 100;
const ON_SAMPLES = 3; // 300 ms of voice before we say "talking"
const OFF_SAMPLES = 8; // 800 ms of quiet before we drop it
@@ -61,94 +61,24 @@ export class MutedSpeechGate {
}
}
const mutedMicTrack = (room: LivekitRoom): MediaStreamTrack | null => {
const pub: LocalTrackPublication | undefined =
room.localParticipant.getTrackPublication(Track.Source.Microphone);
const track = pub?.track?.mediaStreamTrack;
return pub?.isMuted && track && track.readyState === "live" ? track : null;
};
/**
* Emits while the local mic is muted and voice is detected on it. Emits
* `false` whenever the mic is unmuted, unpublished or the connection changes.
* [lotus #146] Fed by the shared local mic sampler (lotusMicLevel.ts), which
* also drives the host's mic level meter while unmuted.
*/
export function observeSpeakingWhileMuted$(
vm: CallViewModel,
): Observable<boolean> {
return vm.allConnections$.pipe(
switchMap(
(data) =>
new Observable<boolean>((subscriber) => {
const rooms = data.getConnections().map((c) => c.livekitRoom);
let ctx: AudioContext | null = null;
let clone: MediaStreamTrack | null = null;
let timer: ReturnType<typeof setInterval> | undefined;
let tapped: MediaStreamTrack | null = null;
const stopTap = (): void => {
if (timer !== undefined) clearInterval(timer);
timer = undefined;
clone?.stop();
clone = null;
void ctx?.close().catch(() => undefined);
ctx = null;
tapped = null;
subscriber.next(false);
};
const startTap = (source: MediaStreamTrack): void => {
try {
clone = source.clone();
clone.enabled = true; // the source is disabled by the mute — the clone must not be
ctx = new AudioContext();
const analyser = ctx.createAnalyser();
analyser.fftSize = 1024;
ctx
.createMediaStreamSource(new MediaStream([clone]))
.connect(analyser);
const buf = new Float32Array(analyser.fftSize);
const gate = new MutedSpeechGate();
tapped = source;
timer = setInterval(() => {
analyser.getFloatTimeDomainData(buf);
let sum = 0;
for (let i = 0; i < buf.length; i += 1) sum += buf[i] * buf[i];
subscriber.next(gate.push(Math.sqrt(sum / buf.length)));
}, SAMPLE_MS);
} catch {
stopTap();
}
};
const reconcile = (): void => {
const track =
rooms.map(mutedMicTrack).find((t) => t !== null) ?? null;
if (track === tapped) return;
if (tapped) stopTap();
if (track) startTap(track);
};
const events = [
RoomEvent.TrackMuted,
RoomEvent.TrackUnmuted,
RoomEvent.LocalTrackPublished,
RoomEvent.LocalTrackUnpublished,
RoomEvent.Reconnected,
RoomEvent.Disconnected,
] as const;
rooms.forEach((room) =>
events.forEach((ev) => room.on(ev, reconcile)),
);
subscriber.next(false);
reconcile();
return () => {
rooms.forEach((room) =>
events.forEach((ev) => room.off(ev, reconcile)),
);
if (tapped) stopTap();
};
}),
),
return observeLocalMicSample$(vm).pipe(
scan((gate: MutedSpeechGate | null, sample: LocalMicSample | null) => {
if (!sample?.muted) return null;
const g = gate ?? new MutedSpeechGate();
g.push(sample.rms);
return g;
}, null),
map((gate) => gate?.value ?? false),
startWith(false),
distinctUntilChanged(),
);
}
+61
View File
@@ -0,0 +1,61 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { beforeEach, describe, expect, it } from "vitest";
import {
MAX_REMEMBERED,
getRememberedVolume,
rememberVolume,
} from "./lotusVolumeMemory";
describe("lotusVolumeMemory", () => {
beforeEach(() => localStorage.clear());
it("defaults to 1 and remembers a set volume", () => {
expect(getRememberedVolume("@bob:x")).toBe(1);
rememberVolume("@bob:x", 0.4);
expect(getRememberedVolume("@bob:x")).toBe(0.4);
});
it("forgets the entry when set back to 1", () => {
rememberVolume("@bob:x", 0.4);
rememberVolume("@bob:x", 1);
expect(getRememberedVolume("@bob:x")).toBe(1);
expect(localStorage.getItem("lotus-per-user-volume")).toBe("{}");
});
it("keeps only the most recently set entries", () => {
for (let i = 0; i < MAX_REMEMBERED + 5; i++)
rememberVolume(`@u${i}:x`, 0.5);
expect(getRememberedVolume("@u0:x")).toBe(1);
expect(getRememberedVolume("@u4:x")).toBe(1);
expect(getRememberedVolume("@u5:x")).toBe(0.5);
expect(getRememberedVolume(`@u${MAX_REMEMBERED + 4}:x`)).toBe(0.5);
});
it("re-setting an old entry makes it recent", () => {
for (let i = 0; i < MAX_REMEMBERED; i++) rememberVolume(`@u${i}:x`, 0.5);
rememberVolume("@u0:x", 0.7);
rememberVolume("@new:x", 0.3);
expect(getRememberedVolume("@u0:x")).toBe(0.7);
expect(getRememberedVolume("@u1:x")).toBe(1);
});
it("ignores junk in storage and invalid volumes", () => {
localStorage.setItem(
"lotus-per-user-volume",
JSON.stringify({ "@a:x": "loud", "@b:x": 0.2 }),
);
expect(getRememberedVolume("@a:x")).toBe(1);
expect(getRememberedVolume("@b:x")).toBe(0.2);
rememberVolume("@c:x", Number.NaN);
expect(getRememberedVolume("@c:x")).toBe(1);
localStorage.setItem("lotus-per-user-volume", "not json");
expect(getRememberedVolume("@b:x")).toBe(1);
});
});
+65
View File
@@ -0,0 +1,65 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
/**
* [element-call #36] Remember the per-participant volume slider across calls,
* reloads and reconnects. Keyed by Matrix user id (not device), so "Bob is
* loud" sticks when Bob switches devices. Local to this browser, never synced.
* Only voice is remembered; screenshare audio stays per-share.
*/
const STORAGE_KEY = "lotus-per-user-volume";
/** Most recently set entries kept; older ones are dropped. */
export const MAX_REMEMBERED = 50;
type VolumeMap = Record<string, number>;
const isVolume = (v: unknown): v is number =>
typeof v === "number" && Number.isFinite(v) && v >= 0 && v <= 4;
function load(): VolumeMap {
try {
const raw = localStorage.getItem(STORAGE_KEY);
if (!raw) return {};
const parsed: unknown = JSON.parse(raw);
if (typeof parsed !== "object" || parsed === null) return {};
const out: VolumeMap = {};
for (const [k, v] of Object.entries(parsed)) if (isVolume(v)) out[k] = v;
return out;
} catch {
return {};
}
}
function save(map: VolumeMap): void {
try {
localStorage.setItem(STORAGE_KEY, JSON.stringify(map));
} catch {
// Storage unavailable (private mode, quota): the slider just won't stick.
}
}
/** The remembered volume for `userId`, or 1 (100 %) if none. */
export function getRememberedVolume(userId: string): number {
return load()[userId] ?? 1;
}
/**
* Remember `volume` for `userId`. 1 (the default) forgets the entry. The entry
* moves to the end so the map is in least-recently-set order, trimmed to
* MAX_REMEMBERED.
*/
export function rememberVolume(userId: string, volume: number): void {
if (!isVolume(volume)) return;
const map = load();
delete map[userId];
if (volume !== 1) map[userId] = volume;
const keys = Object.keys(map);
for (const k of keys.slice(0, Math.max(0, keys.length - MAX_REMEMBERED)))
delete map[k];
save(map);
}
+65
View File
@@ -0,0 +1,65 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { describe, expect, it } from "vitest";
import { isFromHost, restrictToHost } from "./lotusWidgetOrigin";
const parent = {} as Window;
const other = {} as Window;
const HOST = "https://chat.example.org";
describe("isFromHost", () => {
it("accepts only the parent window at the host origin", () => {
expect(isFromHost({ source: parent, origin: HOST }, parent, HOST)).toBe(
true,
);
expect(isFromHost({ source: other, origin: HOST }, parent, HOST)).toBe(
false,
);
expect(
isFromHost(
{ source: parent, origin: "https://evil.example" },
parent,
HOST,
),
).toBe(false);
expect(isFromHost({ source: parent, origin: "null" }, parent, HOST)).toBe(
false,
);
});
});
describe("restrictToHost", () => {
it("replaces the listener and drops foreign messages", () => {
const seen: unknown[] = [];
const original = (ev: MessageEvent): void => {
seen.push(ev.data);
};
const transport = { handleMessage: original };
const listeners = new Set<EventListener>([original as EventListener]);
const inbound = {
addEventListener: (_t: string, l: EventListener): void => {
listeners.add(l);
},
removeEventListener: (_t: string, l: EventListener): void => {
listeners.delete(l);
},
} as unknown as Window;
restrictToHost(transport, HOST, parent, inbound);
expect(listeners.has(original as EventListener)).toBe(false);
expect(listeners.size).toBe(1);
const dispatch = (ev: Partial<MessageEvent>): void =>
listeners.forEach((l) => l(ev as unknown as Event));
dispatch({ source: other, origin: HOST, data: "spoof" });
dispatch({ source: parent, origin: "https://evil.example", data: "bad" });
dispatch({ source: parent, origin: HOST, data: "real" });
expect(seen).toEqual(["real"]);
});
});
+53
View File
@@ -0,0 +1,53 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
type MessageLike = Pick<MessageEvent, "source" | "origin">;
/** True when a message came from our parent window, at the host's origin. */
export function isFromHost(
ev: MessageLike,
parentWindow: Window,
parentOrigin: string,
): boolean {
return ev.source === parentWindow && ev.origin === parentOrigin;
}
interface InboundTransport {
handleMessage: (ev: MessageEvent) => void;
}
/**
* [cinny #43] Only handle widget messages sent by the host (our parent window)
* from the origin it was loaded at (`parentUrl`).
*
* matrix-widget-api's `strictOriginCheck` compares `ev.origin` with this
* frame's OWN origin, which only works while EC is served from the host's
* origin; served from its own origin (e.g. call.chat.lotusguild.org) every
* host message would be dropped. Turning the check off instead would let any
* window post toWidget actions (including io.lotus.*). This compares against
* the host's origin and also requires the sender to be our parent, so it is
* correct same-origin and cross-origin alike.
*
* Swaps the transport's `message` listener; safe whether or not the transport
* has started (`stop()` removes `handleMessage`, the guarded one after this).
*/
export function restrictToHost(
transport: unknown,
parentOrigin: string,
parentWindow: Window = window.parent,
inbound: Pick<Window, "addEventListener" | "removeEventListener"> = window,
): void {
const t = transport as InboundTransport;
const original = t.handleMessage;
const guarded = (ev: MessageEvent): void => {
if (!isFromHost(ev, parentWindow, parentOrigin)) return;
original(ev);
};
inbound.removeEventListener("message", original as EventListener);
t.handleMessage = guarded;
inbound.addEventListener("message", guarded as EventListener);
}
+7
View File
@@ -32,6 +32,9 @@ import { widget } from "../widget";
import { startLotusCallState } from "../lotus/lotusCallState";
import { startLotusFocus } from "../lotus/lotusFocus";
import { startLotusControls } from "../lotus/lotusControls";
import { startLotusHotkeys } from "../lotus/lotusHotkeys";
import { LotusFrameScreenshare } from "../lotus/LotusFrameScreenshare";
import { startLotusMicLevel } from "../lotus/lotusMicLevel";
import { startLotusAudioInject } from "../lotus/lotusAudioInject";
import { startLotusQuality } from "../lotus/lotusQuality";
import { startLotusDecorations } from "../lotus/lotusDecorations";
@@ -304,6 +307,9 @@ export const InCallView: FC<InCallViewProps> = ({
// [cinny #43] layout / settings / reactions over the widget API, plus a
// screensharing + layout report, replacing the host's DOM access.
useEffect(() => startLotusControls(vm), [vm]);
useEffect(() => startLotusHotkeys(), []);
// [cinny #146] Local mic level for the host's mute-button meter.
useEffect(() => startLotusMicLevel(vm), [vm]);
// [lotus] Handle the host's io.lotus.inject_audio action to mix a soundboard
// clip into the call as a separate track (#3). No-op unless the host sends it.
useEffect(() => startLotusAudioInject(vm), [vm]);
@@ -687,6 +693,7 @@ export const InCallView: FC<InCallViewProps> = ({
{earpieceOverlay}
<ReactionsOverlay vm={vm} />
{footer}
<LotusFrameScreenshare vm={vm} />
{showModals && (
<>
<RageshakeRequestModal {...rageshakeRequestModalProps} />
+71 -27
View File
@@ -5,7 +5,16 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { combineLatest, map, merge, of, Subject, switchMap } from "rxjs";
import {
combineLatest,
distinctUntilChanged,
map,
merge,
of,
skip,
Subject,
switchMap,
} from "rxjs";
import { type Behavior } from "./Behavior";
import { type ObservableScope } from "./ObservableScope";
@@ -25,7 +34,12 @@ export interface VolumeControls {
playbackMuted$: Behavior<boolean>;
togglePlaybackMuted: () => void;
adjustPlaybackVolume: (value: number) => void;
commitPlaybackVolume: () => void;
/**
* Commit the volume. [lotus #36] Pass the slider's committed value: with the
* keyboard the slider commits before its last change reaches us, which left
* the committed (and remembered) volume one step behind.
*/
commitPlaybackVolume: (value?: number) => void;
}
interface VolumeControlsInputs {
@@ -35,6 +49,10 @@ interface VolumeControlsInputs {
* requested volume.
*/
sink$: Behavior<(volume: number) => void>;
/** [lotus #36] Starting volume (a remembered one); defaults to 1. */
initialVolume?: number;
/** [lotus #36] Called with each newly committed volume. */
onCommit?: (volume: number) => void;
}
/**
@@ -43,38 +61,61 @@ interface VolumeControlsInputs {
*/
export function createVolumeControls(
scope: ObservableScope,
{ pretendToBeDisconnected$, sink$ }: VolumeControlsInputs,
{
pretendToBeDisconnected$,
sink$,
initialVolume = 1,
onCommit,
}: VolumeControlsInputs,
): VolumeControls {
const toggleMuted$ = new Subject<"toggle mute">();
const adjustVolume$ = new Subject<number>();
const commitVolume$ = new Subject<"commit">();
const playbackVolume$ = scope.behavior<number>(
const state$ = scope.behavior(
merge(toggleMuted$, adjustVolume$, commitVolume$).pipe(
accumulate({ volume: 1, committedVolume: 1 }, (state, event) => {
switch (event) {
case "toggle mute":
return {
...state,
volume: state.volume === 0 ? state.committedVolume : 0,
};
case "commit":
// Dragging the slider to zero should have the same effect as
// muting: keep the original committed volume, as if it were never
// dragged
return {
...state,
committedVolume:
state.volume === 0 ? state.committedVolume : state.volume,
};
default:
// Volume adjustment
return { ...state, volume: event };
}
}),
map(({ volume }) => volume),
accumulate(
{ volume: initialVolume, committedVolume: initialVolume },
(state, event) => {
switch (event) {
case "toggle mute":
return {
...state,
volume: state.volume === 0 ? state.committedVolume : 0,
};
case "commit":
// Dragging the slider to zero should have the same effect as
// muting: keep the original committed volume, as if it were never
// dragged
return {
...state,
committedVolume:
state.volume === 0 ? state.committedVolume : state.volume,
};
default:
// Volume adjustment
return { ...state, volume: event };
}
},
),
),
);
const playbackVolume$ = scope.behavior<number>(
state$.pipe(map(({ volume }) => volume)),
);
// [lotus #36] Report committed changes (not the starting value) so the
// caller can remember them.
if (onCommit) {
state$
.pipe(
map(({ committedVolume }) => committedVolume),
distinctUntilChanged(),
skip(1),
scope.bind(),
)
.subscribe(onCommit);
}
// Sync the requested volume with the audio playback module
combineLatest([
@@ -96,6 +137,9 @@ export function createVolumeControls(
),
togglePlaybackMuted: () => toggleMuted$.next("toggle mute"),
adjustPlaybackVolume: (value: number) => adjustVolume$.next(value),
commitPlaybackVolume: () => commitVolume$.next("commit"),
commitPlaybackVolume: (value?: number) => {
if (value !== undefined) adjustVolume$.next(value);
commitVolume$.next("commit");
},
};
}
+31 -1
View File
@@ -5,7 +5,7 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { expect, onTestFinished, test, vi } from "vitest";
import { afterEach, expect, onTestFinished, test, vi } from "vitest";
import {
type LocalTrackPublication,
LocalVideoTrack,
@@ -44,6 +44,36 @@ vi.mock("../../Platform", () => ({
const rtcMembership = mockRtcMembership("@alice:example.org", "AAAA");
// [lotus #36] Remote volumes are remembered in localStorage; keep tests apart.
afterEach(() => localStorage.clear());
test("a remembered volume is restored and a committed slider value is saved", () => {
localStorage.setItem(
"lotus-per-user-volume",
JSON.stringify({ "@alice:example.org": 0.5 }),
);
const setVolumeSpy = vi.fn();
const vm = mockRemoteMedia(
rtcMembership,
{},
mockRemoteParticipant({ setVolume: setVolumeSpy }),
);
withTestScheduler(({ expectObservable, schedule }) => {
schedule("-a|", {
a() {
// Keyboard order: the slider commits with its final value before that
// value's change event reaches the view model.
vm.commitPlaybackVolume(0.3);
expect(setVolumeSpy).toHaveBeenLastCalledWith(0.3);
expect(
JSON.parse(localStorage.getItem("lotus-per-user-volume") ?? "{}"),
).toEqual({ "@alice:example.org": 0.3 });
},
});
expectObservable(vm.playbackVolume$).toBe("ab", { a: 0.5, b: 0.3 });
});
});
test("control a participant's volume", () => {
const setVolumeSpy = vi.fn();
const vm = mockRemoteMedia(
@@ -11,6 +11,10 @@ import { combineLatest, map, of, switchMap } from "rxjs";
import { type Behavior } from "../Behavior";
import { createVolumeControls, type VolumeControls } from "../VolumeControls";
import {
getRememberedVolume,
rememberVolume,
} from "../../lotus/lotusVolumeMemory";
import {
type BaseUserMediaInputs,
type BaseUserMediaViewModel,
@@ -52,6 +56,9 @@ export function createRemoteUserMedia(
sink$: scope.behavior(
inputs.participant$.pipe(map((p) => (volume) => p?.setVolume(volume))),
),
// [lotus #36] The slider sticks per user across calls and reconnects.
initialVolume: getRememberedVolume(inputs.userId),
onCommit: (volume) => rememberVolume(inputs.userId, volume),
}),
local: false,
speaking$: scope.behavior(
+4 -1
View File
@@ -337,7 +337,10 @@ const ScreenShareVolumeButton: FC<ScreenShareVolumeButtonProps> = ({ vm }) => {
(v: number) => vm.adjustPlaybackVolume(v),
[vm],
);
const onVolumeCommit = useCallback(() => vm.commitPlaybackVolume(), [vm]);
const onVolumeCommit = useCallback(
(value: number) => vm.commitPlaybackVolume(value),
[vm],
);
return (
audioEnabled && (
+4
View File
@@ -14,6 +14,7 @@ import {
ReactionSet,
ReactionsRowSize,
} from "./reactions";
import { isLotusHostHotkey } from "./lotus/lotusHotkeys";
/**
* Determines whether focus is in the same part of the tree as the given
@@ -86,6 +87,8 @@ export function useCallViewKeyboardShortcuts(
useCallback(
(event: KeyboardEvent) => {
logger.info("Keydown event", event);
// [lotus] The Lotus host owns its PTT / deafen keys (cinny #43).
if (isLotusHostHotkey(event.code)) return;
if (!mayReceiveKeyEvents()) return;
if (event.altKey || event.ctrlKey || event.metaKey || event.shiftKey)
return;
@@ -132,6 +135,7 @@ export function useCallViewKeyboardShortcuts(
"keyup",
useCallback(
(event: KeyboardEvent) => {
if (isLotusHostHotkey(event.code)) return;
if (!mayReceiveKeyEvents() || !mayReceiveSpaceKeyEvents()) return;
if (event.key === " ") {
spacebarHeld.current = false;
+11 -1
View File
@@ -76,7 +76,17 @@ export const useTheme = (): void => {
"lotusTheme anyway since the two flags are only meaningful together.",
);
}
if (lotusTheme || lotusTransparent)
if (lotusTheme || lotusTransparent) {
document.body.classList.add("lotus-theme");
// [cinny #43] The frame's root colour scheme must match the host's, or
// the browser paints an opaque backdrop behind the transparent frame.
// (The host used to inject `:root { color-scheme }` for this.)
document.documentElement.style.colorScheme = theme;
}
// [cinny #43] The host draws its own call bar: hide EC's footer. Replaces
// the host's injected styles. Where the host can't start a share for us,
// LotusFrameScreenshare shows EC's screenshare button on its own.
if (lotusFlag("lotusHostControls"))
document.body.classList.add("lotus-host-controls");
}, [previousTheme, requestedTheme]);
};
+6 -10
View File
@@ -24,6 +24,7 @@ import { Config } from "./config/Config";
import { seedSettingsFromConfig } from "./settings/settings";
import { ElementCallReactionEventType } from "./reactions";
import { LOTUS_TO_WIDGET_ACTIONS } from "./lotus/lotusActions";
import { restrictToHost } from "./lotus/lotusWidgetOrigin";
// Subset of the actions in element-web
export enum ElementWidgetActions {
@@ -94,16 +95,11 @@ export const initializeWidget = (
const parentOrigin = new URL(parentUrl).origin;
logger.info("Widget API is available");
const api = new WidgetApi(widgetId, parentOrigin);
// [lotus] matrix-widget-api's PostmessageTransport defaults
// strictOriginCheck to false, which would let any frame holding a
// handle to our window post toWidget actions (including the
// io.lotus.* actions below). The Lotus deployment serves EC
// same-origin with the host (cinny loads /public/element-call/index.html),
// so globalThis.origin === parentOrigin and this check passes safely.
// A cross-origin deployment would need to compare ev.origin to
// parentOrigin instead, since strictOriginCheck compares against
// globalThis.origin.
api.transport.strictOriginCheck = true;
// [lotus] Only messages from the host (our parent, at parentUrl's
// origin) reach the widget API. matrix-widget-api's own
// strictOriginCheck compares against THIS frame's origin, so it only
// worked while EC was served from the host's origin (cinny #43).
restrictToHost(api.transport, parentOrigin);
api.requestCapability(MatrixCapabilities.AlwaysOnScreen);
api.requestCapability(MatrixCapabilities.MSC4039DownloadFile);