Compare commits
18
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fbf8e44276 | ||
|
|
5693e81bb9 | ||
|
|
5f4844cad5 | ||
|
|
848dcaafa9 | ||
|
|
1ad32e56c7 | ||
|
|
090edc5c2c | ||
|
|
dbf77cdc60 | ||
|
|
2ef80cef2c | ||
|
|
88c648ba8b | ||
|
|
0aa543cc7d | ||
|
|
cc57292bf4 | ||
|
|
22f9083768 | ||
|
|
d7d75cbba5 | ||
|
|
4be3e92eb6 | ||
|
|
a1dd0e254c | ||
|
|
888ed1a66c | ||
|
|
73bf2133dc | ||
|
|
07e1506dd8 |
@@ -33,33 +33,35 @@ The host sets these on the widget URL (cinny: `CallEmbed.getWidget`).
|
|||||||
The table is generated from `src/lotus/lotusActions.ts`; the last column was
|
The table is generated from `src/lotus/lotusActions.ts`; the last column was
|
||||||
checked against cinny's `lotus` branch.
|
checked against cinny's `lotus` branch.
|
||||||
|
|
||||||
| Action | Direction | Payload / meaning | Handled by cinny (`lotus`) |
|
| Action | Direction | Payload / meaning | Handled by cinny (`lotus`) |
|
||||||
| -------------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- |
|
| ---------------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- |
|
||||||
| `io.lotus.call_state` | call → host | in-call per-participant speaking / mute state. NOTE: matrix-widget-api `transport.send` is request/response — the host MUST register a handler that replies/acks each one (cinny's `listenAction` does, replying `{}`). If the host has no handler, ClientWidgetApi immediately error-replies "unsupported from-widget action", so the data is silently dropped and each throttled send rejects (caught) — functional miss + log churn, not a hang. | yes |
|
| `io.lotus.call_state` | call → host | in-call per-participant speaking / mute state. NOTE: matrix-widget-api `transport.send` is request/response — the host MUST register a handler that replies/acks each one (cinny's `listenAction` does, replying `{}`). If the host has no handler, ClientWidgetApi immediately error-replies "unsupported from-widget action", so the data is silently dropped and each throttled send rejects (caught) — functional miss + log churn, not a hang. | yes |
|
||||||
| `io.lotus.focus_participant` | host → call | pin/spotlight (or clear, with userId=null) a participant. | yes |
|
| `io.lotus.focus_participant` | host → call | pin/spotlight (or clear, with userId=null) a participant. | yes |
|
||||||
| `io.lotus.inject_audio` | host → call | play an audio clip into the call as a separate published track. | yes |
|
| `io.lotus.inject_audio` | host → call | play an audio clip into the call as a separate published track. | yes |
|
||||||
| `io.lotus.set_quality` | host → call | set audio/screenshare encoding quality (bitrate/framerate). | yes |
|
| `io.lotus.set_quality` | host → call | set audio/screenshare encoding quality (bitrate/framerate). | yes |
|
||||||
| `io.lotus.decorations` | host → call | per-user avatar-decoration image URLs for in-call tiles. | yes |
|
| `io.lotus.decorations` | host → call | per-user avatar-decoration image URLs for in-call tiles. | yes |
|
||||||
| `io.lotus.set_deafen` | host → call | deafen remote audio (and optionally mute screenshare audio). | yes |
|
| `io.lotus.set_deafen` | host → call | deafen remote audio (and optionally mute screenshare audio). | yes |
|
||||||
| `io.lotus.request_state` | call → host | sent on (re)registration of a lotus toWidget handler (e.g. decorations) so the host can re-push state that would otherwise only be sent on change (decorations roster, the current focus pin) after an EC-side reconnect/remount. cinny's `resendForkState()` should respond to this the same way it responds to a fresh join. | yes |
|
| `io.lotus.request_state` | call → host | sent on (re)registration of a lotus toWidget handler (e.g. decorations) so the host can re-push state that would otherwise only be sent on change (decorations roster, the current focus pin) after an EC-side reconnect/remount. cinny's `resendForkState()` should respond to this the same way it responds to a fresh join. | yes |
|
||||||
| `io.lotus.denoise_state` | call → host | real state of the in-source denoise engine — `{ active: boolean, model: string, error?: string }` — sent once the processor attaches (or after the rnnoise fallback also fails), so the host toggle can reflect reality rather than the requested state. | **no** |
|
| `io.lotus.denoise_state` | call → host | real state of the in-source denoise engine — `{ active: boolean, model: string, error?: string }` — sent once the processor attaches (or after the rnnoise fallback also fails), so the host toggle can reflect reality rather than the requested state. | yes |
|
||||||
| `io.lotus.call_summary` | call → host | one-shot end-of-call readout for the local participant — `{ durationMs, reconnects, poorMs, verdict }` (#143) — sent on SFU disconnect or in-call teardown, whichever comes first. | yes |
|
| `io.lotus.call_summary` | call → host | one-shot end-of-call readout for the local participant — `{ durationMs, reconnects, poorMs, verdict }` (#143) — sent on SFU disconnect or in-call teardown, whichever comes first. | yes |
|
||||||
| `io.lotus.set_audio_output` | host → call | select the audio output device `{ deviceId }` (#119). | yes |
|
| `io.lotus.set_audio_output` | host → call | select the audio output device `{ deviceId }` (#119). | yes |
|
||||||
| `io.lotus.audio_output_state` | call → host | the currently selected output `{ deviceId }` (#119). | **no** |
|
| `io.lotus.audio_output_state` | call → host | the currently selected output `{ deviceId }` (#119). | cinny #417 |
|
||||||
| `io.lotus.screenshare_notice` | call → host | local screenshare reminder `{ kind: "ended" \| "no-frames" \| "alone" }` — window closed, no frames for 15 s, or 30 min of sharing with nobody else in the call (#39). Each fires at most once per share. | yes |
|
| `io.lotus.screenshare_audio_state` | call → host | `{ present }`: whether any remote screenshare audio is being received, sent on change (cinny #277). | cinny #277 |
|
||||||
| `io.lotus.set_layout` | host → call | switch the call layout `{ layout: "grid" \| "spotlight" }` (cinny #43 — replaces the host clicking EC's hidden layout radio). | yes |
|
| `io.lotus.screenshare_notice` | call → host | local screenshare reminder `{ kind: "ended" \| "no-frames" \| "alone" }` — window closed, no frames for 15 s, or 30 min of sharing with nobody else in the call (#39). Each fires at most once per share. | yes |
|
||||||
| `io.lotus.open_settings` | host → call | open EC's settings modal, or close it with `{ open: false }`; omit `open` to toggle (cinny #43). | yes |
|
| `io.lotus.set_layout` | host → call | switch the call layout `{ layout: "grid" \| "spotlight" }` (cinny #43 — replaces the host clicking EC's hidden layout radio). | yes |
|
||||||
| `io.lotus.toggle_reactions` | host → call | toggle the reactions / raise-hand menu (cinny #43). | yes |
|
| `io.lotus.open_settings` | host → call | open EC's settings modal, or close it with `{ open: false }`; omit `open` to toggle (cinny #43). | yes |
|
||||||
| `io.lotus.set_screenshare` | host → call | start/stop sharing `{ on?: boolean }` (omit to toggle; cinny #43). Starting calls getDisplayMedia, which needs the user's click: the host sends this with Capability Delegation (`delegate: "display-capture"`), and without it the browser rejects the share. | yes |
|
| `io.lotus.toggle_reactions` | host → call | toggle the reactions / raise-hand menu (cinny #43). | yes |
|
||||||
| `io.lotus.set_hotkeys` | host → call | `{ codes: string[] }` — KeyboardEvent.code values (PTT, deafen) to report while focus is inside this frame (cinny #43). `[]` stops. | yes |
|
| `io.lotus.set_screenshare` | host → call | start/stop sharing `{ on?: boolean }` (omit to toggle; cinny #43). Starting calls getDisplayMedia, which needs the user's click: the host sends this with Capability Delegation (`delegate: "display-capture"`), and without it the browser rejects the share. | yes |
|
||||||
| `io.lotus.set_frame_screenshare` | host → call | `{ visible: boolean }` — whether the in-frame screenshare button (`lotusFrameScreenshare`) may show, e.g. false while the room's call policy forbids screensharing (cinny #43). Defaults to visible. | yes |
|
| `io.lotus.set_hotkeys` | host → call | `{ codes: string[] }` — KeyboardEvent.code values (PTT, deafen) to report while focus is inside this frame (cinny #43). `[]` stops. | yes |
|
||||||
| `io.lotus.prompt_screenshare` | host → call | `{}` — show the "Share your screen?" prompt inside this frame (cinny #43). For hosts that can't delegate the user's click: the prompt's Share button is clicked here, so getDisplayMedia gets its activation. | yes |
|
| `io.lotus.set_frame_screenshare` | host → call | `{ visible: boolean }` — whether the in-frame screenshare button (`lotusFrameScreenshare`) may show, e.g. false while the room's call policy forbids screensharing (cinny #43). Defaults to visible. | yes |
|
||||||
| `io.lotus.hotkey` | call → host | a watched key went down/up here, or this window's focus changed; see `LotusHotkeyReport` (cinny #43). | yes |
|
| `io.lotus.prompt_screenshare` | host → call | `{}` — show the "Share your screen?" prompt inside this frame (cinny #43). For hosts that can't delegate the user's click: the prompt's Share button is clicked here, so getDisplayMedia gets its activation. | yes |
|
||||||
| `io.lotus.controls_state` | call → host | `{ screensharing: boolean, layout: "grid" \| "spotlight" \| null, screenshareAction: true, hotkeys: true, frameScreenshare: false, screensharePrompt: boolean, screensharePromptOpen: boolean }` whenever either changes (cinny #43), so the host stops reading EC's DOM for them. Its arrival also tells the host this fork supports the actions above. | yes |
|
| `io.lotus.hotkey` | call → host | a watched key went down/up here, or this window's focus changed; see `LotusHotkeyReport` (cinny #43). | yes |
|
||||||
| `io.lotus.mic_level` | call → host | local mic level `{ bars: 0 \| 1 \| 2 \| 3 }` (cinny #146), sent only when it changes (≤ 10 Hz); 0 while muted or with no mic. | yes |
|
| `io.lotus.controls_state` | call → host | `{ screensharing: boolean, layout: "grid" \| "spotlight" \| null, screenshareAction: true, hotkeys: true, frameScreenshare: false, screensharePrompt: boolean, screensharePromptOpen: boolean }` whenever either changes (cinny #43), so the host stops reading EC's DOM for them. Its arrival also tells the host this fork supports the actions above. | yes |
|
||||||
|
| `io.lotus.mic_level` | call → host | local mic level `{ bars: 0 \| 1 \| 2 \| 3 }` (cinny #146), sent only when it changes (≤ 10 Hz); 0 while muted or with no mic. | yes |
|
||||||
|
| `io.lotus.call_error` | call → host | the call's error page is up (`{ code, recoverable }`) or cleared (`{ code: null }`) (cinny #318), so the host stops showing the call as joined after "Connection lost". | cinny #418 |
|
||||||
|
| `io.lotus.reconnect` | host → call | press the error page's Reconnect / Try again (cinny #318). | cinny #418 |
|
||||||
|
|
||||||
The two unhandled actions are covered by pending cinny PRs (#413 for
|
Entries naming a cinny PR are handled once that PR merges.
|
||||||
`denoise_state`; the #310 change for `audio_output_state`).
|
|
||||||
|
|
||||||
## Fork-only behaviour (no flag needed)
|
## Fork-only behaviour (no flag needed)
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "@lotusguild/element-call-embedded",
|
"name": "@lotusguild/element-call-embedded",
|
||||||
"version": "0.25.0-lotus.24",
|
"version": "0.25.0-lotus.26",
|
||||||
"files": [
|
"files": [
|
||||||
"README.md",
|
"README.md",
|
||||||
"LICENSE-AGPL-3.0",
|
"LICENSE-AGPL-3.0",
|
||||||
|
|||||||
@@ -19,6 +19,10 @@ import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
|
|||||||
import { useEarpieceAudioConfig } from "../MediaDevicesContext";
|
import { useEarpieceAudioConfig } from "../MediaDevicesContext";
|
||||||
import { useReactiveState } from "../useReactiveState";
|
import { useReactiveState } from "../useReactiveState";
|
||||||
import * as controls from "../controls";
|
import * as controls from "../controls";
|
||||||
|
import {
|
||||||
|
clearScreenshareAudio,
|
||||||
|
reportScreenshareAudio,
|
||||||
|
} from "../lotus/lotusScreenshareAudio";
|
||||||
|
|
||||||
export interface MatrixAudioRendererProps {
|
export interface MatrixAudioRendererProps {
|
||||||
/**
|
/**
|
||||||
@@ -97,6 +101,18 @@ export function LivekitRoomAudioRenderer({
|
|||||||
return true;
|
return true;
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// [lotus / cinny #277] Tell the host whether remote screenshare audio is
|
||||||
|
// playing, so its "Mute Screenshare Audio" control only shows when useful.
|
||||||
|
const hasScreenshareAudio = tracks.some(
|
||||||
|
(ref) =>
|
||||||
|
!ref.participant.isLocal &&
|
||||||
|
ref.publication.source === Track.Source.ScreenShareAudio,
|
||||||
|
);
|
||||||
|
useEffect(() => {
|
||||||
|
reportScreenshareAudio(url, hasScreenshareAudio);
|
||||||
|
}, [url, hasScreenshareAudio]);
|
||||||
|
useEffect(() => (): void => clearScreenshareAudio(url), [url]);
|
||||||
|
|
||||||
// This component is also (in addition to the "only play audio for connected members" logic above)
|
// This component is also (in addition to the "only play audio for connected members" logic above)
|
||||||
// responsible for mimicking earpiece audio on iPhones.
|
// responsible for mimicking earpiece audio on iPhones.
|
||||||
// The Safari audio devices enumeration does not expose an earpiece audio device.
|
// The Safari audio devices enumeration does not expose an earpiece audio device.
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ describe("LotusWidgetActions", () => {
|
|||||||
LotusWidgetActions.SetHotkeys,
|
LotusWidgetActions.SetHotkeys,
|
||||||
LotusWidgetActions.SetFrameScreenshare,
|
LotusWidgetActions.SetFrameScreenshare,
|
||||||
LotusWidgetActions.PromptScreenshare,
|
LotusWidgetActions.PromptScreenshare,
|
||||||
|
LotusWidgetActions.Reconnect,
|
||||||
];
|
];
|
||||||
|
|
||||||
expect(new Set(LOTUS_TO_WIDGET_ACTIONS)).toEqual(new Set(expectedToWidget));
|
expect(new Set(LOTUS_TO_WIDGET_ACTIONS)).toEqual(new Set(expectedToWidget));
|
||||||
@@ -56,5 +57,6 @@ describe("LotusWidgetActions", () => {
|
|||||||
);
|
);
|
||||||
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.MicLevel);
|
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.MicLevel);
|
||||||
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.Hotkey);
|
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.Hotkey);
|
||||||
|
expect(LOTUS_TO_WIDGET_ACTIONS).not.toContain(LotusWidgetActions.CallError);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -56,6 +56,12 @@ export enum LotusWidgetActions {
|
|||||||
SetAudioOutput = "io.lotus.set_audio_output",
|
SetAudioOutput = "io.lotus.set_audio_output",
|
||||||
/** fromWidget: the currently selected output `{ deviceId }` (#119). */
|
/** fromWidget: the currently selected output `{ deviceId }` (#119). */
|
||||||
AudioOutputState = "io.lotus.audio_output_state",
|
AudioOutputState = "io.lotus.audio_output_state",
|
||||||
|
/**
|
||||||
|
* fromWidget: `{ present: boolean }`, whether any remote screenshare audio
|
||||||
|
* is being received, so the host's "Mute Screenshare Audio" control only
|
||||||
|
* shows when there is something to mute (cinny #277). Sent on change.
|
||||||
|
*/
|
||||||
|
ScreenshareAudioState = "io.lotus.screenshare_audio_state",
|
||||||
/**
|
/**
|
||||||
* fromWidget: local screenshare reminder `{ kind: "ended" | "no-frames" | "alone" }`
|
* fromWidget: local screenshare reminder `{ kind: "ended" | "no-frames" | "alone" }`
|
||||||
* — window closed, no frames for 15 s, or 30 min of sharing with nobody
|
* — window closed, no frames for 15 s, or 30 min of sharing with nobody
|
||||||
@@ -116,6 +122,14 @@ export enum LotusWidgetActions {
|
|||||||
* only when it changes (≤ 10 Hz); 0 while muted or with no mic.
|
* only when it changes (≤ 10 Hz); 0 while muted or with no mic.
|
||||||
*/
|
*/
|
||||||
MicLevel = "io.lotus.mic_level",
|
MicLevel = "io.lotus.mic_level",
|
||||||
|
/**
|
||||||
|
* fromWidget: the call's error page is up (`{ code, recoverable }`) or
|
||||||
|
* cleared (`{ code: null }`) (cinny #318), so the host stops showing the call
|
||||||
|
* as joined after "Connection lost".
|
||||||
|
*/
|
||||||
|
CallError = "io.lotus.call_error",
|
||||||
|
/** toWidget: press the error page's Reconnect / Try again (cinny #318). */
|
||||||
|
Reconnect = "io.lotus.reconnect",
|
||||||
}
|
}
|
||||||
|
|
||||||
/** toWidget Lotus actions that must be allow-listed in `initializeWidget`. */
|
/** toWidget Lotus actions that must be allow-listed in `initializeWidget`. */
|
||||||
@@ -133,4 +147,5 @@ export const LOTUS_TO_WIDGET_ACTIONS: LotusWidgetActions[] = [
|
|||||||
LotusWidgetActions.SetHotkeys,
|
LotusWidgetActions.SetHotkeys,
|
||||||
LotusWidgetActions.SetFrameScreenshare,
|
LotusWidgetActions.SetFrameScreenshare,
|
||||||
LotusWidgetActions.PromptScreenshare,
|
LotusWidgetActions.PromptScreenshare,
|
||||||
|
LotusWidgetActions.Reconnect,
|
||||||
];
|
];
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import { of } from "rxjs";
|
|||||||
|
|
||||||
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
|
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
|
||||||
import {
|
import {
|
||||||
|
canPublishSoundboard,
|
||||||
MAX_INJECT_BYTES,
|
MAX_INJECT_BYTES,
|
||||||
parseInjectSource,
|
parseInjectSource,
|
||||||
startLotusAudioInject,
|
startLotusAudioInject,
|
||||||
@@ -258,3 +259,17 @@ test("parseInjectSource rejects oversized or non-buffer audio", () => {
|
|||||||
expect(parseInjectSource({ audio: "not bytes" })).toBeNull();
|
expect(parseInjectSource({ audio: "not bytes" })).toBeNull();
|
||||||
expect(parseInjectSource({ audio: new Uint8Array(4) })).toBeNull();
|
expect(parseInjectSource({ audio: new Uint8Array(4) })).toBeNull();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("canPublishSoundboard: refused by a microphone-only grant (EC #47)", () => {
|
||||||
|
expect(canPublishSoundboard(undefined)).toBe(true);
|
||||||
|
expect(
|
||||||
|
canPublishSoundboard({ canPublish: true, canPublishSources: [] }),
|
||||||
|
).toBe(true);
|
||||||
|
expect(
|
||||||
|
canPublishSoundboard({ canPublish: true, canPublishSources: [2] }),
|
||||||
|
).toBe(false);
|
||||||
|
expect(
|
||||||
|
canPublishSoundboard({ canPublish: true, canPublishSources: [2, 0] }),
|
||||||
|
).toBe(true);
|
||||||
|
expect(canPublishSoundboard({ canPublish: false })).toBe(false);
|
||||||
|
});
|
||||||
|
|||||||
@@ -60,6 +60,22 @@ function acquireSharedAudio(): {
|
|||||||
* No effect unless the host sends the action. Returns a teardown function that
|
* No effect unless the host sends the action. Returns a teardown function that
|
||||||
* also aborts any clip still playing.
|
* also aborts any clip still playing.
|
||||||
*/
|
*/
|
||||||
|
/**
|
||||||
|
* Pure: whether these LiveKit permissions allow publishing the soundboard's
|
||||||
|
* Unknown-source track (EC #47). No permissions or an empty source list
|
||||||
|
* means unrestricted; otherwise UNKNOWN (TrackSource 0) must be allowed.
|
||||||
|
*/
|
||||||
|
export function canPublishSoundboard(
|
||||||
|
perm:
|
||||||
|
| { canPublish?: boolean; canPublishSources?: readonly number[] }
|
||||||
|
| undefined,
|
||||||
|
): boolean {
|
||||||
|
if (!perm) return true;
|
||||||
|
if (perm.canPublish === false) return false;
|
||||||
|
const sources = perm.canPublishSources ?? [];
|
||||||
|
return sources.length === 0 || sources.includes(0);
|
||||||
|
}
|
||||||
|
|
||||||
export function startLotusAudioInject(vm: CallViewModel): () => void {
|
export function startLotusAudioInject(vm: CallViewModel): () => void {
|
||||||
const w = widget;
|
const w = widget;
|
||||||
if (!w) return () => undefined;
|
if (!w) return () => undefined;
|
||||||
@@ -116,6 +132,19 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// [EC #47] Clips publish as an Unknown-source track. Audio-only rooms get
|
||||||
|
// a token limited to the microphone source, where LiveKit refuses that
|
||||||
|
// publish *after* we replied; say so up front instead of failing silently.
|
||||||
|
if (
|
||||||
|
rooms.some((r) => !canPublishSoundboard(r.localParticipant.permissions))
|
||||||
|
) {
|
||||||
|
w.api.transport.reply(ev.detail, {
|
||||||
|
played: false,
|
||||||
|
reason: "not_permitted",
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
w.api.transport.reply(ev.detail, {});
|
w.api.transport.reply(ev.detail, {});
|
||||||
void playInjectedClip(clip, volume, rooms, activeClips).catch((e) =>
|
void playInjectedClip(clip, volume, rooms, activeClips).catch((e) =>
|
||||||
logger.warn("[lotus] inject_audio failed", e),
|
logger.warn("[lotus] inject_audio failed", e),
|
||||||
|
|||||||
@@ -83,6 +83,36 @@ test("[lotus #28] safeImageUrl only accepts the pinned decoration CDN origin (or
|
|||||||
stop();
|
stop();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test("[matrix #22] both the new decoration host and the old Nextcloud one are accepted", () => {
|
||||||
|
const stop = startLotusDecorations();
|
||||||
|
const { result } = renderHook(() => useLotusDecoration("@alice:example.org"));
|
||||||
|
|
||||||
|
pushDecorations({
|
||||||
|
"@alice:example.org":
|
||||||
|
"https://chat.lotusguild.org/decorations/joystick.png",
|
||||||
|
});
|
||||||
|
expect(result.current).toBe(
|
||||||
|
"https://chat.lotusguild.org/decorations/joystick.png",
|
||||||
|
);
|
||||||
|
|
||||||
|
pushDecorations({
|
||||||
|
"@alice:example.org":
|
||||||
|
"https://drive.lotusguild.org/public.php/dav/files/x/cinny-decorations/joystick.png",
|
||||||
|
});
|
||||||
|
expect(result.current).toBe(
|
||||||
|
"https://drive.lotusguild.org/public.php/dav/files/x/cinny-decorations/joystick.png",
|
||||||
|
);
|
||||||
|
|
||||||
|
// A look-alike subdomain is a different origin.
|
||||||
|
pushDecorations({
|
||||||
|
"@alice:example.org":
|
||||||
|
"https://chat.lotusguild.org.evil.example/joystick.png",
|
||||||
|
});
|
||||||
|
expect(result.current).toBeUndefined();
|
||||||
|
|
||||||
|
stop();
|
||||||
|
});
|
||||||
|
|
||||||
test("[lotus #17] the roster survives a handler remount within the same page session", () => {
|
test("[lotus #17] the roster survives a handler remount within the same page session", () => {
|
||||||
const stop1 = startLotusDecorations();
|
const stop1 = startLotusDecorations();
|
||||||
pushDecorations({
|
pushDecorations({
|
||||||
|
|||||||
@@ -45,7 +45,11 @@ export function useLotusDecoration(userId: string): string | undefined {
|
|||||||
// to the actual decoration CDN origin(s) — cinny's `DECORATION_CDN`
|
// to the actual decoration CDN origin(s) — cinny's `DECORATION_CDN`
|
||||||
// (`avatarDecorations.ts`) — plus `blob:`, which EC itself may use for local
|
// (`avatarDecorations.ts`) — plus `blob:`, which EC itself may use for local
|
||||||
// previews. Exported so a future CDN move is a one-line edit here.
|
// previews. Exported so a future CDN move is a one-line edit here.
|
||||||
|
//
|
||||||
|
// matrix #22: decorations moved off Nextcloud to chat.lotusguild.org/decorations
|
||||||
|
// (first entry). drive.lotusguild.org stays until no host build points there.
|
||||||
export const ALLOWED_DECORATION_ORIGINS: readonly string[] = [
|
export const ALLOWED_DECORATION_ORIGINS: readonly string[] = [
|
||||||
|
"https://chat.lotusguild.org",
|
||||||
"https://drive.lotusguild.org",
|
"https://drive.lotusguild.org",
|
||||||
];
|
];
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ Please see LICENSE in the repository root for full details.
|
|||||||
|
|
||||||
import { describe, expect, it } from "vitest";
|
import { describe, expect, it } from "vitest";
|
||||||
|
|
||||||
import { MicLevelQuantizer } from "./lotusMicLevel";
|
import { MicLevelQuantizer, sameRooms } from "./lotusMicLevel";
|
||||||
|
|
||||||
describe("MicLevelQuantizer", () => {
|
describe("MicLevelQuantizer", () => {
|
||||||
it("maps RMS to 0–3 bars", () => {
|
it("maps RMS to 0–3 bars", () => {
|
||||||
@@ -35,3 +35,14 @@ describe("MicLevelQuantizer", () => {
|
|||||||
expect(q.push(0)).toBe(0);
|
expect(q.push(0)).toBe(0);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe("sameRooms (EC #45)", () => {
|
||||||
|
it("compares rooms by identity and order", () => {
|
||||||
|
const a = {};
|
||||||
|
const b = {};
|
||||||
|
expect(sameRooms([a, b], [a, b])).toBe(true);
|
||||||
|
expect(sameRooms([a], [a, b])).toBe(false);
|
||||||
|
expect(sameRooms([a, b], [b, a])).toBe(false);
|
||||||
|
expect(sameRooms([{}], [{}])).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -46,6 +46,10 @@ const micPublication = (
|
|||||||
: null;
|
: null;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/** Pure: the same LiveKit rooms, in the same order (by identity). */
|
||||||
|
export const sameRooms = <T>(a: readonly T[], b: readonly T[]): boolean =>
|
||||||
|
a.length === b.length && a.every((r, i) => r === b[i]);
|
||||||
|
|
||||||
const samplers = new WeakMap<
|
const samplers = new WeakMap<
|
||||||
CallViewModel,
|
CallViewModel,
|
||||||
Observable<LocalMicSample | null>
|
Observable<LocalMicSample | null>
|
||||||
@@ -62,10 +66,15 @@ export function observeLocalMicSample$(
|
|||||||
const cached = samplers.get(vm);
|
const cached = samplers.get(vm);
|
||||||
if (cached) return cached;
|
if (cached) return cached;
|
||||||
const sampler$ = vm.allConnections$.pipe(
|
const sampler$ = vm.allConnections$.pipe(
|
||||||
|
map((data) => data.getConnections().map((c) => c.livekitRoom)),
|
||||||
|
// [EC #45] allConnections$ re-emits far more often than the set of
|
||||||
|
// LiveKit rooms changes; restarting the tap each time closed the
|
||||||
|
// AudioContext ~1×/s and emitted null, which reset the muted-speech
|
||||||
|
// gate so the "You're muted" nudge could never fire.
|
||||||
|
distinctUntilChanged(sameRooms),
|
||||||
switchMap(
|
switchMap(
|
||||||
(data) =>
|
(rooms) =>
|
||||||
new Observable<LocalMicSample | null>((subscriber) => {
|
new Observable<LocalMicSample | null>((subscriber) => {
|
||||||
const rooms = data.getConnections().map((c) => c.livekitRoom);
|
|
||||||
let ctx: AudioContext | null = null;
|
let ctx: AudioContext | null = null;
|
||||||
let clone: MediaStreamTrack | null = null;
|
let clone: MediaStreamTrack | null = null;
|
||||||
let timer: ReturnType<typeof setInterval> | undefined;
|
let timer: ReturnType<typeof setInterval> | undefined;
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
/*
|
||||||
|
Copyright 2026 Lotus Guild
|
||||||
|
|
||||||
|
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
|
||||||
|
Please see LICENSE in the repository root for full details.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { beforeEach, expect, test, vi } from "vitest";
|
||||||
|
|
||||||
|
import { LotusWidgetActions } from "./lotusActions";
|
||||||
|
import {
|
||||||
|
clearScreenshareAudio,
|
||||||
|
reportScreenshareAudio,
|
||||||
|
resetScreenshareAudioPresence,
|
||||||
|
} from "./lotusScreenshareAudio";
|
||||||
|
|
||||||
|
const send = vi.fn().mockResolvedValue({});
|
||||||
|
|
||||||
|
vi.mock("../widget", () => ({
|
||||||
|
widget: {
|
||||||
|
api: { transport: { send: (...a: unknown[]): unknown => send(...a) } },
|
||||||
|
},
|
||||||
|
}));
|
||||||
|
|
||||||
|
const sent = (): unknown[] =>
|
||||||
|
send.mock.calls
|
||||||
|
.filter(([action]) => action === LotusWidgetActions.ScreenshareAudioState)
|
||||||
|
.map(([, data]) => data);
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
resetScreenshareAudioPresence();
|
||||||
|
send.mockClear();
|
||||||
|
});
|
||||||
|
|
||||||
|
test("[cinny #277] reports presence only when the combined answer changes", () => {
|
||||||
|
reportScreenshareAudio("sfu-a", false);
|
||||||
|
reportScreenshareAudio("sfu-a", false);
|
||||||
|
reportScreenshareAudio("sfu-a", true);
|
||||||
|
reportScreenshareAudio("sfu-a", true);
|
||||||
|
reportScreenshareAudio("sfu-a", false);
|
||||||
|
expect(sent()).toEqual([
|
||||||
|
{ present: false },
|
||||||
|
{ present: true },
|
||||||
|
{ present: false },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("[cinny #277] any room with screenshare audio counts as present", () => {
|
||||||
|
reportScreenshareAudio("sfu-a", true);
|
||||||
|
reportScreenshareAudio("sfu-b", false);
|
||||||
|
expect(sent()).toEqual([{ present: true }]);
|
||||||
|
// A share is still playing in sfu-b after sfu-a's stops.
|
||||||
|
reportScreenshareAudio("sfu-b", true);
|
||||||
|
reportScreenshareAudio("sfu-a", false);
|
||||||
|
expect(sent()).toEqual([{ present: true }]);
|
||||||
|
// The renderer for sfu-b unmounts (call left): nothing is playing.
|
||||||
|
clearScreenshareAudio("sfu-b");
|
||||||
|
expect(sent()).toEqual([{ present: true }, { present: false }]);
|
||||||
|
});
|
||||||
@@ -8,6 +8,8 @@ Please see LICENSE in the repository root for full details.
|
|||||||
import { BehaviorSubject } from "rxjs";
|
import { BehaviorSubject } from "rxjs";
|
||||||
|
|
||||||
import { globalScope } from "../state/ObservableScope";
|
import { globalScope } from "../state/ObservableScope";
|
||||||
|
import { LotusWidgetActions } from "./lotusActions";
|
||||||
|
import { lotusSendToHost } from "./lotusWidget";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Whether the host has muted remote screenshare AUDIO (`io.lotus.set_deafen`
|
* Whether the host has muted remote screenshare AUDIO (`io.lotus.set_deafen`
|
||||||
@@ -25,3 +27,40 @@ export const setScreenshareAudioMuted$ = new BehaviorSubject(false);
|
|||||||
export const muteScreenshareAudio$ = globalScope.behavior(
|
export const muteScreenshareAudio$ = globalScope.behavior(
|
||||||
setScreenshareAudioMuted$,
|
setScreenshareAudioMuted$,
|
||||||
);
|
);
|
||||||
|
|
||||||
|
/**
|
||||||
|
* [cinny #277] Whether any remote participant's screenshare AUDIO is being
|
||||||
|
* received, so the host shows its "Mute Screenshare Audio" control only when
|
||||||
|
* there is something to mute. Each `LivekitRoomAudioRenderer` (one per SFU)
|
||||||
|
* reports for its own room; the host gets `io.lotus.screenshare_audio_state
|
||||||
|
* { present }` whenever the combined answer changes.
|
||||||
|
*/
|
||||||
|
const presentByRoom = new Map<string, boolean>();
|
||||||
|
let lastSent: boolean | undefined;
|
||||||
|
|
||||||
|
function sendIfChanged(): void {
|
||||||
|
const present = [...presentByRoom.values()].some(Boolean);
|
||||||
|
if (present === lastSent) return;
|
||||||
|
lastSent = present;
|
||||||
|
lotusSendToHost(LotusWidgetActions.ScreenshareAudioState, { present });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function reportScreenshareAudio(
|
||||||
|
roomKey: string,
|
||||||
|
present: boolean,
|
||||||
|
): void {
|
||||||
|
presentByRoom.set(roomKey, present);
|
||||||
|
sendIfChanged();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The renderer for `roomKey` went away (call left, SFU switched). */
|
||||||
|
export function clearScreenshareAudio(roomKey: string): void {
|
||||||
|
presentByRoom.delete(roomKey);
|
||||||
|
sendIfChanged();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Test-only: forget all reports. */
|
||||||
|
export function resetScreenshareAudioPresence(): void {
|
||||||
|
presentByRoom.clear();
|
||||||
|
lastSent = undefined;
|
||||||
|
}
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import {
|
|||||||
type ReactNode,
|
type ReactNode,
|
||||||
type SVGAttributes,
|
type SVGAttributes,
|
||||||
useCallback,
|
useCallback,
|
||||||
|
useEffect,
|
||||||
} from "react";
|
} from "react";
|
||||||
import { Trans, useTranslation } from "react-i18next";
|
import { Trans, useTranslation } from "react-i18next";
|
||||||
import {
|
import {
|
||||||
@@ -24,6 +25,7 @@ import {
|
|||||||
import { Button } from "@vector-im/compound-web";
|
import { Button } from "@vector-im/compound-web";
|
||||||
import { logger } from "matrix-js-sdk/lib/logger";
|
import { logger } from "matrix-js-sdk/lib/logger";
|
||||||
import { MatrixError } from "matrix-js-sdk";
|
import { MatrixError } from "matrix-js-sdk";
|
||||||
|
import { type IWidgetApiRequest } from "matrix-widget-api";
|
||||||
|
|
||||||
import {
|
import {
|
||||||
ConnectionLostError,
|
ConnectionLostError,
|
||||||
@@ -38,6 +40,7 @@ import { FullScreenView } from "../FullScreenView.tsx";
|
|||||||
import { ErrorView } from "../ErrorView.tsx";
|
import { ErrorView } from "../ErrorView.tsx";
|
||||||
import { type WidgetHelpers } from "../widget.ts";
|
import { type WidgetHelpers } from "../widget.ts";
|
||||||
import styles from "../ErrorView.module.css";
|
import styles from "../ErrorView.module.css";
|
||||||
|
import { LotusWidgetActions, lotusSendToHost } from "../lotus/lotusWidget";
|
||||||
|
|
||||||
export type CallErrorRecoveryAction = "reconnect"; // | "retry" ;
|
export type CallErrorRecoveryAction = "reconnect"; // | "retry" ;
|
||||||
|
|
||||||
@@ -74,6 +77,29 @@ const ErrorPage: FC<ErrorPageProps> = ({
|
|||||||
icon = ErrorSolidIcon;
|
icon = ErrorSolidIcon;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// [cinny #318] Tell the host the call dropped (it kept showing the call as
|
||||||
|
// joined), and let its call bar press Reconnect for us.
|
||||||
|
const recoverable =
|
||||||
|
error instanceof ConnectionLostError ||
|
||||||
|
error instanceof LivekitConnectionError ||
|
||||||
|
error instanceof PeerConnectionTimeoutError;
|
||||||
|
useEffect(() => {
|
||||||
|
lotusSendToHost(LotusWidgetActions.CallError, {
|
||||||
|
code: error.code,
|
||||||
|
recoverable,
|
||||||
|
});
|
||||||
|
const w = widget;
|
||||||
|
const onReconnect = (ev: CustomEvent<IWidgetApiRequest>): void => {
|
||||||
|
w?.api.transport.reply(ev.detail, {});
|
||||||
|
if (recoverable) void recoveryActionHandler("reconnect");
|
||||||
|
};
|
||||||
|
w?.lazyActions?.on(LotusWidgetActions.Reconnect, onReconnect);
|
||||||
|
return (): void => {
|
||||||
|
w?.lazyActions?.off(LotusWidgetActions.Reconnect, onReconnect);
|
||||||
|
lotusSendToHost(LotusWidgetActions.CallError, { code: null });
|
||||||
|
};
|
||||||
|
}, [error.code, recoverable, recoveryActionHandler, widget]);
|
||||||
|
|
||||||
const actions: { label: string; onClick: () => void }[] = [];
|
const actions: { label: string; onClick: () => void }[] = [];
|
||||||
if (error instanceof ConnectionLostError) {
|
if (error instanceof ConnectionLostError) {
|
||||||
actions.push({
|
actions.push({
|
||||||
|
|||||||
@@ -451,14 +451,16 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test("one-on-one mobile layout shows local tile when video is enabled", () => {
|
// [lotus / cinny #265] The local tile stays on mobile with the camera off
|
||||||
|
// (upstream hid it), so you can see yourself in a voice call.
|
||||||
|
test("one-on-one mobile layout shows the local tile whether or not video is enabled", () => {
|
||||||
withTestScheduler(({ behavior, schedule, expectObservable }) => {
|
withTestScheduler(({ behavior, schedule, expectObservable }) => {
|
||||||
// Local participant enables their video, then disables it
|
// Local participant enables their video, then disables it
|
||||||
const videoInputMarbles = " ny--n";
|
const videoInputMarbles = " ny--n";
|
||||||
// While tile is shown, tap the screen twice
|
// Tap the screen twice
|
||||||
const tapScreenInputMarbles = "--aa-";
|
const tapScreenInputMarbles = "--aa-";
|
||||||
// Layout should show local tile, make it small, enlarge it again, then hide it
|
// The local tile is shown throughout; taps make it small, then large again
|
||||||
const expectedLayoutMarbles = "abcba";
|
const expectedLayoutMarbles = "b-cb";
|
||||||
|
|
||||||
withCallViewModel(
|
withCallViewModel(
|
||||||
{
|
{
|
||||||
@@ -477,12 +479,6 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
|
|||||||
expectObservable(summarizeLayout$(vm.layout$)).toBe(
|
expectObservable(summarizeLayout$(vm.layout$)).toBe(
|
||||||
expectedLayoutMarbles,
|
expectedLayoutMarbles,
|
||||||
{
|
{
|
||||||
a: {
|
|
||||||
type: "one-on-one-mobile",
|
|
||||||
spotlight: [`${aliceId}:0`],
|
|
||||||
pip: undefined,
|
|
||||||
pipSize: "lg",
|
|
||||||
},
|
|
||||||
b: {
|
b: {
|
||||||
type: "one-on-one-mobile",
|
type: "one-on-one-mobile",
|
||||||
spotlight: [`${aliceId}:0`],
|
spotlight: [`${aliceId}:0`],
|
||||||
@@ -518,7 +514,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
|
|||||||
a: {
|
a: {
|
||||||
type: "one-on-one-mobile",
|
type: "one-on-one-mobile",
|
||||||
spotlight: [`${aliceId}:0`],
|
spotlight: [`${aliceId}:0`],
|
||||||
pip: undefined,
|
pip: `${localId}:0`,
|
||||||
pipSize: "lg",
|
pipSize: "lg",
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
@@ -566,7 +562,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
|
|||||||
a: {
|
a: {
|
||||||
type: "one-on-one-mobile",
|
type: "one-on-one-mobile",
|
||||||
spotlight: [`${aliceId}:0`],
|
spotlight: [`${aliceId}:0`],
|
||||||
pip: undefined,
|
pip: `${localId}:0`,
|
||||||
pipSize: "sm",
|
pipSize: "sm",
|
||||||
},
|
},
|
||||||
b: {
|
b: {
|
||||||
@@ -765,7 +761,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
|
|||||||
// This is the expected one-on-one layout for a narrow window
|
// This is the expected one-on-one layout for a narrow window
|
||||||
type: "one-on-one-mobile",
|
type: "one-on-one-mobile",
|
||||||
spotlight: [`${aliceId}:0`],
|
spotlight: [`${aliceId}:0`],
|
||||||
pip: undefined,
|
pip: `${localId}:0`,
|
||||||
pipSize: "lg",
|
pipSize: "lg",
|
||||||
},
|
},
|
||||||
b: {
|
b: {
|
||||||
|
|||||||
@@ -1220,19 +1220,22 @@ export function createCallViewModel$(
|
|||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
|
|
||||||
|
// [lotus / cinny #265] Upstream drops the local PiP on mobile while your
|
||||||
|
// camera is off; in Lotus voice rooms that is almost always, so on a phone
|
||||||
|
// you never saw yourself (or your own speaking ring). Show it as desktop
|
||||||
|
// does: the tile falls back to your avatar.
|
||||||
const oneOnOneMobileLayoutMedia$: Observable<OneOnOneMobileLayoutMedia | null> =
|
const oneOnOneMobileLayoutMedia$: Observable<OneOnOneMobileLayoutMedia | null> =
|
||||||
oneOnOneLayoutMedia$.pipe(
|
oneOnOneLayoutMedia$.pipe(
|
||||||
switchMap((media) => {
|
map((media) =>
|
||||||
if (media === null) return of(null);
|
media === null
|
||||||
return media.local.videoEnabled$.pipe(
|
? null
|
||||||
map((videoEnabled) => ({
|
: {
|
||||||
type: "one-on-one-mobile" as const,
|
type: "one-on-one-mobile" as const,
|
||||||
edgeToEdge: true as const,
|
edgeToEdge: true as const,
|
||||||
spotlight: media.remote,
|
spotlight: media.remote,
|
||||||
pip: videoEnabled ? media.local : undefined,
|
pip: media.local,
|
||||||
})),
|
},
|
||||||
);
|
),
|
||||||
}),
|
|
||||||
);
|
);
|
||||||
|
|
||||||
const pipLayoutMedia$: Observable<LayoutMedia> = spotlight$.pipe(
|
const pipLayoutMedia$: Observable<LayoutMedia> = spotlight$.pipe(
|
||||||
|
|||||||
@@ -868,16 +868,19 @@ export function enterRTCSession(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
// Calculates `maximumNetworkErrorRetryCount`. The connection is failed if EITHER:
|
// Calculates `maximumNetworkErrorRetryCount`: enough retries to outlast the
|
||||||
// - The /sync loop is unresponsive for > `gracePeriod` ms, or
|
// server's delayed-leave window (see [lotus #46] below).
|
||||||
// - A delayed leave event is emitted (after `leaveDelay` ms period).
|
|
||||||
// Note: Use leaveDelay >> gracePeriod for delegated leave events.
|
|
||||||
const gracePeriod = Config.get().sync_disconnect_grace_period_ms;
|
const gracePeriod = Config.get().sync_disconnect_grace_period_ms;
|
||||||
const leaveDelay = matrixRtcSessionConfig?.delayed_leave_event_delay_ms;
|
const leaveDelay = matrixRtcSessionConfig?.delayed_leave_event_delay_ms;
|
||||||
const retryInterval = matrixRtcSessionConfig?.network_error_retry_ms;
|
const retryInterval = matrixRtcSessionConfig?.network_error_retry_ms;
|
||||||
|
|
||||||
// Math.min is used to account for the respective worst case: /sync not available or leave event emitted.
|
// [lotus #46] Keep retrying for as long as the homeserver itself would keep
|
||||||
const maxWaitTime = Math.min(gracePeriod, leaveDelay);
|
// our membership: until its delayed leave fires (`leaveDelay`). Capping at
|
||||||
|
// the 10 s sync grace period made a ~20–35 s homeserver stall end everyone's
|
||||||
|
// call ("Connection lost") even with a 60 s server-side leave delay. The
|
||||||
|
// embedded call has no /sync loop of its own, so the grace period doesn't
|
||||||
|
// describe anything it can observe.
|
||||||
|
const maxWaitTime = Math.max(gracePeriod, leaveDelay);
|
||||||
const maximumNetworkErrorRetryCount =
|
const maximumNetworkErrorRetryCount =
|
||||||
Math.ceil(maxWaitTime / retryInterval) + 1;
|
Math.ceil(maxWaitTime / retryInterval) + 1;
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user