Author SHA1 Message Date
Lotus CIandClaude Opus 5.5 d8880daca5 chore(lotus): 0.25.0-lotus.13
CI / Build embedded bundle (push) Successful in 4m3s
CI / Publish to Gitea npm registry (push) Successful in 2m33s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-23 20:51:43 -04:00
Lotus CIandClaude Opus 5.5 3e69a18a39 fix(lotus): explain a failed voice connection and offer Try again
"Failed to connect to Livekit server (Reason: ServerUnreachable)" left a
user with nothing to try; they retried for a long time and gave up.

- Title "Couldn't connect to voice". The message depends on the reason:
  unreachable/timeout-style errors explain that voice needs its own live
  connection, which VPNs, antivirus web shields and some networks block,
  even when chat works, and say what to try; NotAllowed says the call may
  be full or access changed.
- The raw reason, status and message move under Technical details.
- LivekitConnectionError and PeerConnectionTimeoutError now get a
  Try again button (the existing reconnect path), shown as primary: the
  Lotus theme renders `secondary` as dark-on-dark.

Verified in the Lotus client against a local LiveKit with the signal
socket and validate request blocked: the dialog appears after ~20 s;
unblocking and pressing Try again joins with the mic published.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-23 20:51:43 -04:00
Lotus CIandClaude Opus 5 e9723a21b5 chore(lotus): 0.25.0-lotus.12
CI / Build embedded bundle (push) Successful in 1m32s
CI / Publish to Gitea npm registry (push) Successful in 54s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 15:51:43 -04:00
Lotus CIandClaude Opus 5 9f472fd710 fix(lotus): report an ended screenshare even when LiveKit unpublishes first (#39)
LiveKit's own ended handler runs before ours and unpublishes the share;
a listener removed during that dispatch never fires. Decide from the
track's readyState on LocalTrackUnpublished as well. Verified end to end:
closing the shared source now toasts in the host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 15:51:43 -04:00
Lotus CIandClaude Opus 5 e7c27dd8a3 feat(lotus): io.lotus.screenshare_notice — window closed, no frames, sharing alone (#39)
Watches the local screenshare publication and tells the host, once per
share each: the MediaStreamTrack ended (window closed), it has been muted
(no frames — minimised/occluded window) for 15 s, or 30 min of sharing
with nobody else in the call. Detection only; the host renders the
notices. LiveKit already unpublishes an ended share, so nothing lingers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 15:44:22 -04:00
Lotus CIandClaude Opus 5 33b51e5dc6 feat(lotus): screenshare tile speaker glyph reflects real audio state (#38)
The spotlight screenshare tile already shows a speaker glyph only when the
share publishes audio. Two gaps: a publication the sharer had muted still
counted as audio, and the host's screenshare-audio mute (io.lotus.set_deafen
→ renderer muted) wasn't reflected — the glyph stayed "on" while nothing
played. audioEnabled$ now tracks publication presence AND !isMuted via
observeParticipantMedia; the glyph shows crossed when either the tile's
own mute or the Lotus mute is active.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 15:42:48 -04:00
Lotus CIandClaude Opus 5 07def55469 chore(lotus): 0.25.0-lotus.11
CI / Build embedded bundle (push) Successful in 1m42s
CI / Publish to Gitea npm registry (push) Successful in 47s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 15:31:33 -04:00
Lotus CIandClaude Opus 5 4a48ec98d0 feat(lotus): io.lotus.set_audio_output — host-driven output device switch (#119)
The embed hides Element Call's footer, so its own output picker is out of
reach; the host's call bar now sends io.lotus.set_audio_output { deviceId }
and the fork applies it via mediaDevices.audioOutput.select(), answering
with io.lotus.audio_output_state { deviceId } on every change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 15:31:33 -04:00
Lotus CIandClaude Opus 5 b9d0ac7cac chore(lotus): 0.25.0-lotus.10
CI / Build embedded bundle (push) Successful in 1m41s
CI / Publish to Gitea npm registry (push) Successful in 45s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 13:47:46 -04:00
Lotus CIandClaude Opus 5 1e34923f6a feat(lotus): io.lotus.call_summary — per-call quality readout at hangup (#143)
Tracks the local participant's LiveKit ConnectionQuality (time spent
poor/lost) and Reconnecting events for the life of the in-call view and
sends one io.lotus.call_summary { durationMs, reconnects, poorMs, verdict }
to the host on the first SFU disconnect or on teardown. Nothing is stored
or sent anywhere else. CallQualityTracker is pure and unit-tested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-20 13:39:18 -04:00
16 changed files with 857 additions and 85 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@lotusguild/element-call-embedded",
"version": "0.25.0-lotus.9",
"version": "0.25.0-lotus.13",
"files": [
"README.md",
"LICENSE-AGPL-3.0",
+4 -2
View File
@@ -104,8 +104,9 @@
"generic_description": "Submitting debug logs will help us track down the problem.",
"insufficient_capacity": "Insufficient capacity",
"insufficient_capacity_description": "The server has reached its maximum capacity and you cannot join the call at this time. Try again later, or contact your server admin if the problem persists.",
"livekit_connection_error": "Failed to connect to Livekit server",
"livekit_connection_error_description": "An error occurred while connecting to the Livekit server (<1>Reason:</1> <2>{{ reason }}</2>).",
"livekit_connection_error": "Couldnt connect to voice",
"livekit_not_allowed_description": "The voice server didnt let you in. The call may be full, or your access to this room may have changed. Try again in a moment.",
"livekit_unreachable_description": "Your device couldnt reach the voice server. Chat can still work when this happens: voice needs its own live connection, which VPNs, antivirus web protection and some work or school networks block. Try again. If it keeps failing, pause your VPN or antivirus web shield, or try another network such as a phone hotspot.",
"matrix_rtc_transport_missing": "The server is not configured to work with {{brand}}. Please contact your server admin (Domain: {{domain}}, Error Code: {{ errorCode }}).",
"membership_manager": "Membership Manager Error",
"membership_manager_description": "The Membership Manager had to shut down. This is caused by many consecutive failed network requests.",
@@ -119,6 +120,7 @@
"sfu_token_refused": "Can't join this call",
"sticky_events_required": "Homeserver does not support Matrix 2.0 calls",
"sticky_events_required_description": "This deployment is configured to use Matrix 2.0 call mode, but the homeserver does not advertise support for sticky events (MSC4354). Ask your server admin to upgrade, or switch the deployment to a compatible mode.",
"try_again": "Try again",
"unexpected_ec_error": "An unexpected error occurred (<0>Error Code:</0> <1>{{ errorCode }}</1>). Please contact your server admin."
},
"group_call_loader": {
+1
View File
@@ -26,6 +26,7 @@ describe("LotusWidgetActions", () => {
LotusWidgetActions.SetQuality,
LotusWidgetActions.Decorations,
LotusWidgetActions.SetDeafen,
LotusWidgetActions.SetAudioOutput,
];
expect(new Set(LOTUS_TO_WIDGET_ACTIONS)).toEqual(new Set(expectedToWidget));
+17
View File
@@ -46,6 +46,22 @@ export enum LotusWidgetActions {
* toggle can reflect reality rather than the requested state.
*/
DenoiseState = "io.lotus.denoise_state",
/**
* fromWidget: one-shot end-of-call readout for the local participant —
* `{ durationMs, reconnects, poorMs, verdict }` (#143) — sent on SFU
* disconnect or in-call teardown, whichever comes first.
*/
CallSummary = "io.lotus.call_summary",
/** toWidget: select the audio output device `{ deviceId }` (#119). */
SetAudioOutput = "io.lotus.set_audio_output",
/** fromWidget: the currently selected output `{ deviceId }` (#119). */
AudioOutputState = "io.lotus.audio_output_state",
/**
* fromWidget: local screenshare reminder `{ kind: "ended" | "no-frames" | "alone" }`
* — window closed, no frames for 15 s, or 30 min of sharing with nobody
* else in the call (#39). Each fires at most once per share.
*/
ScreenshareNotice = "io.lotus.screenshare_notice",
}
/** toWidget Lotus actions that must be allow-listed in `initializeWidget`. */
@@ -55,4 +71,5 @@ export const LOTUS_TO_WIDGET_ACTIONS: LotusWidgetActions[] = [
LotusWidgetActions.SetQuality,
LotusWidgetActions.Decorations,
LotusWidgetActions.SetDeafen,
LotusWidgetActions.SetAudioOutput,
];
+46
View File
@@ -0,0 +1,46 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { type IWidgetApiRequest } from "matrix-widget-api";
import { logger } from "matrix-js-sdk/lib/logger";
import { type MediaDevices } from "../state/MediaDevices";
import { widget } from "../widget";
import { LotusWidgetActions } from "./lotusActions";
import { lotusSendToHost } from "./lotusWidget";
/**
* [Gitea #119] Let the host switch the audio output device (headset ↔
* speakers) from its own call bar: `io.lotus.set_audio_output { deviceId }`
* selects the output; the fork answers each change (and the initial state)
* with `io.lotus.audio_output_state { deviceId, available: [{id,label}] }`.
* No effect unless the host sends the action. Returns a teardown function.
*/
export function startLotusAudioOutput(mediaDevices: MediaDevices): () => void {
const w = widget;
if (!w) return () => undefined;
const handler = (ev: CustomEvent<IWidgetApiRequest>): void => {
w.api.transport.reply(ev.detail, {});
const data = ev.detail.data as { deviceId?: string } | undefined;
if (typeof data?.deviceId !== "string") return;
logger.debug(`[lotus] set_audio_output: ${data.deviceId}`);
mediaDevices.audioOutput.select(data.deviceId);
};
w.lazyActions.on(LotusWidgetActions.SetAudioOutput, handler);
const sub = mediaDevices.audioOutput.selected$.subscribe((selected) => {
lotusSendToHost(LotusWidgetActions.AudioOutputState, {
deviceId: selected?.id ?? null,
});
});
return () => {
sub.unsubscribe();
w.lazyActions.off(LotusWidgetActions.SetAudioOutput, handler);
};
}
+93
View File
@@ -0,0 +1,93 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { ConnectionQuality } from "livekit-client";
import { describe, expect, it } from "vitest";
import { CallQualityTracker } from "./lotusCallSummary";
const clock = (): { now: () => number; tick: (ms: number) => void } => {
let t = 1_000_000;
return {
now: () => t,
tick: (ms) => {
t += ms;
},
};
};
describe("CallQualityTracker", () => {
it("reports unknown when nothing was sampled", () => {
const c = clock();
const tr = new CallQualityTracker(c.now);
tr.start();
c.tick(60_000);
expect(tr.summary()).toEqual({
durationMs: 60_000,
reconnects: 0,
poorMs: 0,
verdict: "unknown",
});
});
it("is good when quality stayed fine", () => {
const c = clock();
const tr = new CallQualityTracker(c.now);
tr.start();
tr.setQuality(ConnectionQuality.Excellent);
c.tick(30 * 60_000);
expect(tr.summary().verdict).toBe("good");
});
it("accumulates poor time across episodes, including an open one", () => {
const c = clock();
const tr = new CallQualityTracker(c.now);
tr.start();
tr.setQuality(ConnectionQuality.Good);
c.tick(60_000);
tr.setQuality(ConnectionQuality.Poor);
c.tick(10_000);
tr.setQuality(ConnectionQuality.Good);
c.tick(60_000);
tr.setQuality(ConnectionQuality.Lost);
c.tick(5_000);
const s = tr.summary();
expect(s.poorMs).toBe(15_000);
expect(s.durationMs).toBe(135_000);
expect(s.verdict).toBe("fair");
});
it("is poor with many reconnects or mostly-poor quality", () => {
const c = clock();
const tr = new CallQualityTracker(c.now);
tr.start();
tr.setQuality(ConnectionQuality.Good);
for (let i = 0; i < 4; i += 1) tr.reconnect();
c.tick(60_000);
expect(tr.summary()).toMatchObject({ reconnects: 4, verdict: "poor" });
const tr2 = new CallQualityTracker(c.now);
tr2.start();
tr2.setQuality(ConnectionQuality.Poor);
c.tick(60_000);
expect(tr2.summary().verdict).toBe("poor");
});
it("ignores unknown samples and only starts once", () => {
const c = clock();
const tr = new CallQualityTracker(c.now);
tr.start();
c.tick(1_000);
tr.start();
tr.setQuality(ConnectionQuality.Unknown);
c.tick(1_000);
expect(tr.summary()).toMatchObject({
durationMs: 2_000,
verdict: "unknown",
});
});
});
+144
View File
@@ -0,0 +1,144 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
ConnectionQuality,
type Participant,
type Room as LivekitRoom,
RoomEvent,
} from "livekit-client";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { widget } from "../widget";
import { LotusWidgetActions } from "./lotusActions";
import { lotusSendToHost } from "./lotusWidget";
export type CallQualityVerdict = "good" | "fair" | "poor" | "unknown";
export interface CallSummary {
/** Wall-clock time connected to the SFU, ms. */
durationMs: number;
/** LiveKit reconnect attempts during the call. */
reconnects: number;
/** Time the local connection quality was reported poor/lost, ms. */
poorMs: number;
verdict: CallQualityVerdict;
}
/**
* [Gitea #143] Per-call connection quality, kept in memory for the local
* participant only and summarised once at hangup. Nothing is stored or sent
* anywhere but to the host at the end.
*/
export class CallQualityTracker {
private startedAt: number | undefined;
private poorSince: number | undefined;
private poorMs = 0;
private reconnects = 0;
private sampled = false;
public constructor(private readonly now: () => number = () => Date.now()) {}
public start(): void {
if (this.startedAt === undefined) this.startedAt = this.now();
}
public setQuality(quality: ConnectionQuality): void {
if (quality === ConnectionQuality.Unknown) return;
this.sampled = true;
const bad =
quality === ConnectionQuality.Poor || quality === ConnectionQuality.Lost;
if (bad && this.poorSince === undefined) this.poorSince = this.now();
if (!bad && this.poorSince !== undefined) {
this.poorMs += this.now() - this.poorSince;
this.poorSince = undefined;
}
}
public reconnect(): void {
this.reconnects += 1;
}
public summary(): CallSummary {
const end = this.now();
const durationMs =
this.startedAt === undefined ? 0 : Math.max(0, end - this.startedAt);
const poorMs =
this.poorMs + (this.poorSince === undefined ? 0 : end - this.poorSince);
let verdict: CallQualityVerdict = "unknown";
if (this.sampled && durationMs > 0) {
const poorShare = poorMs / durationMs;
if (poorShare < 0.05 && this.reconnects <= 1) verdict = "good";
else if (poorShare < 0.25 && this.reconnects <= 3) verdict = "fair";
else verdict = "poor";
}
return { durationMs, reconnects: this.reconnects, poorMs, verdict };
}
}
/**
* Track the local participant's LiveKit connection quality and reconnects
* for the life of the in-call view, and send `io.lotus.call_summary` to the
* host once — on the first SFU disconnect or on teardown, whichever comes
* first — so the host can show "41 min · connection was good" at hangup.
*/
export function startLotusCallSummary(vm: CallViewModel): () => void {
if (!widget) return () => undefined;
const tracker = new CallQualityTracker();
let sent = false;
const send = (): void => {
if (sent) return;
sent = true;
lotusSendToHost(LotusWidgetActions.CallSummary, tracker.summary());
};
const listeners = new Map<LivekitRoom, () => void>();
const attach = (room: LivekitRoom): void => {
const onQuality = (
quality: ConnectionQuality,
participant: Participant,
): void => {
if (participant.isLocal) tracker.setQuality(quality);
};
const onConnected = (): void => tracker.start();
const onReconnecting = (): void => tracker.reconnect();
const onDisconnected = (): void => send();
room.on(RoomEvent.ConnectionQualityChanged, onQuality);
room.on(RoomEvent.Connected, onConnected);
room.on(RoomEvent.Reconnecting, onReconnecting);
room.on(RoomEvent.Disconnected, onDisconnected);
if (room.state === "connected") tracker.start();
listeners.set(room, () => {
room.off(RoomEvent.ConnectionQualityChanged, onQuality);
room.off(RoomEvent.Connected, onConnected);
room.off(RoomEvent.Reconnecting, onReconnecting);
room.off(RoomEvent.Disconnected, onDisconnected);
});
};
const sub = vm.allConnections$.subscribe((data) => {
const next = data.getConnections().map((c) => c.livekitRoom);
for (const [room, off] of listeners) {
if (!next.includes(room)) {
off();
listeners.delete(room);
}
}
for (const room of next) if (!listeners.has(room)) attach(room);
});
return () => {
sub.unsubscribe();
for (const off of listeners.values()) off();
listeners.clear();
send();
};
}
+171
View File
@@ -0,0 +1,171 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { RoomEvent, Track } from "livekit-client";
import { BehaviorSubject } from "rxjs";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import {
ALONE_MS,
NO_FRAMES_MS,
startLotusScreenshareWatch,
} from "./lotusScreenshareWatch";
const sent: unknown[] = [];
vi.mock("../widget", () => ({ widget: { api: {} } }));
vi.mock("./lotusWidget", () => ({
lotusSendToHost: (action: string, data: unknown): boolean => {
sent.push({ action, data });
return true;
},
}));
class FakeTrack extends EventTarget {}
const makeRoom = (): {
room: Record<string, unknown>;
emit: (event: string, ...args: unknown[]) => void;
remote: Map<string, unknown>;
} => {
const handlers = new Map<string, ((...args: unknown[]) => void)[]>();
const remote = new Map<string, unknown>();
const room = {
remoteParticipants: remote,
localParticipant: { trackPublications: new Map() },
on: (event: string, h: (...args: unknown[]) => void): void => {
handlers.set(event, [...(handlers.get(event) ?? []), h]);
},
off: (event: string, h: (...args: unknown[]) => void): void => {
handlers.set(
event,
(handlers.get(event) ?? []).filter((x) => x !== h),
);
},
};
return {
room,
remote,
emit: (event, ...args): void => {
(handlers.get(event) ?? []).forEach((h) => h(...args));
},
};
};
describe("startLotusScreenshareWatch", () => {
let clock = 0;
beforeEach(() => {
sent.length = 0;
clock = 1_000_000;
vi.useFakeTimers();
});
afterEach(() => vi.useRealTimers());
const start = (): ReturnType<typeof makeRoom> & { stop: () => void } => {
const fake = makeRoom();
const connections = new BehaviorSubject({
getConnections: () => [{ livekitRoom: fake.room }],
});
const stop = startLotusScreenshareWatch(
{ allConnections$: connections } as unknown as CallViewModel,
() => clock,
);
return { ...fake, stop };
};
it("reports a share whose track ended", () => {
const { emit, stop } = start();
const mst = new FakeTrack();
emit(RoomEvent.LocalTrackPublished, {
source: Track.Source.ScreenShare,
track: { mediaStreamTrack: mst },
});
mst.dispatchEvent(new Event("ended"));
expect(sent).toEqual([
{ action: "io.lotus.screenshare_notice", data: { kind: "ended" } },
]);
stop();
});
it("reports no frames after a sustained mute, once, and not after an unmute", () => {
const { emit, stop } = start();
const mst = new FakeTrack();
emit(RoomEvent.LocalTrackPublished, {
source: Track.Source.ScreenShare,
track: { mediaStreamTrack: mst },
});
mst.dispatchEvent(new Event("mute"));
vi.advanceTimersByTime(NO_FRAMES_MS / 2);
mst.dispatchEvent(new Event("unmute"));
vi.advanceTimersByTime(NO_FRAMES_MS);
expect(sent).toEqual([]);
mst.dispatchEvent(new Event("mute"));
vi.advanceTimersByTime(NO_FRAMES_MS);
mst.dispatchEvent(new Event("mute"));
vi.advanceTimersByTime(NO_FRAMES_MS);
expect(sent).toEqual([
{ action: "io.lotus.screenshare_notice", data: { kind: "no-frames" } },
]);
stop();
});
it("nudges after 30 min of sharing with nobody else, never while others are present", () => {
const { emit, remote, stop } = start();
emit(RoomEvent.LocalTrackPublished, {
source: Track.Source.ScreenShare,
track: { mediaStreamTrack: new FakeTrack() },
});
remote.set("bob", {});
clock += ALONE_MS + 60_000;
vi.advanceTimersByTime(60_000);
expect(sent).toEqual([]);
remote.clear();
vi.advanceTimersByTime(60_000);
vi.advanceTimersByTime(60_000);
expect(sent).toEqual([
{ action: "io.lotus.screenshare_notice", data: { kind: "alone" } },
]);
stop();
});
it("reports ended from the unpublish when LiveKit's handler ran first", () => {
const { emit, stop } = start();
const share = new FakeTrack() as FakeTrack & { readyState: string };
share.readyState = "ended";
const pub = {
source: Track.Source.ScreenShare,
track: { mediaStreamTrack: share },
};
emit(RoomEvent.LocalTrackPublished, pub);
emit(RoomEvent.LocalTrackUnpublished, pub);
share.dispatchEvent(new Event("ended"));
expect(sent).toEqual([
{ action: "io.lotus.screenshare_notice", data: { kind: "ended" } },
]);
stop();
});
it("ignores non-screenshare publications and stops watching on unpublish", () => {
const { emit, stop } = start();
const cam = new FakeTrack();
emit(RoomEvent.LocalTrackPublished, {
source: Track.Source.Camera,
track: { mediaStreamTrack: cam },
});
cam.dispatchEvent(new Event("ended"));
const share = new FakeTrack();
const pub = {
source: Track.Source.ScreenShare,
track: { mediaStreamTrack: share },
};
emit(RoomEvent.LocalTrackPublished, pub);
emit(RoomEvent.LocalTrackUnpublished, pub);
share.dispatchEvent(new Event("ended"));
expect(sent).toEqual([]);
stop();
});
});
+136
View File
@@ -0,0 +1,136 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import {
type LocalTrackPublication,
type Room as LivekitRoom,
RoomEvent,
Track,
} from "livekit-client";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { widget } from "../widget";
import { LotusWidgetActions } from "./lotusActions";
import { lotusSendToHost } from "./lotusWidget";
export type ScreenshareNoticeKind = "ended" | "no-frames" | "alone";
/** A shared window minimised/occluded this long with no frames → tell the host. */
export const NO_FRAMES_MS = 15_000;
/** Sharing this long with nobody else in the call → one "still sharing?" nudge. */
export const ALONE_MS = 30 * 60_000;
const ALONE_CHECK_MS = 60_000;
/**
* [lotus #39] Watch the local screenshare and tell the host about the two
* cases people miss: the share went black/ended (window closed or minimised)
* and a long share with nobody else in the call. Detection only — the host
* renders the notices. Each notice fires at most once per share.
*/
export function startLotusScreenshareWatch(
vm: CallViewModel,
now: () => number = () => Date.now(),
): () => void {
if (!widget) return () => undefined;
const notify = (kind: ScreenshareNoticeKind): void => {
lotusSendToHost(LotusWidgetActions.ScreenshareNotice, { kind });
};
const perRoom = new Map<LivekitRoom, () => void>();
const attach = (room: LivekitRoom): void => {
let cleanupTrack: (() => void) | undefined;
let endedHook: (() => void) | undefined;
const watchPublication = (pub: LocalTrackPublication): void => {
if (pub.source !== Track.Source.ScreenShare) return;
cleanupTrack?.();
const mst = pub.track?.mediaStreamTrack;
const startedAt = now();
let noFramesTimer: ReturnType<typeof setTimeout> | undefined;
let sentNoFrames = false;
let sentAlone = false;
let sentEnded = false;
const onEnded = (): void => {
if (sentEnded) return;
sentEnded = true;
notify("ended");
};
const onMute = (): void => {
if (sentNoFrames) return;
noFramesTimer = setTimeout(() => {
sentNoFrames = true;
notify("no-frames");
}, NO_FRAMES_MS);
};
const onUnmute = (): void => {
if (noFramesTimer !== undefined) clearTimeout(noFramesTimer);
noFramesTimer = undefined;
};
endedHook = onEnded;
mst?.addEventListener("ended", onEnded);
mst?.addEventListener("mute", onMute);
mst?.addEventListener("unmute", onUnmute);
const aloneTimer = setInterval(() => {
if (sentAlone) return;
if (room.remoteParticipants.size > 0) return;
if (now() - startedAt < ALONE_MS) return;
sentAlone = true;
notify("alone");
}, ALONE_CHECK_MS);
cleanupTrack = (): void => {
mst?.removeEventListener("ended", onEnded);
mst?.removeEventListener("mute", onMute);
mst?.removeEventListener("unmute", onUnmute);
if (noFramesTimer !== undefined) clearTimeout(noFramesTimer);
clearInterval(aloneTimer);
cleanupTrack = undefined;
endedHook = undefined;
};
};
// LiveKit's own `ended` handler runs first and unpublishes the share; a
// listener removed during that dispatch never fires, so decide from the
// track's state here as well.
const onUnpublished = (pub: LocalTrackPublication): void => {
if (pub.source !== Track.Source.ScreenShare) return;
if (pub.track?.mediaStreamTrack?.readyState === "ended") endedHook?.();
cleanupTrack?.();
};
room.on(RoomEvent.LocalTrackPublished, watchPublication);
room.on(RoomEvent.LocalTrackUnpublished, onUnpublished);
room.localParticipant.trackPublications.forEach(watchPublication);
perRoom.set(room, () => {
cleanupTrack?.();
room.off(RoomEvent.LocalTrackPublished, watchPublication);
room.off(RoomEvent.LocalTrackUnpublished, onUnpublished);
});
};
const sub = vm.allConnections$.subscribe((data) => {
const rooms = data.getConnections().map((c) => c.livekitRoom);
for (const [room, off] of perRoom) {
if (!rooms.includes(room)) {
off();
perRoom.delete(room);
}
}
for (const room of rooms) if (!perRoom.has(room)) attach(room);
});
return () => {
sub.unsubscribe();
for (const off of perRoom.values()) off();
perRoom.clear();
};
}
+24 -7
View File
@@ -365,17 +365,18 @@ describe("LiveKit ConnectionError variants", () => {
expectedReason: "InternalError",
},
])(
"should display LiveKit $name error correctly",
"should explain the LiveKit $name error and offer a retry",
async ({ error, expectedReason }) => {
const TestComponent = (): ReactNode => {
throw new LivekitConnectionError(error);
};
const recoveryActionHandler = vi.fn(async () => Promise.resolve());
const { asFragment } = render(
<BrowserRouter>
<GroupCallErrorBoundary
onError={vi.fn()}
recoveryActionHandler={vi.fn()}
recoveryActionHandler={recoveryActionHandler}
widget={null}
>
<TestComponent />
@@ -383,12 +384,28 @@ describe("LiveKit ConnectionError variants", () => {
</BrowserRouter>,
);
// Check title
await screen.findByText("Failed to connect to Livekit server");
await screen.findByText("Couldnt connect to voice");
// Check that reason is displayed in the description
expect(screen.getByText(/Reason:/i)).toBeInTheDocument();
expect(screen.getByText(expectedReason)).toBeInTheDocument();
// [lotus] Plain-language guidance instead of "(Reason: X)".
expect(
screen.getByText(
expectedReason === "NotAllowed"
? /The voice server didnt let you in/
: /Your device couldnt reach the voice server/,
),
).toBeInTheDocument();
// The raw reason is still there, under Technical details.
expect(screen.getByText("Technical details")).toBeInTheDocument();
expect(
screen.getByText(new RegExp(`Reason: ${expectedReason}`), {
selector: "pre",
}),
).toBeInTheDocument();
// Try again re-enters the call.
await userEvent.click(screen.getByRole("button", { name: "Try again" }));
expect(recoveryActionHandler).toHaveBeenCalledWith("reconnect");
expect(asFragment()).toMatchSnapshot();
},
+19 -5
View File
@@ -30,6 +30,8 @@ import {
ElementCallError,
ErrorCategory,
ErrorCode,
LivekitConnectionError,
PeerConnectionTimeoutError,
UnknownCallError,
} from "../utils/errors.ts";
import { FullScreenView } from "../FullScreenView.tsx";
@@ -78,10 +80,22 @@ const ErrorPage: FC<ErrorPageProps> = ({
label: t("call_ended_view.reconnect_button"),
onClick: () => void recoveryActionHandler("reconnect"),
});
} else if (
// [lotus] A failed connect is often transient (busy server, flaky Wi-Fi);
// offer the same re-enter path instead of a dead end.
error instanceof LivekitConnectionError ||
error instanceof PeerConnectionTimeoutError
) {
actions.push({
label: t("error.try_again"),
onClick: () => void recoveryActionHandler("reconnect"),
});
}
const technicalError =
error.cause instanceof MatrixError ? error.cause : null;
error.cause instanceof MatrixError
? error.cause.message
: (error.technicalDetails ?? null);
return (
<FullScreenView>
@@ -124,15 +138,15 @@ const ErrorPage: FC<ErrorPageProps> = ({
<summary className={styles.technicalDetailsSummary}>
{t("technical_details")}
</summary>
<pre className={styles.technicalDetailsPre}>
{technicalError.message}
</pre>
<pre className={styles.technicalDetailsPre}>{technicalError}</pre>
</details>
) : null}
{actions &&
actions.map((action, index) => (
<Button
kind="secondary"
// [lotus] primary: the Lotus theme renders `secondary` as dark
// text on a dark fill, and this is the action we want taken.
kind="primary"
onClick={action.onClick}
key={`action${index}`}
>
+12
View File
@@ -35,6 +35,9 @@ import { startLotusAudioInject } from "../lotus/lotusAudioInject";
import { startLotusQuality } from "../lotus/lotusQuality";
import { startLotusDecorations } from "../lotus/lotusDecorations";
import { startLotusDenoise } from "../lotus/lotusDenoise";
import { startLotusCallSummary } from "../lotus/lotusCallSummary";
import { startLotusAudioOutput } from "../lotus/lotusAudioOutput";
import { startLotusScreenshareWatch } from "../lotus/lotusScreenshareWatch";
import { startLotusDeafen } from "../lotus/lotusDeafen";
import styles from "./InCallView.module.css";
import { GridTile } from "../tile/GridTile";
@@ -309,10 +312,19 @@ export const InCallView: FC<InCallViewProps> = ({
// [lotus] Apply ML denoise to the mic as a first-class audio processor that
// survives reconnects (#1 / A7). No-op unless lotusDenoiseSource=1.
useEffect(() => startLotusDenoise(vm), [vm]);
useEffect(() => startLotusCallSummary(vm), [vm]);
// [lotus #39] Screenshare reminders (window closed / black / sharing alone).
useEffect(() => startLotusScreenshareWatch(vm), [vm]);
// [lotus] Handle the host's io.lotus.set_deafen action to silence remote
// audio (and optionally screenshare audio) at the LiveKit source. No-op
// unless the host sends the action.
useEffect(() => startLotusDeafen(), []);
// [lotus #119] Let the host pick the audio output device from its call bar.
const lotusMediaDevices = useMediaDevices();
useEffect(
() => startLotusAudioOutput(lotusMediaDevices),
[lotusMediaDevices],
);
const fatalCallError = useBehavior(vm.fatalError$);
// Stop the rendering and throw for the error boundary
@@ -135,7 +135,7 @@ exports[`ConnectionLostError: Action handling should reset error state 1`] = `
</p>
<button
class="_button_1nw83_8"
data-kind="secondary"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
@@ -158,7 +158,7 @@ exports[`ConnectionLostError: Action handling should reset error state 1`] = `
</DocumentFragment>
`;
exports[`LiveKit ConnectionError variants > should display LiveKit 'internal' error correctly 1`] = `
exports[`LiveKit ConnectionError variants > should explain the LiveKit 'internal' error and offer a retry 1`] = `
<DocumentFragment>
<div
class="_page_4be5c0"
@@ -286,19 +286,35 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'internal' er
<h1
class="_typography_6v6n8_153 _font-heading-md-semibold_6v6n8_112"
>
Failed to connect to Livekit server
Couldnt connect to voice
</h1>
<p>
An error occurred while connecting to the Livekit server (
<b>
Reason:
</b>
<code>
InternalError
</code>
).
Your device couldnt reach the voice server. Chat can still work when this happens: voice needs its own live connection, which VPNs, antivirus web protection and some work or school networks block. Try again. If it keeps failing, pause your VPN or antivirus web shield, or try another network such as a phone hotspot.
</p>
<details
class="_technicalDetails_a69dc5"
>
<summary
class="_technicalDetailsSummary_a69dc5"
>
Technical details
</summary>
<pre
class="_technicalDetailsPre_a69dc5"
>
Reason: InternalError
Internal server error
</pre>
</details>
<button
class="_button_1nw83_8"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
>
Try again
</button>
<button
class="_button_1nw83_8"
data-kind="tertiary"
@@ -315,7 +331,7 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'internal' er
</DocumentFragment>
`;
exports[`LiveKit ConnectionError variants > should display LiveKit 'notAllowed' error correctly 1`] = `
exports[`LiveKit ConnectionError variants > should explain the LiveKit 'notAllowed' error and offer a retry 1`] = `
<DocumentFragment>
<div
class="_page_4be5c0"
@@ -443,19 +459,36 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'notAllowed'
<h1
class="_typography_6v6n8_153 _font-heading-md-semibold_6v6n8_112"
>
Failed to connect to Livekit server
Couldnt connect to voice
</h1>
<p>
An error occurred while connecting to the Livekit server (
<b>
Reason:
</b>
<code>
NotAllowed
</code>
).
The voice server didnt let you in. The call may be full, or your access to this room may have changed. Try again in a moment.
</p>
<details
class="_technicalDetails_a69dc5"
>
<summary
class="_technicalDetailsSummary_a69dc5"
>
Technical details
</summary>
<pre
class="_technicalDetailsPre_a69dc5"
>
Reason: NotAllowed
Status: 403
Permission denied by server
</pre>
</details>
<button
class="_button_1nw83_8"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
>
Try again
</button>
<button
class="_button_1nw83_8"
data-kind="tertiary"
@@ -472,7 +505,7 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'notAllowed'
</DocumentFragment>
`;
exports[`LiveKit ConnectionError variants > should display LiveKit 'serverUnreachable' error correctly 1`] = `
exports[`LiveKit ConnectionError variants > should explain the LiveKit 'serverUnreachable' error and offer a retry 1`] = `
<DocumentFragment>
<div
class="_page_4be5c0"
@@ -600,19 +633,36 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'serverUnreac
<h1
class="_typography_6v6n8_153 _font-heading-md-semibold_6v6n8_112"
>
Failed to connect to Livekit server
Couldnt connect to voice
</h1>
<p>
An error occurred while connecting to the Livekit server (
<b>
Reason:
</b>
<code>
ServerUnreachable
</code>
).
Your device couldnt reach the voice server. Chat can still work when this happens: voice needs its own live connection, which VPNs, antivirus web protection and some work or school networks block. Try again. If it keeps failing, pause your VPN or antivirus web shield, or try another network such as a phone hotspot.
</p>
<details
class="_technicalDetails_a69dc5"
>
<summary
class="_technicalDetailsSummary_a69dc5"
>
Technical details
</summary>
<pre
class="_technicalDetailsPre_a69dc5"
>
Reason: ServerUnreachable
Status: 503
Server is unreachable
</pre>
</details>
<button
class="_button_1nw83_8"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
>
Try again
</button>
<button
class="_button_1nw83_8"
data-kind="tertiary"
@@ -629,7 +679,7 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'serverUnreac
</DocumentFragment>
`;
exports[`LiveKit ConnectionError variants > should display LiveKit 'serviceNotFound' error correctly 1`] = `
exports[`LiveKit ConnectionError variants > should explain the LiveKit 'serviceNotFound' error and offer a retry 1`] = `
<DocumentFragment>
<div
class="_page_4be5c0"
@@ -757,19 +807,35 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'serviceNotFo
<h1
class="_typography_6v6n8_153 _font-heading-md-semibold_6v6n8_112"
>
Failed to connect to Livekit server
Couldnt connect to voice
</h1>
<p>
An error occurred while connecting to the Livekit server (
<b>
Reason:
</b>
<code>
ServiceNotFound
</code>
).
Your device couldnt reach the voice server. Chat can still work when this happens: voice needs its own live connection, which VPNs, antivirus web protection and some work or school networks block. Try again. If it keeps failing, pause your VPN or antivirus web shield, or try another network such as a phone hotspot.
</p>
<details
class="_technicalDetails_a69dc5"
>
<summary
class="_technicalDetailsSummary_a69dc5"
>
Technical details
</summary>
<pre
class="_technicalDetailsPre_a69dc5"
>
Reason: ServiceNotFound
RTC service not found
</pre>
</details>
<button
class="_button_1nw83_8"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
>
Try again
</button>
<button
class="_button_1nw83_8"
data-kind="tertiary"
@@ -786,7 +852,7 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'serviceNotFo
</DocumentFragment>
`;
exports[`LiveKit ConnectionError variants > should display LiveKit 'timeout' error correctly 1`] = `
exports[`LiveKit ConnectionError variants > should explain the LiveKit 'timeout' error and offer a retry 1`] = `
<DocumentFragment>
<div
class="_page_4be5c0"
@@ -914,19 +980,35 @@ exports[`LiveKit ConnectionError variants > should display LiveKit 'timeout' err
<h1
class="_typography_6v6n8_153 _font-heading-md-semibold_6v6n8_112"
>
Failed to connect to Livekit server
Couldnt connect to voice
</h1>
<p>
An error occurred while connecting to the Livekit server (
<b>
Reason:
</b>
<code>
Timeout
</code>
).
Your device couldnt reach the voice server. Chat can still work when this happens: voice needs its own live connection, which VPNs, antivirus web protection and some work or school networks block. Try again. If it keeps failing, pause your VPN or antivirus web shield, or try another network such as a phone hotspot.
</p>
<details
class="_technicalDetails_a69dc5"
>
<summary
class="_technicalDetailsSummary_a69dc5"
>
Technical details
</summary>
<pre
class="_technicalDetailsPre_a69dc5"
>
Reason: Timeout
Connection timed out
</pre>
</details>
<button
class="_button_1nw83_8"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
>
Try again
</button>
<button
class="_button_1nw83_8"
data-kind="tertiary"
@@ -1084,6 +1166,15 @@ exports[`LiveKit ConnectionError variants > should link to troubleshoot guide wh
</a>
or contact your server administrator.
</p>
<button
class="_button_1nw83_8"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
>
Try again
</button>
<button
class="_button_1nw83_8"
data-kind="tertiary"
@@ -1697,7 +1788,7 @@ exports[`should report correct error for 'Connection lost' 1`] = `
</p>
<button
class="_button_1nw83_8"
data-kind="secondary"
data-kind="primary"
data-size="lg"
role="button"
tabindex="0"
+15 -5
View File
@@ -7,7 +7,7 @@ Please see LICENSE in the repository root for full details.
*/
import { Track, type RemoteParticipant } from "livekit-client";
import { map, of, switchMap } from "rxjs";
import { distinctUntilChanged, map, of, switchMap } from "rxjs";
import { type Behavior } from "../Behavior";
import {
@@ -17,7 +17,7 @@ import {
} from "./ScreenShareViewModel";
import { type ObservableScope } from "../ObservableScope";
import { createVolumeControls, type VolumeControls } from "../VolumeControls";
import { observeTrackReference$ } from "../observeTrackReference";
import { observeParticipantMedia } from "@livekit/components-core";
export interface RemoteScreenShareViewModel
extends BaseScreenShareViewModel, VolumeControls {
@@ -58,14 +58,24 @@ export function createRemoteScreenShare(
videoEnabled$: scope.behavior(
pretendToBeDisconnected$.pipe(map((disconnected) => !disconnected)),
),
// [lotus #38] "Has audio" means the sharer publishes screenshare audio AND
// hasn't muted it — a muted publication would show a speaker glyph for a
// share nobody can hear.
audioEnabled$: scope.behavior(
inputs.participant$.pipe(
switchMap((p) =>
p
? observeTrackReference$(p, Track.Source.ScreenShareAudio)
: of(null),
? observeParticipantMedia(p).pipe(
map(() => {
const pub = p.getTrackPublication(
Track.Source.ScreenShareAudio,
);
return !!pub && !pub.isMuted;
}),
)
: of(false),
),
map(Boolean),
distinctUntilChanged(),
),
),
};
+7 -4
View File
@@ -43,6 +43,7 @@ import { useReactiveState } from "../useReactiveState";
import { useLatest } from "../useLatest";
import { type SpotlightTileViewModel } from "../state/TileViewModel";
import { useBehavior } from "../useBehavior";
import { muteScreenshareAudio$ } from "../lotus/lotusScreenshareAudio";
import { type MemberMediaViewModel } from "../state/media/MemberMediaViewModel";
import { type LocalUserMediaViewModel } from "../state/media/LocalUserMediaViewModel";
import { type RemoteUserMediaViewModel } from "../state/media/RemoteUserMediaViewModel";
@@ -322,11 +323,13 @@ const ScreenShareVolumeButton: FC<ScreenShareVolumeButtonProps> = ({ vm }) => {
const audioEnabled = useBehavior(vm.audioEnabled$);
const playbackMuted = useBehavior(vm.playbackMuted$);
const playbackVolume = useBehavior(vm.playbackVolume$);
// [lotus #38] The host's screenshare-audio mute (io.lotus.set_deafen) mutes
// at the renderer, not through the volume controls; show it as muted too.
const lotusMuted = useBehavior(muteScreenshareAudio$);
const shownMuted = playbackMuted || lotusMuted;
const VolumeIcon = playbackMuted ? VolumeOffIcon : VolumeOnIcon;
const VolumeSolidIcon = playbackMuted
? VolumeOffSolidIcon
: VolumeOnSolidIcon;
const VolumeIcon = shownMuted ? VolumeOffIcon : VolumeOnIcon;
const VolumeSolidIcon = shownMuted ? VolumeOffSolidIcon : VolumeOnSolidIcon;
const [volumeMenuOpen, setVolumeMenuOpen] = useState(false);
const onMuteButtonClick = useCallback(() => vm.togglePlaybackMuted(), [vm]);
+19 -4
View File
@@ -6,7 +6,7 @@ Please see LICENSE in the repository root for full details.
*/
import { t } from "i18next";
import { type ConnectionError } from "livekit-client";
import { type ConnectionError, ConnectionErrorReason } from "livekit-client";
import { i18nKey } from "./i18n";
@@ -53,6 +53,9 @@ export class ElementCallError extends Error {
public localisedMessageKey?: string;
public localisedMessageValues?: Record<string, string>;
/** [lotus] Raw detail shown under a collapsed "Technical details", not in the message. */
public technicalDetails?: string;
protected constructor(
localisedTitle: string,
code: ErrorCode,
@@ -307,9 +310,21 @@ export class LivekitConnectionError extends ElementCallError {
ErrorCode.SFU_ERROR,
ErrorCategory.NETWORK_CONNECTIVITY,
);
this.localisedMessageKey = i18nKey(
"error.livekit_connection_error_description",
);
// [lotus] "An error occurred while connecting to the Livekit server
// (Reason: ServerUnreachable)" left people with nothing to try (Gitea
// element-call: a friend retried for ages while Element worked). Say what
// it means and what to do; keep the raw reason under Technical details.
this.localisedMessageKey =
cause.reason === ConnectionErrorReason.NotAllowed
? i18nKey("error.livekit_not_allowed_description")
: i18nKey("error.livekit_unreachable_description");
this.localisedMessageValues = { reason: cause.reasonName };
this.technicalDetails = [
`Reason: ${cause.reasonName}`,
cause.status ? `Status: ${cause.status}` : null,
cause.message,
]
.filter(Boolean)
.join("\n");
}
}