merge: upstream v0.25.0 into lotus

Merge upstream element-hq/element-call tag v0.25.0 into the Lotus fork
(previous base: v0.20.1; actual merge-base v0.20.1-rc.1). Every Lotus
feature and all six io.lotus.* widget actions are preserved. Version
bumped to 0.25.0-lotus.1.

Conflict files and how each Lotus hunk was re-expressed:

* src/state/CallViewModel/remoteMembers/ConnectionFactory.ts
  Upstream moved echoCancellation/noiseSuppression/autoGainControl from
  constructor params (fed by URL params) to persisted Settings
  (settings.ts) with a developer-settings UI. The cinny host still drives
  these per call via URL params (noiseSuppression=false /
  autoGainControl=false when the in-source ML denoiser is active, so the
  model gets a raw mic) - taking upstream verbatim would silently break
  the ML denoise tier. Re-wired as AND semantics in generateRoomOption():
  a constraint is enabled only if BOTH the Setting and the URL param
  allow it. Params default to true, so with no params this is
  byte-for-byte upstream behaviour. Upstream's own echoCancellation /
  noiseSuppression URL params (still parsed but dead in v0.25.0) work
  again as a side effect. Lotus autoGainControl URL param kept in
  UrlParams.ts (auto-merged, unchanged).

* src/state/CallViewModel/remoteMembers/ECConnectionFactory.test.ts
  Took upstream (tests now drive via Settings). The lost Lotus coverage
  is restored in a NEW colocated file src/lotus/lotusAudioConstraints.test.ts
  (3 tests) so the upstream test file stays pristine. Verified the new
  test fails against pure-upstream ConnectionFactory and passes with the
  re-wiring.

* src/state/CallViewModel/CallViewModel.ts
  Three small hunks: kept both the Lotus `userMedia$` interface member
  and upstream's new `keyRotationSuppressed$`; dropped the three Lotus
  audio constructor args (mechanism removed upstream, see above); kept
  both in the returned object. The [lotus #4] overrideSpotlight$ routing,
  manualSpotlightUserId$ and setManualSpotlight auto-merged; verified
  against upstream's changed ringingMedia$ (now single-or-null instead
  of array) - the merge correctly took upstream's outer branch and the
  inner screenShares$/spotlightSpeaker$ logic that lotusSpotlight.ts
  mirrors is unchanged upstream.

* src/index.css
  Kept both: Lotus lotus-transparent / lotus-theme blocks and upstream's
  new body[data-background="gradient"]::before full-viewport gradient.
  The naive merge swallowed the closing brace of body.lotus-theme -
  restored. Added a rule hiding the new gradient pseudo-element under
  body.lotus-transparent, since it would otherwise paint over the
  transparent body and hide the host wallpaper.

* src/components/CallFooterViewModel.tsx, src/components/CallFooter.stories.tsx
  No Lotus content - pure upstream-vs-upstream conflicts caused by the
  merge base being v0.20.1-rc.1. Took upstream (layoutMode ->
  layoutSwitchVm; setLayoutMode removed). No Lotus code uses
  setGridMode/layoutMode.

Non-conflicting but reviewed:

* src/widget.ts auto-merged cleanly. Upstream's removal of .well-known
  transport advertisement and the new RTC-transport capability request
  did not touch the action registration loop the LOTUS_TO_WIDGET_ACTIONS
  spread and widget.lazyActions ride on - nothing to re-wire.
* src/room/InCallView.tsx, src/useAudioContext.tsx, src/useTheme.ts,
  src/tile/MediaView.tsx(+.module.css), src/UrlParams.ts(+test),
  all *.module.css and .gitea/workflows/ci.yml auto-merged; each diff
  against v0.25.0 was checked to equal the original Lotus hunk.
* src/button/Button.module.css: the merge appended an exact duplicate
  of upstream's `.rotate`/`@keyframes spin` block (rc.1 merge-base
  artefact) - reset to upstream verbatim.
* src/grid/OneOnOnePortraitLayout.module.css was renamed upstream to
  OneOnOneMobileLayout.module.css; git followed the rename and the Lotus
  safe-area PiP inset fix applies there (the --content-inset-* vars it
  uses still exist upstream).

Tooling changes inherited from upstream that affect the fork:

* eslint + prettier were replaced by oxlint + oxfmt (`pnpm lint:oxlint`,
  `pnpm format:check`). oxlint flagged 10 issues, all in src/lotus/*:
  8x no-meaningless-void-operator (dropped the `void` before void-typed
  widget transport.reply / callbacks - no behaviour change), 1x
  consistent-type-imports (lotusWidget.ts: `import type`), and 2x
  unicorn/no-useless-spread in lotusAudioInject.ts which are FALSE
  POSITIVES - `[...activeClips]` is a required defensive copy because
  abort() deletes from the Set during iteration; suppressed with an
  explanatory eslint-disable-next-line. oxfmt reformatted 7 Lotus
  touched files (whitespace only).
* packageManager bumped by upstream to pnpm@11.21.0, which requires
  Node >= 22.13 (uses node:sqlite). Node 20 cannot run it; pnpm 10.33
  cannot read the new lockfile either (matrix-js-sdk is now a git
  dependency on develop, using a version-union pnpm 10 rejects). Fork CI
  already uses Node 24 (.node-version), so CI is unaffected.
* matrix-js-sdk is now github:matrix-org/matrix-js-sdk#develop (pinned
  by commit in pnpm-lock.yaml).

Lotus behaviour NOT preserved: none found.

Verification (Node 24.11.1, pnpm 11.21.0): pnpm install --frozen-lockfile
OK (lockfile taken from upstream unchanged, no regeneration needed);
tsc clean; oxlint clean; oxfmt --check clean; knip exit 0 (2 config
hints in upstream knip.ts only); vitest unit 84 files / 627 passed /
9 skipped; build:embedded OK, staged to embedded/web/dist (44M), all
six io.lotus.* action strings present in the bundle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
This commit is contained in:
Lotus CI
2026-09-12 11:35:57 -04:00
co-authored by Claude Fable 5.1
243 changed files with 13311 additions and 10280 deletions
+130
View File
@@ -0,0 +1,130 @@
/*
Copyright 2026 Lotus Guild
SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
Please see LICENSE in the repository root for full details.
*/
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { Room as LivekitRoom } from "livekit-client";
import { BehaviorSubject } from "rxjs";
import { logger } from "matrix-js-sdk/lib/logger";
import EventEmitter from "events";
import { ObservableScope } from "../state/ObservableScope.ts";
import { ECConnectionFactory } from "../state/CallViewModel/remoteMembers/ConnectionFactory.ts";
import type { OpenIDClientParts } from "../livekit/openIDSFU.ts";
import {
exampleTransport,
mockMediaDevices,
ownMemberMock,
} from "../utils/test.ts";
import type { ProcessorState } from "../livekit/TrackProcessorContext.tsx";
import {
autoGainControlSetting,
echoCancellationSetting,
noiseSuppressionSetting,
} from "../settings/settings.ts";
// [lotus] Upstream v0.25.0 moved the audio-capture constraints to Settings.
// The Lotus host still drives them per-call via URL params (it turns browser
// noiseSuppression/AGC OFF for the in-source ML denoise tier), so
// ConnectionFactory ANDs the Setting with the URL param. These tests pin that
// contract.
const { getUrlParams } = vi.hoisted(() => ({ getUrlParams: vi.fn() }));
vi.mock("../UrlParams", () => ({ getUrlParams }));
vi.mock("livekit-client", async (importOriginal) => ({
...(await importOriginal()),
Room: vi.fn().mockImplementation(function (this: LivekitRoom) {
const emitter = new EventEmitter();
return {
on: emitter.on.bind(emitter),
off: emitter.off.bind(emitter),
emit: emitter.emit.bind(emitter),
disconnect: vi.fn(),
remoteParticipants: new Map(),
} as unknown as LivekitRoom;
}),
}));
let testScope: ObservableScope;
const mockClient: OpenIDClientParts = {
getOpenIdToken: vi.fn().mockReturnValue(""),
getDeviceId: vi.fn().mockReturnValue("DEV000"),
};
function createRoom(): void {
new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
).createConnection(testScope, exampleTransport, ownMemberMock, logger);
}
function capturedAudioDefaults(): Record<string, unknown> {
const RoomConstructor = vi.mocked(LivekitRoom);
const options = RoomConstructor.mock.calls.at(-1)?.[0];
return (options?.audioCaptureDefaults ?? {}) as Record<string, unknown>;
}
beforeEach(() => {
testScope = new ObservableScope();
echoCancellationSetting.setValue(true);
noiseSuppressionSetting.setValue(true);
autoGainControlSetting.setValue(true);
});
afterEach(() => {
testScope.end();
vi.mocked(LivekitRoom).mockClear();
});
describe("[lotus] audio-capture URL param overrides", () => {
test("with params defaulted to true, the Settings govern (upstream behaviour)", () => {
getUrlParams.mockReturnValue({
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
});
autoGainControlSetting.setValue(false);
createRoom();
expect(capturedAudioDefaults()).toMatchObject({
echoCancellation: true,
noiseSuppression: true,
autoGainControl: false,
});
});
test("host-passed noiseSuppression=false / autoGainControl=false force OFF regardless of Settings", () => {
getUrlParams.mockReturnValue({
echoCancellation: true,
noiseSuppression: false,
autoGainControl: false,
});
createRoom();
expect(capturedAudioDefaults()).toMatchObject({
echoCancellation: true,
noiseSuppression: false,
autoGainControl: false,
});
});
test("a URL param cannot force a constraint ON when the Setting is off", () => {
getUrlParams.mockReturnValue({
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
});
echoCancellationSetting.setValue(false);
createRoom();
expect(capturedAudioDefaults()).toMatchObject({ echoCancellation: false });
});
});
+14 -2
View File
@@ -55,7 +55,7 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
// Always ack so the transport doesn't hang, but only act when the host has
// explicitly opted in: audio-inject publishes under the local user's
// identity, so it must not be silently armed for every call.
void w.api.transport.reply(ev.detail, {});
w.api.transport.reply(ev.detail, {});
if (!lotusFlag("lotusAudioInject")) return;
const data = ev.detail.data as
| { url?: unknown; volume?: unknown }
@@ -79,6 +79,12 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
sub.unsubscribe();
w.lazyActions.off(LotusWidgetActions.InjectAudio, handler);
// Abort anything still playing.
// oxlint-disable-next-line unicorn/no-useless-spread -- the spread is a
// required defensive copy: abort() deletes from activeClips while we
// iterate it.
// The spread is a required defensive copy: abort() deletes from
// activeClips while we iterate it.
// eslint-disable-next-line unicorn/no-useless-spread
for (const abort of [...activeClips]) abort();
};
}
@@ -106,6 +112,9 @@ async function playInjectedClip(
// Max ONE clip at a time (replace mode): stop any in-flight or playing clip
// before starting a new one, so clips can't overlap or be spammed.
// The spread is a required defensive copy: abort() deletes from
// activeClips while we iterate it.
// eslint-disable-next-line unicorn/no-useless-spread
for (const abort of [...activeClips]) abort();
// A second inject action can arrive while THIS one is still awaiting its
@@ -240,7 +249,10 @@ async function playInjectedClip(
const durationMs = Number.isFinite(buffer.duration)
? buffer.duration * 1000 + 500
: MAX_CLIP_MS;
const guard = setTimeout(cleanup, Math.min(MAX_CLIP_MS, Math.max(0, durationMs)));
const guard = setTimeout(
cleanup,
Math.min(MAX_CLIP_MS, Math.max(0, durationMs)),
);
source.addEventListener("ended", () => clearTimeout(guard));
source.start();
+11 -2
View File
@@ -6,7 +6,12 @@ Please see LICENSE in the repository root for full details.
*/
import { combineLatest, of, type Subscription } from "rxjs";
import { distinctUntilChanged, map, switchMap, throttleTime } from "rxjs/operators";
import {
distinctUntilChanged,
map,
switchMap,
throttleTime,
} from "rxjs/operators";
import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
import { LotusWidgetActions, lotusFlag, lotusSendToHost } from "./lotusWidget";
@@ -46,7 +51,11 @@ export function startLotusCallState(vm: CallViewModel): () => void {
m.videoEnabled$,
]).pipe(
map(
([speaking, audioEnabled, videoEnabled]): ParticipantState => ({
([
speaking,
audioEnabled,
videoEnabled,
]): ParticipantState => ({
id: m.id,
userId: m.userId,
speaking,
+1 -1
View File
@@ -95,7 +95,7 @@ export function startLotusDeafen(vm: CallViewModel): () => void {
});
const handler = (ev: CustomEvent<IWidgetApiRequest>): void => {
void w.api.transport.reply(ev.detail, {});
w.api.transport.reply(ev.detail, {});
const data = ev.detail.data as
| { deafened?: boolean; screenshareAudioMuted?: boolean }
| undefined;
+1 -1
View File
@@ -65,7 +65,7 @@ export function startLotusDecorations(): () => void {
if (registrations === 0) {
const handler = (ev: CustomEvent<IWidgetApiRequest>): void => {
void w.api.transport.reply(ev.detail, {});
w.api.transport.reply(ev.detail, {});
const data = ev.detail.data as
| { decorations?: Record<string, unknown> }
| undefined;
+31 -17
View File
@@ -12,11 +12,7 @@ import {
} from "livekit-client";
import { logger } from "matrix-js-sdk/lib/logger";
export type LotusDenoiseModel =
| "rnnoise"
| "speex"
| "dtln"
| "deepfilternet";
export type LotusDenoiseModel = "rnnoise" | "speex" | "dtln" | "deepfilternet";
export interface LotusDenoiseConfig {
model: LotusDenoiseModel;
@@ -111,8 +107,8 @@ function supportsSimd(): boolean {
// Minimal SIMD module (v128) — validates only where SIMD is supported.
return WebAssembly.validate(
new Uint8Array([
0, 97, 115, 109, 1, 0, 0, 0, 1, 5, 1, 96, 0, 1, 123, 3, 2, 1, 0, 10,
10, 1, 8, 0, 65, 0, 253, 15, 253, 98, 11,
0, 97, 115, 109, 1, 0, 0, 0, 1, 5, 1, 96, 0, 1, 123, 3, 2, 1, 0, 10, 10,
1, 8, 0, 65, 0, 253, 15, 253, 98, 11,
]),
);
} catch {
@@ -145,9 +141,10 @@ interface Graph {
* stopped track on the sender: on failure it degrades to the RAW mic track
* rather than silence.
*/
export class LotusDenoiseProcessor
implements TrackProcessor<Track.Kind.Audio, AudioProcessorOptions>
{
export class LotusDenoiseProcessor implements TrackProcessor<
Track.Kind.Audio,
AudioProcessorOptions
> {
public readonly name = "lotus-denoise";
public processedTrack?: MediaStreamTrack;
@@ -209,7 +206,11 @@ export class LotusDenoiseProcessor
/** Create (once) the model-rate context + register the flat worklet modules. */
private async ensureContext(): Promise<void> {
const rate = sampleRateFor(this.config.model);
if (this.ctx && this.ctx.state !== "closed" && this.ctx.sampleRate === rate) {
if (
this.ctx &&
this.ctx.state !== "closed" &&
this.ctx.sampleRate === rate
) {
if (this.ctx.state === "suspended") await resumeCtx(this.ctx);
return;
}
@@ -316,7 +317,12 @@ export class LotusDenoiseProcessor
logger.info(
`[lotus] denoise processor active (${this.config.model}, floor=${floor})`,
);
return { source, nodes, disposes, track: dest.stream.getAudioTracks()[0] };
return {
source,
nodes,
disposes,
track: dest.stream.getAudioTracks()[0],
};
} catch (e) {
// A node constructor / model load can throw mid-build; clean up the
// partially-built graph so it doesn't leak (init/restart still fall back
@@ -338,15 +344,20 @@ export class LotusDenoiseProcessor
if (model === "dtln") {
// Self-contained ESM that resolves its own processor + LiteRT wasm +
// TFLite models. bypassUntilReady passes raw audio until the model loads.
const mod = await import(/* @vite-ignore */ `${base}workadventure/audio-worklet.js`);
const mod = await import(
/* @vite-ignore */ `${base}workadventure/audio-worklet.js`
);
return (await mod.createNoiseSuppressionAudioWorklet(ctx, {
bypassUntilReady: true,
})) as MlNode;
}
if (model === "deepfilternet") {
const dfnBase = new URL(`${base}deepfilternet`, window.location.href).href;
const mod = await import(/* @vite-ignore */ `${base}deepfilternet/index.esm.js`);
const dfnBase = new URL(`${base}deepfilternet`, window.location.href)
.href;
const mod = await import(
/* @vite-ignore */ `${base}deepfilternet/index.esm.js`
);
const core = new mod.DeepFilterNet3Core({
sampleRate: 48_000,
// 60, not 80: full-strength suppression is the main source of the
@@ -357,7 +368,7 @@ export class LotusDenoiseProcessor
});
await core.initialize();
const node = (await core.createAudioWorkletNode(ctx)) as AudioNode;
return { node, dispose: () => void safeCall(() => core.destroy()) };
return { node, dispose: () => safeCall(() => core.destroy()) };
}
// Flat sapphi worklet (rnnoise/speex).
@@ -379,7 +390,10 @@ export class LotusDenoiseProcessor
numberOfOutputs: 1,
processorOptions: { maxChannels: 1, wasmBinary },
});
return { node, dispose: () => void safeCall(() => node.port.postMessage("destroy")) };
return {
node,
dispose: () => safeCall(() => node.port.postMessage("destroy")),
};
}
private disposeGraph(graph: Graph | undefined): void {
+2 -3
View File
@@ -26,7 +26,7 @@ export function startLotusFocus(vm: CallViewModel): () => void {
const handler = (ev: CustomEvent<IWidgetApiRequest>): void => {
// Always reply so the host transport doesn't time out.
void w.api.transport.reply(ev.detail, {});
w.api.transport.reply(ev.detail, {});
const data = ev.detail.data as { userId?: unknown } | undefined;
// Mirror deafen's partial-payload semantics: a payload that OMITS `userId`
// must keep the current spotlight, not clear it. Only act when the key is
@@ -38,6 +38,5 @@ export function startLotusFocus(vm: CallViewModel): () => void {
};
w.lazyActions.on(LotusWidgetActions.FocusParticipant, handler);
return () =>
w.lazyActions.off(LotusWidgetActions.FocusParticipant, handler);
return () => w.lazyActions.off(LotusWidgetActions.FocusParticipant, handler);
}
+1 -1
View File
@@ -135,7 +135,7 @@ export function startLotusQuality(vm: CallViewModel): () => void {
});
const handler = (ev: CustomEvent<IWidgetApiRequest>): void => {
void w.api.transport.reply(ev.detail, {});
w.api.transport.reply(ev.detail, {});
const data = ev.detail.data as Record<string, unknown> | undefined;
if (!data) return;
// Clamp to sane ranges so a typo can't brick the encoder (e.g. a 1 bps mic).
+13 -6
View File
@@ -17,7 +17,7 @@ Please see LICENSE in the repository root for full details.
import { logger } from "matrix-js-sdk/lib/logger";
import { widget } from "../widget";
import { LotusWidgetActions } from "./lotusActions";
import type { LotusWidgetActions } from "./lotusActions";
export { LotusWidgetActions } from "./lotusActions";
@@ -33,7 +33,9 @@ export function lotusParam(name: string): string | null {
// Match EC's own ParamParser precedence: the hash fragment wins over the
// query string. So seed from the fragment first, then fill gaps from query.
const hash = window.location.hash.replace(/^#\/?/, "");
const hashQuery = hash.includes("?") ? hash.slice(hash.indexOf("?") + 1) : "";
const hashQuery = hash.includes("?")
? hash.slice(hash.indexOf("?") + 1)
: "";
cachedParams = new URLSearchParams(hashQuery);
for (const [k, v] of new URLSearchParams(window.location.search)) {
if (!cachedParams.has(k)) cachedParams.append(k, v);
@@ -53,11 +55,16 @@ export function lotusFlag(name: string): boolean {
* rejection when the host hasn't (yet) registered a handler for it. Returns
* true if the widget transport was available to attempt the send.
*/
export function lotusSendToHost(action: LotusWidgetActions, data: unknown): boolean {
export function lotusSendToHost(
action: LotusWidgetActions,
data: unknown,
): boolean {
const api = widget?.api;
if (!api) return false;
void api.transport.send(action, data as Record<string, unknown>).catch((e) => {
logger.debug(`[lotus] host did not ack ${action}`, e);
});
void api.transport
.send(action, data as Record<string, unknown>)
.catch((e) => {
logger.debug(`[lotus] host did not ack ${action}`, e);
});
return true;
}