merge: upstream v0.25.0 into lotus

Merge upstream element-hq/element-call tag v0.25.0 into the Lotus fork
(previous base: v0.20.1; actual merge-base v0.20.1-rc.1). Every Lotus
feature and all six io.lotus.* widget actions are preserved. Version
bumped to 0.25.0-lotus.1.

Conflict files and how each Lotus hunk was re-expressed:

* src/state/CallViewModel/remoteMembers/ConnectionFactory.ts
  Upstream moved echoCancellation/noiseSuppression/autoGainControl from
  constructor params (fed by URL params) to persisted Settings
  (settings.ts) with a developer-settings UI. The cinny host still drives
  these per call via URL params (noiseSuppression=false /
  autoGainControl=false when the in-source ML denoiser is active, so the
  model gets a raw mic) - taking upstream verbatim would silently break
  the ML denoise tier. Re-wired as AND semantics in generateRoomOption():
  a constraint is enabled only if BOTH the Setting and the URL param
  allow it. Params default to true, so with no params this is
  byte-for-byte upstream behaviour. Upstream's own echoCancellation /
  noiseSuppression URL params (still parsed but dead in v0.25.0) work
  again as a side effect. Lotus autoGainControl URL param kept in
  UrlParams.ts (auto-merged, unchanged).

* src/state/CallViewModel/remoteMembers/ECConnectionFactory.test.ts
  Took upstream (tests now drive via Settings). The lost Lotus coverage
  is restored in a NEW colocated file src/lotus/lotusAudioConstraints.test.ts
  (3 tests) so the upstream test file stays pristine. Verified the new
  test fails against pure-upstream ConnectionFactory and passes with the
  re-wiring.

* src/state/CallViewModel/CallViewModel.ts
  Three small hunks: kept both the Lotus `userMedia$` interface member
  and upstream's new `keyRotationSuppressed$`; dropped the three Lotus
  audio constructor args (mechanism removed upstream, see above); kept
  both in the returned object. The [lotus #4] overrideSpotlight$ routing,
  manualSpotlightUserId$ and setManualSpotlight auto-merged; verified
  against upstream's changed ringingMedia$ (now single-or-null instead
  of array) - the merge correctly took upstream's outer branch and the
  inner screenShares$/spotlightSpeaker$ logic that lotusSpotlight.ts
  mirrors is unchanged upstream.

* src/index.css
  Kept both: Lotus lotus-transparent / lotus-theme blocks and upstream's
  new body[data-background="gradient"]::before full-viewport gradient.
  The naive merge swallowed the closing brace of body.lotus-theme -
  restored. Added a rule hiding the new gradient pseudo-element under
  body.lotus-transparent, since it would otherwise paint over the
  transparent body and hide the host wallpaper.

* src/components/CallFooterViewModel.tsx, src/components/CallFooter.stories.tsx
  No Lotus content - pure upstream-vs-upstream conflicts caused by the
  merge base being v0.20.1-rc.1. Took upstream (layoutMode ->
  layoutSwitchVm; setLayoutMode removed). No Lotus code uses
  setGridMode/layoutMode.

Non-conflicting but reviewed:

* src/widget.ts auto-merged cleanly. Upstream's removal of .well-known
  transport advertisement and the new RTC-transport capability request
  did not touch the action registration loop the LOTUS_TO_WIDGET_ACTIONS
  spread and widget.lazyActions ride on - nothing to re-wire.
* src/room/InCallView.tsx, src/useAudioContext.tsx, src/useTheme.ts,
  src/tile/MediaView.tsx(+.module.css), src/UrlParams.ts(+test),
  all *.module.css and .gitea/workflows/ci.yml auto-merged; each diff
  against v0.25.0 was checked to equal the original Lotus hunk.
* src/button/Button.module.css: the merge appended an exact duplicate
  of upstream's `.rotate`/`@keyframes spin` block (rc.1 merge-base
  artefact) - reset to upstream verbatim.
* src/grid/OneOnOnePortraitLayout.module.css was renamed upstream to
  OneOnOneMobileLayout.module.css; git followed the rename and the Lotus
  safe-area PiP inset fix applies there (the --content-inset-* vars it
  uses still exist upstream).

Tooling changes inherited from upstream that affect the fork:

* eslint + prettier were replaced by oxlint + oxfmt (`pnpm lint:oxlint`,
  `pnpm format:check`). oxlint flagged 10 issues, all in src/lotus/*:
  8x no-meaningless-void-operator (dropped the `void` before void-typed
  widget transport.reply / callbacks - no behaviour change), 1x
  consistent-type-imports (lotusWidget.ts: `import type`), and 2x
  unicorn/no-useless-spread in lotusAudioInject.ts which are FALSE
  POSITIVES - `[...activeClips]` is a required defensive copy because
  abort() deletes from the Set during iteration; suppressed with an
  explanatory eslint-disable-next-line. oxfmt reformatted 7 Lotus
  touched files (whitespace only).
* packageManager bumped by upstream to pnpm@11.21.0, which requires
  Node >= 22.13 (uses node:sqlite). Node 20 cannot run it; pnpm 10.33
  cannot read the new lockfile either (matrix-js-sdk is now a git
  dependency on develop, using a version-union pnpm 10 rejects). Fork CI
  already uses Node 24 (.node-version), so CI is unaffected.
* matrix-js-sdk is now github:matrix-org/matrix-js-sdk#develop (pinned
  by commit in pnpm-lock.yaml).

Lotus behaviour NOT preserved: none found.

Verification (Node 24.11.1, pnpm 11.21.0): pnpm install --frozen-lockfile
OK (lockfile taken from upstream unchanged, no regeneration needed);
tsc clean; oxlint clean; oxfmt --check clean; knip exit 0 (2 config
hints in upstream knip.ts only); vitest unit 84 files / 627 passed /
9 skipped; build:embedded OK, staged to embedded/web/dist (44M), all
six io.lotus.* action strings present in the bundle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
This commit is contained in:
Lotus CI
2026-09-12 11:35:57 -04:00
co-authored by Claude Fable 5.1
243 changed files with 13311 additions and 10280 deletions
@@ -12,8 +12,9 @@ import {
} from "@livekit/components-core";
import {
ConnectionError,
type Room as LivekitRoom,
ConnectionErrorReason,
type RemoteParticipant,
type Room as LivekitRoom,
} from "livekit-client";
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
import { BehaviorSubject, map } from "rxjs";
@@ -30,10 +31,11 @@ import { type ObservableScope } from "../../ObservableScope.ts";
import {
ElementCallError,
InsufficientCapacityError,
LivekitConnectionError,
PeerConnectionTimeoutError,
SFURoomCreationRestrictedError,
UnknownCallError,
} from "../../../utils/errors.ts";
import { type JwtEndpointVersion } from "../localMember/LocalTransport.ts";
export interface ConnectionOpts {
/**
@@ -41,11 +43,6 @@ export interface ConnectionOpts {
* On top the local transport will send additional data to the jwt server to use delayed event delegation.
*/
existingSFUConfig?: SFUConfig;
/**
* For local connections that use the oldest member pattern. here we have not prefetched the sfuConfig
* and hence we need to let the connection do the jwt token fetching.
*/
forceJwtEndpoint?: JwtEndpointVersion;
/** The identity parts to use on this connection */
ownMembershipIdentity: CallMembershipIdentityParts;
/** The media transport to connect to. */
@@ -248,6 +245,13 @@ export class Connection {
// In the first case there will not be a 404, so we are in the second case.
throw new SFURoomCreationRestrictedError();
}
if (e.reason === ConnectionErrorReason.Timeout) {
// Unabled to establish peer connection within the timeout
throw new PeerConnectionTimeoutError();
}
throw new LivekitConnectionError(e);
}
throw e;
}
@@ -12,7 +12,7 @@ import {
type E2EEManagerOptions,
type BaseE2EEManager,
} from "livekit-client";
import { type Logger } from "matrix-js-sdk/lib/logger";
import { logger, type Logger } from "matrix-js-sdk/lib/logger";
// imported as inline to support worker when loaded from a cdn (cross domain)
import E2EEWorker from "livekit-client/e2ee-worker?worker&inline";
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
@@ -27,7 +27,26 @@ import type {
import type { MediaDevices } from "../../MediaDevices.ts";
import type { Behavior } from "../../Behavior.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { defaultLiveKitOptions } from "../../../livekit/options.ts";
import { getLiveKitOptions } from "../../../livekit/options.ts";
import {
advancedCamera,
cameraResolution,
cameraFramerate,
cameraBitrate,
cameraCodec,
parseResolution,
echoCancellationSetting,
noiseSuppressionSetting,
autoGainControlSetting,
} from "../../../settings/settings.ts";
// [lotus] Upstream v0.25.0 moved the audio-capture constraints from URL params
// to persisted Settings. The Lotus host (cinny) still drives them per-call via
// URL params (it turns browser noiseSuppression/AGC OFF when the in-source ML
// denoiser is active, so the model gets a raw mic). Re-wire: a constraint is
// enabled only if BOTH the Setting and the URL param allow it. The params
// default to true, so with no params this is byte-for-byte upstream behaviour;
// `?autoGainControl=false` forces AGC off regardless of the stored Setting.
import { getUrlParams } from "../../../UrlParams.ts";
// TODO evaluate if this should be done like the Publisher Factory
export interface ConnectionFactory {
@@ -53,9 +72,6 @@ export class ECConnectionFactory implements ConnectionFactory {
* @param livekitKeyProvider - Optional key provider for end-to-end encryption.
* @param controlledAudioDevices - Option to indicate whether audio output device is controlled externally (native mobile app).
* @param livekitRoomFactory - Optional factory function (for testing) to create LivekitRoom instances. If not provided, a default factory is used.
* @param echoCancellation - Whether to enable echo cancellation for audio capture.
* @param noiseSuppression - Whether to enable noise suppression for audio capture.
* @param autoGainControl - Whether to enable auto gain control for audio capture.
*/
public constructor(
private client: OpenIDClientParts,
@@ -65,27 +81,22 @@ export class ECConnectionFactory implements ConnectionFactory {
livekitKeyProvider: BaseKeyProvider | undefined,
private controlledAudioDevices: boolean,
livekitRoomFactory?: () => LivekitRoom,
echoCancellation: boolean = true,
noiseSuppression: boolean = true,
autoGainControl: boolean = true,
) {
const defaultFactory = (): LivekitRoom =>
new LivekitRoom(
generateRoomOption({
devices: this.devices,
processorState: this.processorState$.value,
e2eeLivekitOptions: livekitKeyProvider && {
keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms.
worker: new E2EEWorker(),
},
controlledAudioDevices: this.controlledAudioDevices,
echoCancellation,
noiseSuppression,
autoGainControl,
}),
);
const defaultFactory = (): LivekitRoom => {
const roomOptions = generateRoomOption({
devices: this.devices,
processorState: this.processorState$.value,
e2eeLivekitOptions: livekitKeyProvider && {
keyProvider: livekitKeyProvider,
// It's important that every room use a separate E2EE worker.
// They get confused if given streams from multiple rooms.
worker: new E2EEWorker(),
},
controlledAudioDevices: this.controlledAudioDevices,
});
logger.info("[ECConnectionFactory] livekit room options: ", roomOptions);
return new LivekitRoom(roomOptions);
};
this.livekitRoomFactory = livekitRoomFactory ?? defaultFactory;
}
@@ -122,15 +133,13 @@ export class ECConnectionFactory implements ConnectionFactory {
/**
* Generate the initial LiveKit RoomOptions based on the current media devices and processor state.
* Reads audio processing and camera quality settings directly from Settings.
*/
function generateRoomOption({
devices,
processorState,
e2eeLivekitOptions,
controlledAudioDevices,
echoCancellation,
noiseSuppression,
autoGainControl,
}: {
devices: MediaDevices;
processorState: ProcessorState;
@@ -139,23 +148,50 @@ function generateRoomOption({
| { e2eeManager: BaseE2EEManager }
| undefined;
controlledAudioDevices: boolean;
echoCancellation: boolean;
noiseSuppression: boolean;
autoGainControl: boolean;
}): RoomOptions {
const liveKitOptions = getLiveKitOptions();
// Apply advanced camera settings if enabled
let videoCaptureDefaults = {
...liveKitOptions.videoCaptureDefaults,
deviceId: devices.videoInput.selected$.value?.id,
processor: processorState.processor,
};
let publishDefaults = liveKitOptions.publishDefaults;
if (advancedCamera.getValue()) {
const { width, height } = parseResolution(cameraResolution.getValue());
const fps = cameraFramerate.getValue();
const bps = cameraBitrate.getValue();
const codec = cameraCodec.getValue();
videoCaptureDefaults = {
...videoCaptureDefaults,
resolution: { width, height, frameRate: fps },
};
publishDefaults = {
...publishDefaults,
videoEncoding: { maxBitrate: bps, maxFramerate: fps },
videoCodec: codec,
};
}
return {
...defaultLiveKitOptions,
videoCaptureDefaults: {
...defaultLiveKitOptions.videoCaptureDefaults,
deviceId: devices.videoInput.selected$.value?.id,
processor: processorState.processor,
},
...liveKitOptions,
videoCaptureDefaults,
publishDefaults,
audioCaptureDefaults: {
...defaultLiveKitOptions.audioCaptureDefaults,
...liveKitOptions.audioCaptureDefaults,
deviceId: devices.audioInput.selected$.value?.id,
echoCancellation,
noiseSuppression,
autoGainControl,
echoCancellation:
echoCancellationSetting.getValue() &&
getUrlParams().echoCancellation !== false,
noiseSuppression:
noiseSuppressionSetting.getValue() &&
getUrlParams().noiseSuppression !== false,
autoGainControl:
autoGainControlSetting.getValue() &&
getUrlParams().autoGainControl !== false,
},
audioOutput: {
// When using controlled audio devices, we don't want to set the
@@ -22,6 +22,16 @@ import {
} from "../../../utils/test.ts";
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import { constant } from "../../Behavior";
import {
echoCancellationSetting,
noiseSuppressionSetting,
autoGainControlSetting,
advancedCamera,
cameraResolution,
cameraFramerate,
cameraBitrate,
cameraCodec,
} from "../../../settings/settings.ts";
// At the top of your test file, after imports
vi.mock("livekit-client", async (importOriginal) => {
@@ -53,15 +63,19 @@ beforeEach(() => {
describe("ECConnectionFactory - Audio inputs options", () => {
test.each([
{ echo: true, noise: true, agc: true },
{ echo: true, noise: false, agc: false },
{ echo: false, noise: true, agc: false },
{ echo: false, noise: false, agc: true },
{ echo: true, noise: true },
{ echo: true, noise: false },
{ echo: false, noise: true },
{ echo: false, noise: false },
])(
"it sets echoCancellation=$echo, noiseSuppression=$noise, autoGainControl=$agc based on constructor parameters",
({ echo, noise, agc }) => {
"it sets echoCancellation=$echo and noiseSuppression=$noise based on settings",
({ echo, noise }) => {
const RoomConstructor = vi.mocked(LivekitRoom);
// Set audio processing settings
echoCancellationSetting.setValue(echo);
noiseSuppressionSetting.setValue(noise);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
@@ -72,10 +86,6 @@ describe("ECConnectionFactory - Audio inputs options", () => {
}),
undefined,
false,
undefined,
echo,
noise,
agc,
);
ecConnectionFactory.createConnection(
testScope,
@@ -90,7 +100,6 @@ describe("ECConnectionFactory - Audio inputs options", () => {
audioCaptureDefaults: expect.objectContaining({
echoCancellation: echo,
noiseSuppression: noise,
autoGainControl: agc,
}),
}),
);
@@ -102,9 +111,13 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
test.each([{ controlled: true }, { controlled: false }])(
"it sets controlledAudioDevice=$controlled then uses deviceId accordingly",
({ controlled }) => {
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
// Explicitly set audio settings so the test doesn't depend on defaults
echoCancellationSetting.setValue(true);
noiseSuppressionSetting.setValue(true);
autoGainControlSetting.setValue(true);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
@@ -121,9 +134,6 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
}),
undefined,
controlled,
undefined,
false,
false,
);
ecConnectionFactory.createConnection(
testScope,
@@ -144,6 +154,114 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
);
});
describe("ECConnectionFactory - Camera quality settings", () => {
test("it uses default video options when advancedCamera is disabled", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
advancedCamera.setValue(false);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
);
ecConnectionFactory.createConnection(
testScope,
exampleTransport,
ownMemberMock,
logger,
);
// publishDefaults should use config defaults (vp8), not custom settings
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
publishDefaults: expect.objectContaining({
videoCodec: "vp8",
}),
}),
);
});
test("it applies custom camera resolution, encoding, and codec when advancedCamera is enabled", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
advancedCamera.setValue(true);
cameraResolution.setValue("1920x1080");
cameraFramerate.setValue(60);
cameraBitrate.setValue(4_000_000);
cameraCodec.setValue("vp9");
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
);
ecConnectionFactory.createConnection(
testScope,
exampleTransport,
ownMemberMock,
logger,
);
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
videoCaptureDefaults: expect.objectContaining({
resolution: { width: 1920, height: 1080, frameRate: 60 },
}),
publishDefaults: expect.objectContaining({
videoEncoding: { maxBitrate: 4_000_000, maxFramerate: 60 },
videoCodec: "vp9",
}),
}),
);
});
test("it applies autoGainControl from settings", () => {
const RoomConstructor = vi.mocked(LivekitRoom);
autoGainControlSetting.setValue(false);
echoCancellationSetting.setValue(true);
noiseSuppressionSetting.setValue(true);
const ecConnectionFactory = new ECConnectionFactory(
mockClient,
"!roomid:example.org",
mockMediaDevices({}),
new BehaviorSubject<ProcessorState>({
supported: true,
processor: undefined,
}),
undefined,
false,
);
ecConnectionFactory.createConnection(
testScope,
exampleTransport,
ownMemberMock,
logger,
);
expect(RoomConstructor).toHaveBeenCalledWith(
expect.objectContaining({
audioCaptureDefaults: expect.objectContaining({
autoGainControl: false,
}),
}),
);
});
});
afterEach(() => {
testScope.end();
fetchMock.reset();
@@ -15,7 +15,7 @@ import { BehaviorSubject, combineLatest, map, type Observable } from "rxjs";
import { type IConnectionManager } from "./ConnectionManager.ts";
import {
type RemoteMatrixLivekitMember,
createMatrixLivekitMembers$,
createRemoteMatrixLivekitMembers$,
} from "./MatrixLivekitMembers.ts";
import {
Epoch,
@@ -31,6 +31,7 @@ import {
} from "../../../utils/test.ts";
import { type Connection } from "./Connection.ts";
import { constant } from "../../Behavior.ts";
import { localRtcMember } from "../../../utils/test-fixtures.ts";
let testScope: ObservableScope;
@@ -88,16 +89,17 @@ test("should signal participant not yet connected to livekit", async () => {
mockConnectionManagerData$,
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
localUser: localRtcMember,
});
await flushPromises();
expect(matrixLivekitMember$.value.value).toSatisfy(
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
(data: RemoteMatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].membership$.value).toBe(bobMembership);
@@ -157,16 +159,17 @@ test("should signal participant on a connection that is publishing", async () =>
constant(dataWithPublisher),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
localUser: localRtcMember,
});
await flushPromises();
expect(matrixLivekitMember$.value.value).toSatisfy(
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
(data: RemoteMatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].membership$.value).toBe(bobMembership);
@@ -197,15 +200,16 @@ test("should signal participant on a connection that is not publishing", async (
constant(dataWithPublisher),
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
localUser: localRtcMember,
});
await flushPromises();
expect(matrixLivekitMember$.value.value).toSatisfy(
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
(data: RemoteMatrixLivekitMember[]) => {
expect(data.length).toEqual(1);
expect(data[0].membership$.value).toBe(bobMembership);
@@ -245,15 +249,16 @@ describe("Publication edge case", () => {
constant(connectionWithPublisher),
);
const matrixLivekitMembers$ = createMatrixLivekitMembers$({
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
localUser: localRtcMember,
});
await flushPromises();
expect(matrixLivekitMembers$.value.value).toSatisfy(
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
(data: RemoteMatrixLivekitMember[]) => {
expect(data.length).toEqual(2);
expect(data[0].membership$.value).toBe(bobMembership);
@@ -303,16 +308,17 @@ test("bob is publishing in the wrong connection", async () => {
connectionsWithPublisher$,
);
const matrixLivekitMember$ = createMatrixLivekitMembers$({
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
connectionManager: {
connectionManagerData$: connectionManagerData$,
} as unknown as IConnectionManager,
localUser: localRtcMember,
});
await flushPromises();
expect(matrixLivekitMember$.value.value).toSatisfy(
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
(data: RemoteMatrixLivekitMember[]) => {
expect(data.length).toEqual(2);
expect(data[0].membership$.value).toBe(bobMembership);
@@ -62,7 +62,9 @@ interface Props {
Epoch<{ membership: CallMembership; transport?: LivekitTransportConfig }[]>
>;
connectionManager: IConnectionManager;
localUser: { deviceId: string; userId: string };
}
/**
* Combines MatrixRTC and Livekit worlds.
*
@@ -73,13 +75,14 @@ interface Props {
* - out (via public Observable):
* - `remoteMatrixLivekitMember` an observable of MatrixLivekitMember[] to track the remote members and associated livekit data.
*/
export function createMatrixLivekitMembers$({
export function createRemoteMatrixLivekitMembers$({
scope,
membershipsWithTransport$,
connectionManager,
localUser,
}: Props): Behavior<Epoch<RemoteMatrixLivekitMember[]>> {
/**
* Stream of all the call members and their associated livekit data (if available).
* Behavior of all the remote call members and their associated livekit data (if available).
*/
return scope.behavior(
combineLatest([
@@ -91,12 +94,19 @@ export function createMatrixLivekitMembers$({
),
map(([ms, data]) => new Epoch([ms.value, data.value] as const, ms.epoch)),
generateItemsWithEpoch(
"MatrixLivekitMembers",
"RemoteMatrixLivekitMembers",
// Generator function.
// creates an array of `{key, data}[]`
// Each change in the keys (new key) will result in a call to the factory function.
function* ([membershipsWithTransport, managerData]) {
for (const { membership, transport } of membershipsWithTransport) {
// Exclude the local membership
if (
membership.userId === localUser.userId &&
membership.deviceId === localUser.deviceId
)
continue;
const participants = transport
? managerData.getParticipantsForTransport(transport)
: [];
@@ -105,7 +105,7 @@ describe("MatrixMemberMetadata", () => {
}
it("should show our own user if present in rtc session and room", () => {
withTestScheduler(({ behavior, expectObservable }) => {
withTestScheduler(({ scope, behavior, expectObservable }) => {
fakeMemberWith({
userId: "@local:example.com",
rawDisplayName: "it's a me",
@@ -118,8 +118,10 @@ describe("MatrixMemberMetadata", () => {
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const dn$ =
metadataStore.createDisplayNameBehavior$("@local:example.com");
const dn$ = metadataStore.createDisplayNameBehavior$(
scope,
"@local:example.com",
);
expectObservable(dn$).toBe("a", {
a: "it's a me",
@@ -146,7 +148,7 @@ describe("MatrixMemberMetadata", () => {
it("should get displayName for users", () => {
setUpBasicRoom();
withTestScheduler(({ behavior, expectObservable }) => {
withTestScheduler(({ scope, behavior, expectObservable }) => {
const memberships$ = behavior("a", {
a: [
mockRtcMembership("@alice:example.com", "DEVICE1"),
@@ -158,8 +160,10 @@ describe("MatrixMemberMetadata", () => {
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const aliceDispName$ =
metadataStore.createDisplayNameBehavior$("@alice:example.com");
const aliceDispName$ = metadataStore.createDisplayNameBehavior$(
scope,
"@alice:example.com",
);
expectObservable(aliceDispName$).toBe("a", {
a: "Alice",
@@ -322,7 +326,7 @@ describe("MatrixMemberMetadata", () => {
});
it("should track individual member id with createDisplayNameBehavior", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
withTestScheduler(({ scope, behavior, schedule, expectObservable }) => {
setUpBasicRoom();
const BOB = "@bob:example.com";
const CARL = "@carl:example.com";
@@ -356,8 +360,8 @@ describe("MatrixMemberMetadata", () => {
createRoomMembers$(testScope, mockMatrixRoom),
);
const bob$ = metadataStore.createDisplayNameBehavior$(BOB);
const carl$ = metadataStore.createDisplayNameBehavior$(CARL);
const bob$ = metadataStore.createDisplayNameBehavior$(scope, BOB);
const carl$ = metadataStore.createDisplayNameBehavior$(scope, CARL);
expectObservable(bob$).toBe("abc-", {
a: undefined,
@@ -378,7 +382,7 @@ describe("MatrixMemberMetadata", () => {
});
it("should disambiguate users with invisible characters", () => {
withTestScheduler(({ behavior, expectObservable }) => {
withTestScheduler(({ scope, behavior, expectObservable }) => {
const bobRtcMember = mockRtcMembership("@bob:example.org", "BBBB");
const bobZeroWidthSpaceRtcMember = mockRtcMembership(
"@bob2:example.org",
@@ -411,12 +415,18 @@ describe("MatrixMemberMetadata", () => {
createRoomMembers$(testScope, mockMatrixRoom),
);
const bob$ =
metadataStore.createDisplayNameBehavior$("@bob:example.org");
const bob2$ =
metadataStore.createDisplayNameBehavior$("@bob2:example.org");
const carol$ =
metadataStore.createDisplayNameBehavior$("@carol:example.org");
const bob$ = metadataStore.createDisplayNameBehavior$(
scope,
"@bob:example.org",
);
const bob2$ = metadataStore.createDisplayNameBehavior$(
scope,
"@bob2:example.org",
);
const carol$ = metadataStore.createDisplayNameBehavior$(
scope,
"@carol:example.org",
);
expectObservable(bob$).toBe("ab", {
a: "Bob",
b: "Bob (@bob:example.org)",
@@ -517,7 +527,7 @@ describe("MatrixMemberMetadata", () => {
}
it("should use avatar url from room members", () => {
withTestScheduler(({ behavior, expectObservable }) => {
withTestScheduler(({ scope, behavior, expectObservable }) => {
fakeMemberWith({
userId: "@local:example.com",
});
@@ -536,11 +546,15 @@ describe("MatrixMemberMetadata", () => {
memberships$,
createRoomMembers$(testScope, mockMatrixRoom),
);
const local$ =
metadataStore.createAvatarUrlBehavior$("@local:example.com");
const local$ = metadataStore.createAvatarUrlBehavior$(
scope,
"@local:example.com",
);
const alice$ =
metadataStore.createAvatarUrlBehavior$("@alice:example.com");
const alice$ = metadataStore.createAvatarUrlBehavior$(
scope,
"@alice:example.com",
);
expectObservable(local$).toBe("a", {
a: "mxc://example.com/@local:example.com",
@@ -558,7 +572,7 @@ describe("MatrixMemberMetadata", () => {
});
it("should update on avatar change and user join/leave", () => {
withTestScheduler(({ behavior, schedule, expectObservable }) => {
withTestScheduler(({ scope, behavior, schedule, expectObservable }) => {
fakeMemberWith({ userId: "@carl:example.com" });
fakeMemberWith({ userId: "@bob:example.com" });
const memberships$ = behavior("ab-d", {
@@ -585,9 +599,14 @@ describe("MatrixMemberMetadata", () => {
},
});
const bob$ = metadataStore.createAvatarUrlBehavior$("@bob:example.com");
const carl$ =
metadataStore.createAvatarUrlBehavior$("@carl:example.com");
const bob$ = metadataStore.createAvatarUrlBehavior$(
scope,
"@bob:example.com",
);
const carl$ = metadataStore.createAvatarUrlBehavior$(
scope,
"@carl:example.com",
);
expectObservable(bob$).toBe("a---", {
a: "mxc://example.com/@bob:example.com",
});
@@ -22,8 +22,6 @@ import {
} from "../../../utils/displayname";
import { type Behavior } from "../../Behavior";
const logger = rootLogger.getChild("[MatrixMemberMetadata]");
export type RoomMemberMap = Map<
string,
Pick<RoomMember, "userId" | "getMxcAvatarUrl" | "rawDisplayName">
@@ -67,6 +65,7 @@ export const memberDisplaynames$ = (
memberships$: Behavior<Pick<CallMembership, "userId">[]>,
roomMembers$: Behavior<RoomMemberMap>,
): Behavior<Map<string, string>> => {
const logger = rootLogger.getChild("[MatrixMemberMetadata]");
// This map tracks userIds that at some point needed disambiguation.
// This is a memory leak bound to the number of participants.
// A call application will always increase the memory if there have been more members in a call.
@@ -115,8 +114,14 @@ export const createMatrixMemberMetadata$ = (
memberships$: Behavior<Pick<CallMembership, "userId">[]>,
roomMembers$: Behavior<RoomMemberMap>,
): {
createDisplayNameBehavior$: (userId: string) => Behavior<string | undefined>;
createAvatarUrlBehavior$: (userId: string) => Behavior<string | undefined>;
createDisplayNameBehavior$: (
scope: ObservableScope,
userId: string,
) => Behavior<string | undefined>;
createAvatarUrlBehavior$: (
scope: ObservableScope,
userId: string,
) => Behavior<string | undefined>;
displaynameMap$: Behavior<Map<string, string>>;
avatarMap$: Behavior<Map<string, string | undefined>>;
} => {
@@ -136,13 +141,13 @@ export const createMatrixMemberMetadata$ = (
),
);
return {
createDisplayNameBehavior$: (userId: string) =>
createDisplayNameBehavior$: (scope: ObservableScope, userId: string) =>
scope.behavior(
displaynameMap$.pipe(
map((displaynameMap) => displaynameMap.get(userId)),
),
),
createAvatarUrlBehavior$: (userId: string) =>
createAvatarUrlBehavior$: (scope: ObservableScope, userId: string) =>
scope.behavior(
roomMembers$.pipe(
map((roomMembers) => roomMembers.get(userId)?.getMxcAvatarUrl()),
@@ -29,13 +29,13 @@ import {
import { type ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
import {
areLivekitTransportsEqual,
createMatrixLivekitMembers$,
createRemoteMatrixLivekitMembers$,
type RemoteMatrixLivekitMember,
} from "./MatrixLivekitMembers.ts";
import { createConnectionManager$ } from "./ConnectionManager.ts";
import { membershipsAndTransports$ } from "../../SessionBehaviors.ts";
import { constant } from "../../Behavior.ts";
import { testJWTToken } from "../../../utils/test-fixtures.ts";
import { localRtcMember, testJWTToken } from "../../../utils/test-fixtures.ts";
// Test the integration of ConnectionManager and MatrixLivekitMerger
@@ -130,14 +130,15 @@ test("bob, carl, then bob joining no tracks yet", () => {
ownMembershipIdentity: ownMemberMock,
});
const matrixLivekitMembers$ = createMatrixLivekitMembers$({
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
scope: testScope,
membershipsWithTransport$:
membershipsAndTransports.membershipsWithTransport$,
connectionManager,
localUser: localRtcMember,
});
expectObservable(matrixLivekitMembers$).toBe(vMarble, {
expectObservable(remoteMatrixLivekitMembers$).toBe(vMarble, {
a: expect.toSatisfy((e: Epoch<RemoteMatrixLivekitMember[]>) => {
const items = e.value;
expect(items.length).toBe(1);