merge: upstream v0.25.0 into lotus
Merge upstream element-hq/element-call tag v0.25.0 into the Lotus fork (previous base: v0.20.1; actual merge-base v0.20.1-rc.1). Every Lotus feature and all six io.lotus.* widget actions are preserved. Version bumped to 0.25.0-lotus.1. Conflict files and how each Lotus hunk was re-expressed: * src/state/CallViewModel/remoteMembers/ConnectionFactory.ts Upstream moved echoCancellation/noiseSuppression/autoGainControl from constructor params (fed by URL params) to persisted Settings (settings.ts) with a developer-settings UI. The cinny host still drives these per call via URL params (noiseSuppression=false / autoGainControl=false when the in-source ML denoiser is active, so the model gets a raw mic) - taking upstream verbatim would silently break the ML denoise tier. Re-wired as AND semantics in generateRoomOption(): a constraint is enabled only if BOTH the Setting and the URL param allow it. Params default to true, so with no params this is byte-for-byte upstream behaviour. Upstream's own echoCancellation / noiseSuppression URL params (still parsed but dead in v0.25.0) work again as a side effect. Lotus autoGainControl URL param kept in UrlParams.ts (auto-merged, unchanged). * src/state/CallViewModel/remoteMembers/ECConnectionFactory.test.ts Took upstream (tests now drive via Settings). The lost Lotus coverage is restored in a NEW colocated file src/lotus/lotusAudioConstraints.test.ts (3 tests) so the upstream test file stays pristine. Verified the new test fails against pure-upstream ConnectionFactory and passes with the re-wiring. * src/state/CallViewModel/CallViewModel.ts Three small hunks: kept both the Lotus `userMedia$` interface member and upstream's new `keyRotationSuppressed$`; dropped the three Lotus audio constructor args (mechanism removed upstream, see above); kept both in the returned object. The [lotus #4] overrideSpotlight$ routing, manualSpotlightUserId$ and setManualSpotlight auto-merged; verified against upstream's changed ringingMedia$ (now single-or-null instead of array) - the merge correctly took upstream's outer branch and the inner screenShares$/spotlightSpeaker$ logic that lotusSpotlight.ts mirrors is unchanged upstream. * src/index.css Kept both: Lotus lotus-transparent / lotus-theme blocks and upstream's new body[data-background="gradient"]::before full-viewport gradient. The naive merge swallowed the closing brace of body.lotus-theme - restored. Added a rule hiding the new gradient pseudo-element under body.lotus-transparent, since it would otherwise paint over the transparent body and hide the host wallpaper. * src/components/CallFooterViewModel.tsx, src/components/CallFooter.stories.tsx No Lotus content - pure upstream-vs-upstream conflicts caused by the merge base being v0.20.1-rc.1. Took upstream (layoutMode -> layoutSwitchVm; setLayoutMode removed). No Lotus code uses setGridMode/layoutMode. Non-conflicting but reviewed: * src/widget.ts auto-merged cleanly. Upstream's removal of .well-known transport advertisement and the new RTC-transport capability request did not touch the action registration loop the LOTUS_TO_WIDGET_ACTIONS spread and widget.lazyActions ride on - nothing to re-wire. * src/room/InCallView.tsx, src/useAudioContext.tsx, src/useTheme.ts, src/tile/MediaView.tsx(+.module.css), src/UrlParams.ts(+test), all *.module.css and .gitea/workflows/ci.yml auto-merged; each diff against v0.25.0 was checked to equal the original Lotus hunk. * src/button/Button.module.css: the merge appended an exact duplicate of upstream's `.rotate`/`@keyframes spin` block (rc.1 merge-base artefact) - reset to upstream verbatim. * src/grid/OneOnOnePortraitLayout.module.css was renamed upstream to OneOnOneMobileLayout.module.css; git followed the rename and the Lotus safe-area PiP inset fix applies there (the --content-inset-* vars it uses still exist upstream). Tooling changes inherited from upstream that affect the fork: * eslint + prettier were replaced by oxlint + oxfmt (`pnpm lint:oxlint`, `pnpm format:check`). oxlint flagged 10 issues, all in src/lotus/*: 8x no-meaningless-void-operator (dropped the `void` before void-typed widget transport.reply / callbacks - no behaviour change), 1x consistent-type-imports (lotusWidget.ts: `import type`), and 2x unicorn/no-useless-spread in lotusAudioInject.ts which are FALSE POSITIVES - `[...activeClips]` is a required defensive copy because abort() deletes from the Set during iteration; suppressed with an explanatory eslint-disable-next-line. oxfmt reformatted 7 Lotus touched files (whitespace only). * packageManager bumped by upstream to pnpm@11.21.0, which requires Node >= 22.13 (uses node:sqlite). Node 20 cannot run it; pnpm 10.33 cannot read the new lockfile either (matrix-js-sdk is now a git dependency on develop, using a version-union pnpm 10 rejects). Fork CI already uses Node 24 (.node-version), so CI is unaffected. * matrix-js-sdk is now github:matrix-org/matrix-js-sdk#develop (pinned by commit in pnpm-lock.yaml). Lotus behaviour NOT preserved: none found. Verification (Node 24.11.1, pnpm 11.21.0): pnpm install --frozen-lockfile OK (lockfile taken from upstream unchanged, no regeneration needed); tsc clean; oxlint clean; oxfmt --check clean; knip exit 0 (2 config hints in upstream knip.ts only); vitest unit 84 files / 627 passed / 9 skipped; build:embedded OK, staged to embedded/web/dist (44M), all six io.lotus.* action strings present in the bundle. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
This commit is contained in:
@@ -12,8 +12,9 @@ import {
|
||||
} from "@livekit/components-core";
|
||||
import {
|
||||
ConnectionError,
|
||||
type Room as LivekitRoom,
|
||||
ConnectionErrorReason,
|
||||
type RemoteParticipant,
|
||||
type Room as LivekitRoom,
|
||||
} from "livekit-client";
|
||||
import { type LivekitTransportConfig } from "matrix-js-sdk/lib/matrixrtc";
|
||||
import { BehaviorSubject, map } from "rxjs";
|
||||
@@ -30,10 +31,11 @@ import { type ObservableScope } from "../../ObservableScope.ts";
|
||||
import {
|
||||
ElementCallError,
|
||||
InsufficientCapacityError,
|
||||
LivekitConnectionError,
|
||||
PeerConnectionTimeoutError,
|
||||
SFURoomCreationRestrictedError,
|
||||
UnknownCallError,
|
||||
} from "../../../utils/errors.ts";
|
||||
import { type JwtEndpointVersion } from "../localMember/LocalTransport.ts";
|
||||
|
||||
export interface ConnectionOpts {
|
||||
/**
|
||||
@@ -41,11 +43,6 @@ export interface ConnectionOpts {
|
||||
* On top the local transport will send additional data to the jwt server to use delayed event delegation.
|
||||
*/
|
||||
existingSFUConfig?: SFUConfig;
|
||||
/**
|
||||
* For local connections that use the oldest member pattern. here we have not prefetched the sfuConfig
|
||||
* and hence we need to let the connection do the jwt token fetching.
|
||||
*/
|
||||
forceJwtEndpoint?: JwtEndpointVersion;
|
||||
/** The identity parts to use on this connection */
|
||||
ownMembershipIdentity: CallMembershipIdentityParts;
|
||||
/** The media transport to connect to. */
|
||||
@@ -248,6 +245,13 @@ export class Connection {
|
||||
// In the first case there will not be a 404, so we are in the second case.
|
||||
throw new SFURoomCreationRestrictedError();
|
||||
}
|
||||
|
||||
if (e.reason === ConnectionErrorReason.Timeout) {
|
||||
// Unabled to establish peer connection within the timeout
|
||||
throw new PeerConnectionTimeoutError();
|
||||
}
|
||||
|
||||
throw new LivekitConnectionError(e);
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
|
||||
@@ -12,7 +12,7 @@ import {
|
||||
type E2EEManagerOptions,
|
||||
type BaseE2EEManager,
|
||||
} from "livekit-client";
|
||||
import { type Logger } from "matrix-js-sdk/lib/logger";
|
||||
import { logger, type Logger } from "matrix-js-sdk/lib/logger";
|
||||
// imported as inline to support worker when loaded from a cdn (cross domain)
|
||||
import E2EEWorker from "livekit-client/e2ee-worker?worker&inline";
|
||||
import { type CallMembershipIdentityParts } from "matrix-js-sdk/lib/matrixrtc/EncryptionManager";
|
||||
@@ -27,7 +27,26 @@ import type {
|
||||
import type { MediaDevices } from "../../MediaDevices.ts";
|
||||
import type { Behavior } from "../../Behavior.ts";
|
||||
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
|
||||
import { defaultLiveKitOptions } from "../../../livekit/options.ts";
|
||||
import { getLiveKitOptions } from "../../../livekit/options.ts";
|
||||
import {
|
||||
advancedCamera,
|
||||
cameraResolution,
|
||||
cameraFramerate,
|
||||
cameraBitrate,
|
||||
cameraCodec,
|
||||
parseResolution,
|
||||
echoCancellationSetting,
|
||||
noiseSuppressionSetting,
|
||||
autoGainControlSetting,
|
||||
} from "../../../settings/settings.ts";
|
||||
// [lotus] Upstream v0.25.0 moved the audio-capture constraints from URL params
|
||||
// to persisted Settings. The Lotus host (cinny) still drives them per-call via
|
||||
// URL params (it turns browser noiseSuppression/AGC OFF when the in-source ML
|
||||
// denoiser is active, so the model gets a raw mic). Re-wire: a constraint is
|
||||
// enabled only if BOTH the Setting and the URL param allow it. The params
|
||||
// default to true, so with no params this is byte-for-byte upstream behaviour;
|
||||
// `?autoGainControl=false` forces AGC off regardless of the stored Setting.
|
||||
import { getUrlParams } from "../../../UrlParams.ts";
|
||||
|
||||
// TODO evaluate if this should be done like the Publisher Factory
|
||||
export interface ConnectionFactory {
|
||||
@@ -53,9 +72,6 @@ export class ECConnectionFactory implements ConnectionFactory {
|
||||
* @param livekitKeyProvider - Optional key provider for end-to-end encryption.
|
||||
* @param controlledAudioDevices - Option to indicate whether audio output device is controlled externally (native mobile app).
|
||||
* @param livekitRoomFactory - Optional factory function (for testing) to create LivekitRoom instances. If not provided, a default factory is used.
|
||||
* @param echoCancellation - Whether to enable echo cancellation for audio capture.
|
||||
* @param noiseSuppression - Whether to enable noise suppression for audio capture.
|
||||
* @param autoGainControl - Whether to enable auto gain control for audio capture.
|
||||
*/
|
||||
public constructor(
|
||||
private client: OpenIDClientParts,
|
||||
@@ -65,27 +81,22 @@ export class ECConnectionFactory implements ConnectionFactory {
|
||||
livekitKeyProvider: BaseKeyProvider | undefined,
|
||||
private controlledAudioDevices: boolean,
|
||||
livekitRoomFactory?: () => LivekitRoom,
|
||||
echoCancellation: boolean = true,
|
||||
noiseSuppression: boolean = true,
|
||||
autoGainControl: boolean = true,
|
||||
) {
|
||||
const defaultFactory = (): LivekitRoom =>
|
||||
new LivekitRoom(
|
||||
generateRoomOption({
|
||||
devices: this.devices,
|
||||
processorState: this.processorState$.value,
|
||||
e2eeLivekitOptions: livekitKeyProvider && {
|
||||
keyProvider: livekitKeyProvider,
|
||||
// It's important that every room use a separate E2EE worker.
|
||||
// They get confused if given streams from multiple rooms.
|
||||
worker: new E2EEWorker(),
|
||||
},
|
||||
controlledAudioDevices: this.controlledAudioDevices,
|
||||
echoCancellation,
|
||||
noiseSuppression,
|
||||
autoGainControl,
|
||||
}),
|
||||
);
|
||||
const defaultFactory = (): LivekitRoom => {
|
||||
const roomOptions = generateRoomOption({
|
||||
devices: this.devices,
|
||||
processorState: this.processorState$.value,
|
||||
e2eeLivekitOptions: livekitKeyProvider && {
|
||||
keyProvider: livekitKeyProvider,
|
||||
// It's important that every room use a separate E2EE worker.
|
||||
// They get confused if given streams from multiple rooms.
|
||||
worker: new E2EEWorker(),
|
||||
},
|
||||
controlledAudioDevices: this.controlledAudioDevices,
|
||||
});
|
||||
logger.info("[ECConnectionFactory] livekit room options: ", roomOptions);
|
||||
return new LivekitRoom(roomOptions);
|
||||
};
|
||||
this.livekitRoomFactory = livekitRoomFactory ?? defaultFactory;
|
||||
}
|
||||
|
||||
@@ -122,15 +133,13 @@ export class ECConnectionFactory implements ConnectionFactory {
|
||||
|
||||
/**
|
||||
* Generate the initial LiveKit RoomOptions based on the current media devices and processor state.
|
||||
* Reads audio processing and camera quality settings directly from Settings.
|
||||
*/
|
||||
function generateRoomOption({
|
||||
devices,
|
||||
processorState,
|
||||
e2eeLivekitOptions,
|
||||
controlledAudioDevices,
|
||||
echoCancellation,
|
||||
noiseSuppression,
|
||||
autoGainControl,
|
||||
}: {
|
||||
devices: MediaDevices;
|
||||
processorState: ProcessorState;
|
||||
@@ -139,23 +148,50 @@ function generateRoomOption({
|
||||
| { e2eeManager: BaseE2EEManager }
|
||||
| undefined;
|
||||
controlledAudioDevices: boolean;
|
||||
echoCancellation: boolean;
|
||||
noiseSuppression: boolean;
|
||||
autoGainControl: boolean;
|
||||
}): RoomOptions {
|
||||
const liveKitOptions = getLiveKitOptions();
|
||||
|
||||
// Apply advanced camera settings if enabled
|
||||
let videoCaptureDefaults = {
|
||||
...liveKitOptions.videoCaptureDefaults,
|
||||
deviceId: devices.videoInput.selected$.value?.id,
|
||||
processor: processorState.processor,
|
||||
};
|
||||
let publishDefaults = liveKitOptions.publishDefaults;
|
||||
|
||||
if (advancedCamera.getValue()) {
|
||||
const { width, height } = parseResolution(cameraResolution.getValue());
|
||||
const fps = cameraFramerate.getValue();
|
||||
const bps = cameraBitrate.getValue();
|
||||
const codec = cameraCodec.getValue();
|
||||
|
||||
videoCaptureDefaults = {
|
||||
...videoCaptureDefaults,
|
||||
resolution: { width, height, frameRate: fps },
|
||||
};
|
||||
publishDefaults = {
|
||||
...publishDefaults,
|
||||
videoEncoding: { maxBitrate: bps, maxFramerate: fps },
|
||||
videoCodec: codec,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
...defaultLiveKitOptions,
|
||||
videoCaptureDefaults: {
|
||||
...defaultLiveKitOptions.videoCaptureDefaults,
|
||||
deviceId: devices.videoInput.selected$.value?.id,
|
||||
processor: processorState.processor,
|
||||
},
|
||||
...liveKitOptions,
|
||||
videoCaptureDefaults,
|
||||
publishDefaults,
|
||||
audioCaptureDefaults: {
|
||||
...defaultLiveKitOptions.audioCaptureDefaults,
|
||||
...liveKitOptions.audioCaptureDefaults,
|
||||
deviceId: devices.audioInput.selected$.value?.id,
|
||||
echoCancellation,
|
||||
noiseSuppression,
|
||||
autoGainControl,
|
||||
echoCancellation:
|
||||
echoCancellationSetting.getValue() &&
|
||||
getUrlParams().echoCancellation !== false,
|
||||
noiseSuppression:
|
||||
noiseSuppressionSetting.getValue() &&
|
||||
getUrlParams().noiseSuppression !== false,
|
||||
autoGainControl:
|
||||
autoGainControlSetting.getValue() &&
|
||||
getUrlParams().autoGainControl !== false,
|
||||
},
|
||||
audioOutput: {
|
||||
// When using controlled audio devices, we don't want to set the
|
||||
|
||||
@@ -22,6 +22,16 @@ import {
|
||||
} from "../../../utils/test.ts";
|
||||
import type { ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
|
||||
import { constant } from "../../Behavior";
|
||||
import {
|
||||
echoCancellationSetting,
|
||||
noiseSuppressionSetting,
|
||||
autoGainControlSetting,
|
||||
advancedCamera,
|
||||
cameraResolution,
|
||||
cameraFramerate,
|
||||
cameraBitrate,
|
||||
cameraCodec,
|
||||
} from "../../../settings/settings.ts";
|
||||
|
||||
// At the top of your test file, after imports
|
||||
vi.mock("livekit-client", async (importOriginal) => {
|
||||
@@ -53,15 +63,19 @@ beforeEach(() => {
|
||||
|
||||
describe("ECConnectionFactory - Audio inputs options", () => {
|
||||
test.each([
|
||||
{ echo: true, noise: true, agc: true },
|
||||
{ echo: true, noise: false, agc: false },
|
||||
{ echo: false, noise: true, agc: false },
|
||||
{ echo: false, noise: false, agc: true },
|
||||
{ echo: true, noise: true },
|
||||
{ echo: true, noise: false },
|
||||
{ echo: false, noise: true },
|
||||
{ echo: false, noise: false },
|
||||
])(
|
||||
"it sets echoCancellation=$echo, noiseSuppression=$noise, autoGainControl=$agc based on constructor parameters",
|
||||
({ echo, noise, agc }) => {
|
||||
"it sets echoCancellation=$echo and noiseSuppression=$noise based on settings",
|
||||
({ echo, noise }) => {
|
||||
const RoomConstructor = vi.mocked(LivekitRoom);
|
||||
|
||||
// Set audio processing settings
|
||||
echoCancellationSetting.setValue(echo);
|
||||
noiseSuppressionSetting.setValue(noise);
|
||||
|
||||
const ecConnectionFactory = new ECConnectionFactory(
|
||||
mockClient,
|
||||
"!roomid:example.org",
|
||||
@@ -72,10 +86,6 @@ describe("ECConnectionFactory - Audio inputs options", () => {
|
||||
}),
|
||||
undefined,
|
||||
false,
|
||||
undefined,
|
||||
echo,
|
||||
noise,
|
||||
agc,
|
||||
);
|
||||
ecConnectionFactory.createConnection(
|
||||
testScope,
|
||||
@@ -90,7 +100,6 @@ describe("ECConnectionFactory - Audio inputs options", () => {
|
||||
audioCaptureDefaults: expect.objectContaining({
|
||||
echoCancellation: echo,
|
||||
noiseSuppression: noise,
|
||||
autoGainControl: agc,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
@@ -102,9 +111,13 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
|
||||
test.each([{ controlled: true }, { controlled: false }])(
|
||||
"it sets controlledAudioDevice=$controlled then uses deviceId accordingly",
|
||||
({ controlled }) => {
|
||||
// test("it sets echoCancellation and noiseSuppression based on constructor parameters", () => {
|
||||
const RoomConstructor = vi.mocked(LivekitRoom);
|
||||
|
||||
// Explicitly set audio settings so the test doesn't depend on defaults
|
||||
echoCancellationSetting.setValue(true);
|
||||
noiseSuppressionSetting.setValue(true);
|
||||
autoGainControlSetting.setValue(true);
|
||||
|
||||
const ecConnectionFactory = new ECConnectionFactory(
|
||||
mockClient,
|
||||
"!roomid:example.org",
|
||||
@@ -121,9 +134,6 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
|
||||
}),
|
||||
undefined,
|
||||
controlled,
|
||||
undefined,
|
||||
false,
|
||||
false,
|
||||
);
|
||||
ecConnectionFactory.createConnection(
|
||||
testScope,
|
||||
@@ -144,6 +154,114 @@ describe("ECConnectionFactory - ControlledAudioDevice", () => {
|
||||
);
|
||||
});
|
||||
|
||||
describe("ECConnectionFactory - Camera quality settings", () => {
|
||||
test("it uses default video options when advancedCamera is disabled", () => {
|
||||
const RoomConstructor = vi.mocked(LivekitRoom);
|
||||
advancedCamera.setValue(false);
|
||||
|
||||
const ecConnectionFactory = new ECConnectionFactory(
|
||||
mockClient,
|
||||
"!roomid:example.org",
|
||||
mockMediaDevices({}),
|
||||
new BehaviorSubject<ProcessorState>({
|
||||
supported: true,
|
||||
processor: undefined,
|
||||
}),
|
||||
undefined,
|
||||
false,
|
||||
);
|
||||
ecConnectionFactory.createConnection(
|
||||
testScope,
|
||||
exampleTransport,
|
||||
ownMemberMock,
|
||||
logger,
|
||||
);
|
||||
|
||||
// publishDefaults should use config defaults (vp8), not custom settings
|
||||
expect(RoomConstructor).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
publishDefaults: expect.objectContaining({
|
||||
videoCodec: "vp8",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
test("it applies custom camera resolution, encoding, and codec when advancedCamera is enabled", () => {
|
||||
const RoomConstructor = vi.mocked(LivekitRoom);
|
||||
|
||||
advancedCamera.setValue(true);
|
||||
cameraResolution.setValue("1920x1080");
|
||||
cameraFramerate.setValue(60);
|
||||
cameraBitrate.setValue(4_000_000);
|
||||
cameraCodec.setValue("vp9");
|
||||
|
||||
const ecConnectionFactory = new ECConnectionFactory(
|
||||
mockClient,
|
||||
"!roomid:example.org",
|
||||
mockMediaDevices({}),
|
||||
new BehaviorSubject<ProcessorState>({
|
||||
supported: true,
|
||||
processor: undefined,
|
||||
}),
|
||||
undefined,
|
||||
false,
|
||||
);
|
||||
ecConnectionFactory.createConnection(
|
||||
testScope,
|
||||
exampleTransport,
|
||||
ownMemberMock,
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(RoomConstructor).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
videoCaptureDefaults: expect.objectContaining({
|
||||
resolution: { width: 1920, height: 1080, frameRate: 60 },
|
||||
}),
|
||||
publishDefaults: expect.objectContaining({
|
||||
videoEncoding: { maxBitrate: 4_000_000, maxFramerate: 60 },
|
||||
videoCodec: "vp9",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
test("it applies autoGainControl from settings", () => {
|
||||
const RoomConstructor = vi.mocked(LivekitRoom);
|
||||
|
||||
autoGainControlSetting.setValue(false);
|
||||
echoCancellationSetting.setValue(true);
|
||||
noiseSuppressionSetting.setValue(true);
|
||||
|
||||
const ecConnectionFactory = new ECConnectionFactory(
|
||||
mockClient,
|
||||
"!roomid:example.org",
|
||||
mockMediaDevices({}),
|
||||
new BehaviorSubject<ProcessorState>({
|
||||
supported: true,
|
||||
processor: undefined,
|
||||
}),
|
||||
undefined,
|
||||
false,
|
||||
);
|
||||
ecConnectionFactory.createConnection(
|
||||
testScope,
|
||||
exampleTransport,
|
||||
ownMemberMock,
|
||||
logger,
|
||||
);
|
||||
|
||||
expect(RoomConstructor).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
audioCaptureDefaults: expect.objectContaining({
|
||||
autoGainControl: false,
|
||||
}),
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
testScope.end();
|
||||
fetchMock.reset();
|
||||
|
||||
@@ -15,7 +15,7 @@ import { BehaviorSubject, combineLatest, map, type Observable } from "rxjs";
|
||||
import { type IConnectionManager } from "./ConnectionManager.ts";
|
||||
import {
|
||||
type RemoteMatrixLivekitMember,
|
||||
createMatrixLivekitMembers$,
|
||||
createRemoteMatrixLivekitMembers$,
|
||||
} from "./MatrixLivekitMembers.ts";
|
||||
import {
|
||||
Epoch,
|
||||
@@ -31,6 +31,7 @@ import {
|
||||
} from "../../../utils/test.ts";
|
||||
import { type Connection } from "./Connection.ts";
|
||||
import { constant } from "../../Behavior.ts";
|
||||
import { localRtcMember } from "../../../utils/test-fixtures.ts";
|
||||
|
||||
let testScope: ObservableScope;
|
||||
|
||||
@@ -88,16 +89,17 @@ test("should signal participant not yet connected to livekit", async () => {
|
||||
mockConnectionManagerData$,
|
||||
);
|
||||
|
||||
const matrixLivekitMember$ = createMatrixLivekitMembers$({
|
||||
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
|
||||
scope: testScope,
|
||||
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
|
||||
connectionManager: {
|
||||
connectionManagerData$: connectionManagerData$,
|
||||
} as unknown as IConnectionManager,
|
||||
localUser: localRtcMember,
|
||||
});
|
||||
|
||||
await flushPromises();
|
||||
expect(matrixLivekitMember$.value.value).toSatisfy(
|
||||
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
|
||||
(data: RemoteMatrixLivekitMember[]) => {
|
||||
expect(data.length).toEqual(1);
|
||||
expect(data[0].membership$.value).toBe(bobMembership);
|
||||
@@ -157,16 +159,17 @@ test("should signal participant on a connection that is publishing", async () =>
|
||||
constant(dataWithPublisher),
|
||||
);
|
||||
|
||||
const matrixLivekitMember$ = createMatrixLivekitMembers$({
|
||||
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
|
||||
scope: testScope,
|
||||
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
|
||||
connectionManager: {
|
||||
connectionManagerData$: connectionManagerData$,
|
||||
} as unknown as IConnectionManager,
|
||||
localUser: localRtcMember,
|
||||
});
|
||||
|
||||
await flushPromises();
|
||||
expect(matrixLivekitMember$.value.value).toSatisfy(
|
||||
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
|
||||
(data: RemoteMatrixLivekitMember[]) => {
|
||||
expect(data.length).toEqual(1);
|
||||
expect(data[0].membership$.value).toBe(bobMembership);
|
||||
@@ -197,15 +200,16 @@ test("should signal participant on a connection that is not publishing", async (
|
||||
constant(dataWithPublisher),
|
||||
);
|
||||
|
||||
const matrixLivekitMember$ = createMatrixLivekitMembers$({
|
||||
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
|
||||
scope: testScope,
|
||||
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
|
||||
connectionManager: {
|
||||
connectionManagerData$: connectionManagerData$,
|
||||
} as unknown as IConnectionManager,
|
||||
localUser: localRtcMember,
|
||||
});
|
||||
await flushPromises();
|
||||
expect(matrixLivekitMember$.value.value).toSatisfy(
|
||||
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
|
||||
(data: RemoteMatrixLivekitMember[]) => {
|
||||
expect(data.length).toEqual(1);
|
||||
expect(data[0].membership$.value).toBe(bobMembership);
|
||||
@@ -245,15 +249,16 @@ describe("Publication edge case", () => {
|
||||
constant(connectionWithPublisher),
|
||||
);
|
||||
|
||||
const matrixLivekitMembers$ = createMatrixLivekitMembers$({
|
||||
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
|
||||
scope: testScope,
|
||||
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
|
||||
connectionManager: {
|
||||
connectionManagerData$: connectionManagerData$,
|
||||
} as unknown as IConnectionManager,
|
||||
localUser: localRtcMember,
|
||||
});
|
||||
await flushPromises();
|
||||
expect(matrixLivekitMembers$.value.value).toSatisfy(
|
||||
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
|
||||
(data: RemoteMatrixLivekitMember[]) => {
|
||||
expect(data.length).toEqual(2);
|
||||
expect(data[0].membership$.value).toBe(bobMembership);
|
||||
@@ -303,16 +308,17 @@ test("bob is publishing in the wrong connection", async () => {
|
||||
connectionsWithPublisher$,
|
||||
);
|
||||
|
||||
const matrixLivekitMember$ = createMatrixLivekitMembers$({
|
||||
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
|
||||
scope: testScope,
|
||||
membershipsWithTransport$: testScope.behavior(membershipsWithTransport$),
|
||||
connectionManager: {
|
||||
connectionManagerData$: connectionManagerData$,
|
||||
} as unknown as IConnectionManager,
|
||||
localUser: localRtcMember,
|
||||
});
|
||||
|
||||
await flushPromises();
|
||||
expect(matrixLivekitMember$.value.value).toSatisfy(
|
||||
expect(remoteMatrixLivekitMembers$.value.value).toSatisfy(
|
||||
(data: RemoteMatrixLivekitMember[]) => {
|
||||
expect(data.length).toEqual(2);
|
||||
expect(data[0].membership$.value).toBe(bobMembership);
|
||||
|
||||
@@ -62,7 +62,9 @@ interface Props {
|
||||
Epoch<{ membership: CallMembership; transport?: LivekitTransportConfig }[]>
|
||||
>;
|
||||
connectionManager: IConnectionManager;
|
||||
localUser: { deviceId: string; userId: string };
|
||||
}
|
||||
|
||||
/**
|
||||
* Combines MatrixRTC and Livekit worlds.
|
||||
*
|
||||
@@ -73,13 +75,14 @@ interface Props {
|
||||
* - out (via public Observable):
|
||||
* - `remoteMatrixLivekitMember` an observable of MatrixLivekitMember[] to track the remote members and associated livekit data.
|
||||
*/
|
||||
export function createMatrixLivekitMembers$({
|
||||
export function createRemoteMatrixLivekitMembers$({
|
||||
scope,
|
||||
membershipsWithTransport$,
|
||||
connectionManager,
|
||||
localUser,
|
||||
}: Props): Behavior<Epoch<RemoteMatrixLivekitMember[]>> {
|
||||
/**
|
||||
* Stream of all the call members and their associated livekit data (if available).
|
||||
* Behavior of all the remote call members and their associated livekit data (if available).
|
||||
*/
|
||||
return scope.behavior(
|
||||
combineLatest([
|
||||
@@ -91,12 +94,19 @@ export function createMatrixLivekitMembers$({
|
||||
),
|
||||
map(([ms, data]) => new Epoch([ms.value, data.value] as const, ms.epoch)),
|
||||
generateItemsWithEpoch(
|
||||
"MatrixLivekitMembers",
|
||||
"RemoteMatrixLivekitMembers",
|
||||
// Generator function.
|
||||
// creates an array of `{key, data}[]`
|
||||
// Each change in the keys (new key) will result in a call to the factory function.
|
||||
function* ([membershipsWithTransport, managerData]) {
|
||||
for (const { membership, transport } of membershipsWithTransport) {
|
||||
// Exclude the local membership
|
||||
if (
|
||||
membership.userId === localUser.userId &&
|
||||
membership.deviceId === localUser.deviceId
|
||||
)
|
||||
continue;
|
||||
|
||||
const participants = transport
|
||||
? managerData.getParticipantsForTransport(transport)
|
||||
: [];
|
||||
|
||||
@@ -105,7 +105,7 @@ describe("MatrixMemberMetadata", () => {
|
||||
}
|
||||
|
||||
it("should show our own user if present in rtc session and room", () => {
|
||||
withTestScheduler(({ behavior, expectObservable }) => {
|
||||
withTestScheduler(({ scope, behavior, expectObservable }) => {
|
||||
fakeMemberWith({
|
||||
userId: "@local:example.com",
|
||||
rawDisplayName: "it's a me",
|
||||
@@ -118,8 +118,10 @@ describe("MatrixMemberMetadata", () => {
|
||||
memberships$,
|
||||
createRoomMembers$(testScope, mockMatrixRoom),
|
||||
);
|
||||
const dn$ =
|
||||
metadataStore.createDisplayNameBehavior$("@local:example.com");
|
||||
const dn$ = metadataStore.createDisplayNameBehavior$(
|
||||
scope,
|
||||
"@local:example.com",
|
||||
);
|
||||
|
||||
expectObservable(dn$).toBe("a", {
|
||||
a: "it's a me",
|
||||
@@ -146,7 +148,7 @@ describe("MatrixMemberMetadata", () => {
|
||||
it("should get displayName for users", () => {
|
||||
setUpBasicRoom();
|
||||
|
||||
withTestScheduler(({ behavior, expectObservable }) => {
|
||||
withTestScheduler(({ scope, behavior, expectObservable }) => {
|
||||
const memberships$ = behavior("a", {
|
||||
a: [
|
||||
mockRtcMembership("@alice:example.com", "DEVICE1"),
|
||||
@@ -158,8 +160,10 @@ describe("MatrixMemberMetadata", () => {
|
||||
memberships$,
|
||||
createRoomMembers$(testScope, mockMatrixRoom),
|
||||
);
|
||||
const aliceDispName$ =
|
||||
metadataStore.createDisplayNameBehavior$("@alice:example.com");
|
||||
const aliceDispName$ = metadataStore.createDisplayNameBehavior$(
|
||||
scope,
|
||||
"@alice:example.com",
|
||||
);
|
||||
|
||||
expectObservable(aliceDispName$).toBe("a", {
|
||||
a: "Alice",
|
||||
@@ -322,7 +326,7 @@ describe("MatrixMemberMetadata", () => {
|
||||
});
|
||||
|
||||
it("should track individual member id with createDisplayNameBehavior", () => {
|
||||
withTestScheduler(({ behavior, schedule, expectObservable }) => {
|
||||
withTestScheduler(({ scope, behavior, schedule, expectObservable }) => {
|
||||
setUpBasicRoom();
|
||||
const BOB = "@bob:example.com";
|
||||
const CARL = "@carl:example.com";
|
||||
@@ -356,8 +360,8 @@ describe("MatrixMemberMetadata", () => {
|
||||
createRoomMembers$(testScope, mockMatrixRoom),
|
||||
);
|
||||
|
||||
const bob$ = metadataStore.createDisplayNameBehavior$(BOB);
|
||||
const carl$ = metadataStore.createDisplayNameBehavior$(CARL);
|
||||
const bob$ = metadataStore.createDisplayNameBehavior$(scope, BOB);
|
||||
const carl$ = metadataStore.createDisplayNameBehavior$(scope, CARL);
|
||||
|
||||
expectObservable(bob$).toBe("abc-", {
|
||||
a: undefined,
|
||||
@@ -378,7 +382,7 @@ describe("MatrixMemberMetadata", () => {
|
||||
});
|
||||
|
||||
it("should disambiguate users with invisible characters", () => {
|
||||
withTestScheduler(({ behavior, expectObservable }) => {
|
||||
withTestScheduler(({ scope, behavior, expectObservable }) => {
|
||||
const bobRtcMember = mockRtcMembership("@bob:example.org", "BBBB");
|
||||
const bobZeroWidthSpaceRtcMember = mockRtcMembership(
|
||||
"@bob2:example.org",
|
||||
@@ -411,12 +415,18 @@ describe("MatrixMemberMetadata", () => {
|
||||
createRoomMembers$(testScope, mockMatrixRoom),
|
||||
);
|
||||
|
||||
const bob$ =
|
||||
metadataStore.createDisplayNameBehavior$("@bob:example.org");
|
||||
const bob2$ =
|
||||
metadataStore.createDisplayNameBehavior$("@bob2:example.org");
|
||||
const carol$ =
|
||||
metadataStore.createDisplayNameBehavior$("@carol:example.org");
|
||||
const bob$ = metadataStore.createDisplayNameBehavior$(
|
||||
scope,
|
||||
"@bob:example.org",
|
||||
);
|
||||
const bob2$ = metadataStore.createDisplayNameBehavior$(
|
||||
scope,
|
||||
"@bob2:example.org",
|
||||
);
|
||||
const carol$ = metadataStore.createDisplayNameBehavior$(
|
||||
scope,
|
||||
"@carol:example.org",
|
||||
);
|
||||
expectObservable(bob$).toBe("ab", {
|
||||
a: "Bob",
|
||||
b: "Bob (@bob:example.org)",
|
||||
@@ -517,7 +527,7 @@ describe("MatrixMemberMetadata", () => {
|
||||
}
|
||||
|
||||
it("should use avatar url from room members", () => {
|
||||
withTestScheduler(({ behavior, expectObservable }) => {
|
||||
withTestScheduler(({ scope, behavior, expectObservable }) => {
|
||||
fakeMemberWith({
|
||||
userId: "@local:example.com",
|
||||
});
|
||||
@@ -536,11 +546,15 @@ describe("MatrixMemberMetadata", () => {
|
||||
memberships$,
|
||||
createRoomMembers$(testScope, mockMatrixRoom),
|
||||
);
|
||||
const local$ =
|
||||
metadataStore.createAvatarUrlBehavior$("@local:example.com");
|
||||
const local$ = metadataStore.createAvatarUrlBehavior$(
|
||||
scope,
|
||||
"@local:example.com",
|
||||
);
|
||||
|
||||
const alice$ =
|
||||
metadataStore.createAvatarUrlBehavior$("@alice:example.com");
|
||||
const alice$ = metadataStore.createAvatarUrlBehavior$(
|
||||
scope,
|
||||
"@alice:example.com",
|
||||
);
|
||||
|
||||
expectObservable(local$).toBe("a", {
|
||||
a: "mxc://example.com/@local:example.com",
|
||||
@@ -558,7 +572,7 @@ describe("MatrixMemberMetadata", () => {
|
||||
});
|
||||
|
||||
it("should update on avatar change and user join/leave", () => {
|
||||
withTestScheduler(({ behavior, schedule, expectObservable }) => {
|
||||
withTestScheduler(({ scope, behavior, schedule, expectObservable }) => {
|
||||
fakeMemberWith({ userId: "@carl:example.com" });
|
||||
fakeMemberWith({ userId: "@bob:example.com" });
|
||||
const memberships$ = behavior("ab-d", {
|
||||
@@ -585,9 +599,14 @@ describe("MatrixMemberMetadata", () => {
|
||||
},
|
||||
});
|
||||
|
||||
const bob$ = metadataStore.createAvatarUrlBehavior$("@bob:example.com");
|
||||
const carl$ =
|
||||
metadataStore.createAvatarUrlBehavior$("@carl:example.com");
|
||||
const bob$ = metadataStore.createAvatarUrlBehavior$(
|
||||
scope,
|
||||
"@bob:example.com",
|
||||
);
|
||||
const carl$ = metadataStore.createAvatarUrlBehavior$(
|
||||
scope,
|
||||
"@carl:example.com",
|
||||
);
|
||||
expectObservable(bob$).toBe("a---", {
|
||||
a: "mxc://example.com/@bob:example.com",
|
||||
});
|
||||
|
||||
@@ -22,8 +22,6 @@ import {
|
||||
} from "../../../utils/displayname";
|
||||
import { type Behavior } from "../../Behavior";
|
||||
|
||||
const logger = rootLogger.getChild("[MatrixMemberMetadata]");
|
||||
|
||||
export type RoomMemberMap = Map<
|
||||
string,
|
||||
Pick<RoomMember, "userId" | "getMxcAvatarUrl" | "rawDisplayName">
|
||||
@@ -67,6 +65,7 @@ export const memberDisplaynames$ = (
|
||||
memberships$: Behavior<Pick<CallMembership, "userId">[]>,
|
||||
roomMembers$: Behavior<RoomMemberMap>,
|
||||
): Behavior<Map<string, string>> => {
|
||||
const logger = rootLogger.getChild("[MatrixMemberMetadata]");
|
||||
// This map tracks userIds that at some point needed disambiguation.
|
||||
// This is a memory leak bound to the number of participants.
|
||||
// A call application will always increase the memory if there have been more members in a call.
|
||||
@@ -115,8 +114,14 @@ export const createMatrixMemberMetadata$ = (
|
||||
memberships$: Behavior<Pick<CallMembership, "userId">[]>,
|
||||
roomMembers$: Behavior<RoomMemberMap>,
|
||||
): {
|
||||
createDisplayNameBehavior$: (userId: string) => Behavior<string | undefined>;
|
||||
createAvatarUrlBehavior$: (userId: string) => Behavior<string | undefined>;
|
||||
createDisplayNameBehavior$: (
|
||||
scope: ObservableScope,
|
||||
userId: string,
|
||||
) => Behavior<string | undefined>;
|
||||
createAvatarUrlBehavior$: (
|
||||
scope: ObservableScope,
|
||||
userId: string,
|
||||
) => Behavior<string | undefined>;
|
||||
displaynameMap$: Behavior<Map<string, string>>;
|
||||
avatarMap$: Behavior<Map<string, string | undefined>>;
|
||||
} => {
|
||||
@@ -136,13 +141,13 @@ export const createMatrixMemberMetadata$ = (
|
||||
),
|
||||
);
|
||||
return {
|
||||
createDisplayNameBehavior$: (userId: string) =>
|
||||
createDisplayNameBehavior$: (scope: ObservableScope, userId: string) =>
|
||||
scope.behavior(
|
||||
displaynameMap$.pipe(
|
||||
map((displaynameMap) => displaynameMap.get(userId)),
|
||||
),
|
||||
),
|
||||
createAvatarUrlBehavior$: (userId: string) =>
|
||||
createAvatarUrlBehavior$: (scope: ObservableScope, userId: string) =>
|
||||
scope.behavior(
|
||||
roomMembers$.pipe(
|
||||
map((roomMembers) => roomMembers.get(userId)?.getMxcAvatarUrl()),
|
||||
|
||||
@@ -29,13 +29,13 @@ import {
|
||||
import { type ProcessorState } from "../../../livekit/TrackProcessorContext.tsx";
|
||||
import {
|
||||
areLivekitTransportsEqual,
|
||||
createMatrixLivekitMembers$,
|
||||
createRemoteMatrixLivekitMembers$,
|
||||
type RemoteMatrixLivekitMember,
|
||||
} from "./MatrixLivekitMembers.ts";
|
||||
import { createConnectionManager$ } from "./ConnectionManager.ts";
|
||||
import { membershipsAndTransports$ } from "../../SessionBehaviors.ts";
|
||||
import { constant } from "../../Behavior.ts";
|
||||
import { testJWTToken } from "../../../utils/test-fixtures.ts";
|
||||
import { localRtcMember, testJWTToken } from "../../../utils/test-fixtures.ts";
|
||||
|
||||
// Test the integration of ConnectionManager and MatrixLivekitMerger
|
||||
|
||||
@@ -130,14 +130,15 @@ test("bob, carl, then bob joining no tracks yet", () => {
|
||||
ownMembershipIdentity: ownMemberMock,
|
||||
});
|
||||
|
||||
const matrixLivekitMembers$ = createMatrixLivekitMembers$({
|
||||
const remoteMatrixLivekitMembers$ = createRemoteMatrixLivekitMembers$({
|
||||
scope: testScope,
|
||||
membershipsWithTransport$:
|
||||
membershipsAndTransports.membershipsWithTransport$,
|
||||
connectionManager,
|
||||
localUser: localRtcMember,
|
||||
});
|
||||
|
||||
expectObservable(matrixLivekitMembers$).toBe(vMarble, {
|
||||
expectObservable(remoteMatrixLivekitMembers$).toBe(vMarble, {
|
||||
a: expect.toSatisfy((e: Epoch<RemoteMatrixLivekitMember[]>) => {
|
||||
const items = e.value;
|
||||
expect(items.length).toBe(1);
|
||||
|
||||
Reference in New Issue
Block a user