LotusGuild/element-call
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

merge: upstream v0.25.0 into lotus

Browse Source 
Merge upstream element-hq/element-call tag v0.25.0 into the Lotus fork
(previous base: v0.20.1; actual merge-base v0.20.1-rc.1). Every Lotus
feature and all six io.lotus.* widget actions are preserved. Version
bumped to 0.25.0-lotus.1.

Conflict files and how each Lotus hunk was re-expressed:

* src/state/CallViewModel/remoteMembers/ConnectionFactory.ts
  Upstream moved echoCancellation/noiseSuppression/autoGainControl from
  constructor params (fed by URL params) to persisted Settings
  (settings.ts) with a developer-settings UI. The cinny host still drives
  these per call via URL params (noiseSuppression=false /
  autoGainControl=false when the in-source ML denoiser is active, so the
  model gets a raw mic) - taking upstream verbatim would silently break
  the ML denoise tier. Re-wired as AND semantics in generateRoomOption():
  a constraint is enabled only if BOTH the Setting and the URL param
  allow it. Params default to true, so with no params this is
  byte-for-byte upstream behaviour. Upstream's own echoCancellation /
  noiseSuppression URL params (still parsed but dead in v0.25.0) work
  again as a side effect. Lotus autoGainControl URL param kept in
  UrlParams.ts (auto-merged, unchanged).

* src/state/CallViewModel/remoteMembers/ECConnectionFactory.test.ts
  Took upstream (tests now drive via Settings). The lost Lotus coverage
  is restored in a NEW colocated file src/lotus/lotusAudioConstraints.test.ts
  (3 tests) so the upstream test file stays pristine. Verified the new
  test fails against pure-upstream ConnectionFactory and passes with the
  re-wiring.

* src/state/CallViewModel/CallViewModel.ts
  Three small hunks: kept both the Lotus `userMedia$` interface member
  and upstream's new `keyRotationSuppressed$`; dropped the three Lotus
  audio constructor args (mechanism removed upstream, see above); kept
  both in the returned object. The [lotus #4] overrideSpotlight$ routing,
  manualSpotlightUserId$ and setManualSpotlight auto-merged; verified
  against upstream's changed ringingMedia$ (now single-or-null instead
  of array) - the merge correctly took upstream's outer branch and the
  inner screenShares$/spotlightSpeaker$ logic that lotusSpotlight.ts
  mirrors is unchanged upstream.

* src/index.css
  Kept both: Lotus lotus-transparent / lotus-theme blocks and upstream's
  new body[data-background="gradient"]::before full-viewport gradient.
  The naive merge swallowed the closing brace of body.lotus-theme -
  restored. Added a rule hiding the new gradient pseudo-element under
  body.lotus-transparent, since it would otherwise paint over the
  transparent body and hide the host wallpaper.

* src/components/CallFooterViewModel.tsx, src/components/CallFooter.stories.tsx
  No Lotus content - pure upstream-vs-upstream conflicts caused by the
  merge base being v0.20.1-rc.1. Took upstream (layoutMode ->
  layoutSwitchVm; setLayoutMode removed). No Lotus code uses
  setGridMode/layoutMode.

Non-conflicting but reviewed:

* src/widget.ts auto-merged cleanly. Upstream's removal of .well-known
  transport advertisement and the new RTC-transport capability request
  did not touch the action registration loop the LOTUS_TO_WIDGET_ACTIONS
  spread and widget.lazyActions ride on - nothing to re-wire.
* src/room/InCallView.tsx, src/useAudioContext.tsx, src/useTheme.ts,
  src/tile/MediaView.tsx(+.module.css), src/UrlParams.ts(+test),
  all *.module.css and .gitea/workflows/ci.yml auto-merged; each diff
  against v0.25.0 was checked to equal the original Lotus hunk.
* src/button/Button.module.css: the merge appended an exact duplicate
  of upstream's `.rotate`/`@keyframes spin` block (rc.1 merge-base
  artefact) - reset to upstream verbatim.
* src/grid/OneOnOnePortraitLayout.module.css was renamed upstream to
  OneOnOneMobileLayout.module.css; git followed the rename and the Lotus
  safe-area PiP inset fix applies there (the --content-inset-* vars it
  uses still exist upstream).

Tooling changes inherited from upstream that affect the fork:

* eslint + prettier were replaced by oxlint + oxfmt (`pnpm lint:oxlint`,
  `pnpm format:check`). oxlint flagged 10 issues, all in src/lotus/*:
  8x no-meaningless-void-operator (dropped the `void` before void-typed
  widget transport.reply / callbacks - no behaviour change), 1x
  consistent-type-imports (lotusWidget.ts: `import type`), and 2x
  unicorn/no-useless-spread in lotusAudioInject.ts which are FALSE
  POSITIVES - `[...activeClips]` is a required defensive copy because
  abort() deletes from the Set during iteration; suppressed with an
  explanatory eslint-disable-next-line. oxfmt reformatted 7 Lotus
  touched files (whitespace only).
* packageManager bumped by upstream to pnpm@11.21.0, which requires
  Node >= 22.13 (uses node:sqlite). Node 20 cannot run it; pnpm 10.33
  cannot read the new lockfile either (matrix-js-sdk is now a git
  dependency on develop, using a version-union pnpm 10 rejects). Fork CI
  already uses Node 24 (.node-version), so CI is unaffected.
* matrix-js-sdk is now github:matrix-org/matrix-js-sdk#develop (pinned
  by commit in pnpm-lock.yaml).

Lotus behaviour NOT preserved: none found.

Verification (Node 24.11.1, pnpm 11.21.0): pnpm install --frozen-lockfile
OK (lockfile taken from upstream unchanged, no regeneration needed);
tsc clean; oxlint clean; oxfmt --check clean; knip exit 0 (2 config
hints in upstream knip.ts only); vitest unit 84 files / 627 passed /
9 skipped; build:embedded OK, staged to embedded/web/dist (44M), all
six io.lotus.* action strings present in the bundle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
This commit is contained in:
Lotus CI
2026-09-12 11:35:57 -04:00
co-authored by Claude Fable 5.1
parent e36aef8aa3 efde51b8c1
commit c6f9727870
243 changed files with 13311 additions and 10280 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/state/CallViewModel/CallViewModel.ts
+304 -212
View File
@@ -29,7 +29,6 @@ import {
pairwise,
race,
scan,
skipWhile,
startWith,
Subject,
switchAll,
@@ -39,8 +38,9 @@ import {
tap,
throttleTime,
timer,
takeUntil,
} from "rxjs";
import { logger as rootLogger } from "matrix-js-sdk/lib/logger";
import { type Logger, logger as rootLogger } from "matrix-js-sdk/lib/logger";
import {
MembershipManagerEvent,
type LivekitTransportConfig,
@@ -60,17 +60,18 @@ import {
} from "../../utils/observable";
import {
duplicateTiles,
MatrixRTCMode,
playReactionsSound,
showReactions,
} from "../../settings/settings";
import { Config } from "../../config/Config";
import { MatrixRTCMode } from "../../config/ConfigOptions";
import { isFirefox, platform } from "../../Platform";
import { setPipEnabled$ } from "../../controls";
import { TileStore } from "../TileStore";
import { gridLikeLayout } from "../GridLikeLayout";
import { spotlightExpandedLayout } from "../SpotlightExpandedLayout";
import { oneOnOneLandscapeLayout } from "../OneOnOneLandscapeLayout";
import { oneOnOnePortraitLayout } from "../OneOnOnePortraitLayout";
import { oneOnOneDesktopLayout } from "../OneOnOneDesktopLayout";
import { oneOnOneMobileLayout } from "../OneOnOneMobileLayout";
import { pipLayout } from "../PipLayout";
import { type EncryptionSystem } from "../../e2ee/sharedKeyManagement";
import {
@@ -78,7 +79,7 @@ import {
type ReactionInfo,
type ReactionOption,
} from "../../reactions";
import { shallowEquals } from "../../utils/array";
import { shallowEquals as shallowArrayEquals } from "../../utils/array";
import { type MediaDevices } from "../MediaDevices";
import { constant, type Behavior } from "../Behavior";
import { E2eeType } from "../../e2ee/e2eeType";
@@ -88,12 +89,13 @@ import { getUrlParams, HeaderStyle } from "../../UrlParams";
import { type ProcessorState } from "../../livekit/TrackProcessorContext";
import { ElementWidgetActions, widget } from "../../widget";
import {
layoutShallowEquals,
type Alignment,
type GridLayoutMedia,
type Layout,
type LayoutMedia,
type OneOnOneLandscapeLayoutMedia,
type OneOnOnePortraitLayoutMedia,
type OneOnOneDesktopLayoutMedia,
type OneOnOneMobileLayoutMedia,
type SpotlightExpandedLayoutMedia,
type SpotlightLandscapeLayoutMedia,
type SpotlightPortraitLayoutMedia,
@@ -112,6 +114,7 @@ import {
type LocalTransport,
} from "./localMember/LocalTransport.ts";
import {
createKeyRotationSuppressed$,
createMemberships$,
membershipsAndTransports$,
} from "../SessionBehaviors.ts";
@@ -124,10 +127,9 @@ import {
createConnectionManager$,
} from "./remoteMembers/ConnectionManager.ts";
import {
createMatrixLivekitMembers$,
createRemoteMatrixLivekitMembers$,
type LocalMatrixLivekitMember,
type RemoteMatrixLivekitMember,
type MatrixLivekitMember,
} from "./remoteMembers/MatrixLivekitMembers.ts";
import {
type AutoLeaveReason,
@@ -141,7 +143,10 @@ import {
} from "./remoteMembers/MatrixMemberMetadata.ts";
import { Publisher } from "./localMember/Publisher.ts";
import { type Connection } from "./remoteMembers/Connection.ts";
import { createLayoutModeSwitch } from "./LayoutSwitch.ts";
import {
type LayoutSwitchViewModel,
createLayoutSwitchViewModel,
} from "../LayoutSwitchViewModel.ts";
import {
createWrappedUserMedia,
type WrappedUserMediaViewModel,
@@ -158,8 +163,8 @@ import {
createRingingMedia,
type RingingMediaViewModel,
} from "../media/RingingMediaViewModel.ts";
import { type GridTileViewModel } from "../TileViewModel.ts";
const logger = rootLogger.getChild("[CallViewModel]");
//TODO
// Larger rename
// member,membership -> rtcMember
@@ -204,12 +209,11 @@ const smallMobileCallThreshold = 3;
// with the interface
const showFooterMs = 4000;
export type GridMode = "grid" | "spotlight";
export type WindowMode = "normal" | "narrow" | "flat" | "pip";
interface LayoutScanState {
layout: Layout | null;
overflowing: boolean;
tiles: TileStore;
}
@@ -232,9 +236,13 @@ export interface CallViewModel {
// lifecycle
autoLeave$: Observable<AutoLeaveReason>;
/**
* Whether we are ringing a call recipient.
* View model for info relating to ringing, timing out, calling back, etc.
*/
ringing$: Behavior<boolean>;
ringingVm$: Behavior<RingingMediaViewModel | null>;
/**
* Which visual element the ringing status should be shown in.
*/
ringingStatusLocation: "app_bar" | "tile";
/** Observable that emits when the user should leave the call (hangup pressed, widget action, error).
* THIS DOES NOT LEAVE THE CALL YET. The only way to leave the call (send the hangup event) is
* - by ending the scope
@@ -305,11 +313,17 @@ export interface CallViewModel {
participantCount$: Behavior<number>;
/** [lotus] All participants' user media, exposed for the Lotus call-state widget bridge. */
userMedia$: Behavior<WrappedUserMediaViewModel[]>;
/**
* Whether the call has grown large enough that MatrixRTC has stopped rotating the media
* encryption key. While this is true the key in use is still shared with new joiners, but no new
* key is generated when someone joins or leaves.
*/
keyRotationSuppressed$: Behavior<boolean>;
allConnections$: Behavior<ConnectionManagerData>;
/** Participants sorted by livekit room so they can be used in the audio rendering */
livekitRoomItems$: Behavior<LivekitRoomItem[]>;
/** use the layout instead, this is just for the sdk export. */
matrixLivekitMembers$: Behavior<RemoteMatrixLivekitMember[]>;
remoteMatrixLivekitMembers$: Behavior<RemoteMatrixLivekitMember[]>;
localMatrixLivekitMember$: Behavior<LocalMatrixLivekitMember | null>;
/** List of participants raising their hand */
handsRaised$: Behavior<Record<string, RaisedHandInfo>>;
@@ -354,8 +368,7 @@ export interface CallViewModel {
showNameTags$: Behavior<boolean>;
spotlightExpanded$: Behavior<boolean>;
toggleSpotlightExpanded$: Behavior<(() => void) | null>;
gridMode$: Behavior<GridMode>;
setGridMode: (value: GridMode) => void;
layoutSwitchVm$: Behavior<LayoutSwitchViewModel | null>;
// header/footer visibility
showHeader$: Behavior<boolean>;
@@ -365,6 +378,15 @@ export interface CallViewModel {
* and header as overlays.
*/
edgeToEdge$: Behavior<boolean>;
/**
* Whether the call layout is overflowing the interface (causing it to scroll).
*/
overflowing$: Behavior<boolean>;
/**
* Whether modals such as settings and reactions should be accessible at all.
*/
showModals$: Behavior<boolean>;
settingsOpen$: Behavior<boolean>;
setSettingsOpen$: Behavior<(open: boolean) => void>;
@@ -416,6 +438,7 @@ export function createCallViewModel$(
reactionsSubject$: Observable<Record<string, ReactionInfo>>,
trackProcessorState$: Behavior<ProcessorState>,
): CallViewModel {
const logger = rootLogger.getChild("[CallViewModel]");
const client = matrixRoom.client;
const userId = client.getUserId();
const deviceId = client.getDeviceId();
@@ -425,9 +448,17 @@ export function createCallViewModel$(
const livekitKeyProvider = getE2eeKeyProvider(
options.encryptionSystem,
matrixRTCSession,
logger,
);
// matrix_rtc_mode in config.json overrides the user's Developer Settings choice.
// It is validated at config load (src/config/Config.ts) so the cast is safe.
const configMatrixRTCMode = Config.get().matrix_rtc_mode as
| MatrixRTCMode
| undefined;
const matrixRTCMode$ =
options.matrixRTCMode$ ?? constant(MatrixRTCMode.Legacy);
configMatrixRTCMode !== undefined
? constant(configMatrixRTCMode)
: (options.matrixRTCMode$ ?? constant(MatrixRTCMode.Compatibility));
// Each hbar seperates a block of input variables required for the CallViewModel to function.
// The outputs of this block is written under the hbar.
@@ -489,7 +520,6 @@ export function createCallViewModel$(
mode === MatrixRTCMode.Matrix_2_0
? JwtEndpointVersion.Matrix_2_0
: JwtEndpointVersion.Legacy,
useOldestMember: mode === MatrixRTCMode.Legacy,
}),
),
),
@@ -505,9 +535,6 @@ export function createCallViewModel$(
livekitKeyProvider,
getUrlParams().controlledAudioDevices,
options.livekitRoomFactory,
getUrlParams().echoCancellation,
getUrlParams().noiseSuppression,
getUrlParams().autoGainControl,
);
const connectionManager = createConnectionManager$({
@@ -530,13 +557,15 @@ export function createCallViewModel$(
ownMembershipIdentity,
});
const matrixLivekitMembers$: Behavior<Epoch<RemoteMatrixLivekitMember[]>> =
createMatrixLivekitMembers$({
scope: scope,
membershipsWithTransport$:
membershipsAndTransports.membershipsWithTransport$,
connectionManager: connectionManager,
});
const remoteMatrixLivekitMembers$: Behavior<
Epoch<RemoteMatrixLivekitMember[]>
> = createRemoteMatrixLivekitMembers$({
scope: scope,
membershipsWithTransport$:
membershipsAndTransports.membershipsWithTransport$,
connectionManager: connectionManager,
localUser: { userId, deviceId },
});
const connectOptions$ = scope.behavior(
matrixRTCMode$.pipe(
@@ -613,19 +642,12 @@ export function createCallViewModel$(
),
);
// ------------------------------------------------------------------------
// callLifecycle
// TODO if we are in "unknown" state we need a loading rendering (or empty screen)
// Otherwise it looks like we already connected and only than the ringing starts which is weird.
const { callPickupState$, autoLeave$ } = createCallNotificationLifecycle$({
scope: scope,
memberships$: memberships$,
sentCallNotification$: createSentCallNotification$(scope, matrixRTCSession),
receivedDecline$: createReceivedDecline$(matrixRoom),
options: options,
localUser: { userId: userId, deviceId: deviceId },
});
const matrixLivekitMembers$ = scope.behavior(
combineLatest(
[localMatrixLivekitMember$, remoteMatrixLivekitMembers$],
(local, remote) => [...(local === null ? [] : [local]), ...remote.value],
),
);
// ------------------------------------------------------------------------
// matrixMemberMetadataStore
@@ -637,11 +659,26 @@ export function createCallViewModel$(
matrixRoomMembers$,
);
// ------------------------------------------------------------------------
// callLifecycle
// TODO if we are in "unknown" state we need a loading rendering (or empty screen)
// Otherwise it looks like we already connected and only than the ringing starts which is weird.
const { ringAttempts$, autoLeave$ } = createCallNotificationLifecycle$({
scope,
memberships$,
matrixRoomMembers$,
sentCallNotification$: createSentCallNotification$(scope, matrixRTCSession),
receivedDecline$: createReceivedDecline$(matrixRoom),
options,
localUser: { userId, deviceId },
});
const allConnections$ = scope.behavior(
connectionManager.connectionManagerData$.pipe(map((d) => d.value)),
);
const livekitRoomItems$ = scope.behavior(
matrixLivekitMembers$.pipe(
remoteMatrixLivekitMembers$.pipe(
switchMap((members) => {
const a$ = combineLatest(
members.value.map((member) =>
@@ -707,43 +744,20 @@ export function createCallViewModel$(
* List of user media (camera feeds) that we want tiles for.
*/
const userMedia$ = scope.behavior<WrappedUserMediaViewModel[]>(
combineLatest([
localMatrixLivekitMember$,
matrixLivekitMembers$,
duplicateTiles.value$,
]).pipe(
combineLatest([matrixLivekitMembers$, duplicateTiles.value$]).pipe(
// Generate a collection of user media from the list of expected (whether
// present or missing) LiveKit participants.
generateItems(
"CallViewModel userMedia$",
function* ([
localMatrixLivekitMember,
matrixLivekitMembers,
duplicateTiles,
]) {
const computeMediaId = (m: MatrixLivekitMember): string =>
`${m.userId}:${m.membership$.value.deviceId}`;
const localUserMediaId = localMatrixLivekitMember
? computeMediaId(localMatrixLivekitMember)
: undefined;
const localAsArray = localMatrixLivekitMember
? [localMatrixLivekitMember]
: [];
const remoteWithoutLocal = matrixLivekitMembers.value.filter(
(m) => computeMediaId(m) !== localUserMediaId,
);
const allMatrixLivekitMembers = [
...localAsArray,
...remoteWithoutLocal,
];
for (const matrixLivekitMember of allMatrixLivekitMembers) {
const { userId, participant, connection$, membership$ } =
matrixLivekitMember;
const rtcId = membership$.value.rtcBackendIdentity; // rtcBackendIdentity
const mediaId = computeMediaId(matrixLivekitMember);
function* ([members, duplicateTiles]) {
for (const {
userId,
participant,
connection$,
membership$,
} of members) {
const rtcId = membership$.value.rtcBackendIdentity;
const mediaId = `${userId}:${membership$.value.deviceId}`;
for (let dup = 0; dup < 1 + duplicateTiles; dup++) {
yield {
keys: [dup, mediaId, userId, participant, connection$, rtcId],
@@ -769,11 +783,13 @@ export function createCallViewModel$(
pretendToBeDisconnected$: localMembership.reconnecting$,
displayName$: scope.behavior(
matrixMemberMetadataStore
.createDisplayNameBehavior$(userId)
.createDisplayNameBehavior$(scope, userId)
.pipe(map((name) => name ?? userId)),
),
mxcAvatarUrl$:
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
mxcAvatarUrl$: matrixMemberMetadataStore.createAvatarUrlBehavior$(
scope,
userId,
),
handRaised$: scope.behavior(
handsRaised$.pipe(map((v) => v[mediaId]?.time ?? null)),
),
@@ -785,49 +801,42 @@ export function createCallViewModel$(
),
);
const ringingMedia$ = scope.behavior<RingingMediaViewModel[]>(
combineLatest([userMedia$, matrixRoomMembers$, callPickupState$]).pipe(
generateItems(
"CallViewModel ringingMedia$",
function* ([userMedia, roomMembers, callPickupState]) {
if (
callPickupState === "ringing" ||
callPickupState === "timeout" ||
callPickupState === "decline"
) {
// TODO: Respect io.element.functional_members
for (const member of roomMembers.values()) {
if (!userMedia.some((vm) => vm.userId === member.userId))
yield {
keys: [member.userId],
data: callPickupState,
};
}
}
},
(scope, pickupState$, userId) =>
createRingingMedia({
id: `ringing:${userId}`,
userId,
displayName$: scope.behavior(
matrixRoomMembers$.pipe(
map((members) => members.get(userId)?.rawDisplayName || userId),
),
),
mxcAvatarUrl$:
matrixMemberMetadataStore.createAvatarUrlBehavior$(userId),
pickupState$,
muteStates,
}),
const ringingMedia$ = scope.behavior<RingingMediaViewModel | null>(
ringAttempts$.pipe(
switchMap(({ intent, recipient, outcome$ }) =>
outcome$.pipe(
startWith("ringing" as const),
generateItems(
"CallViewModel ringingMedia$",
function* (pickupState) {
if (pickupState !== "accept")
yield { keys: [intent, recipient], data: pickupState };
},
(scope, pickupState$, intent, userId) =>
createRingingMedia({
id: `ringing:${userId}`,
userId,
displayName$: scope.behavior(
matrixRoomMembers$.pipe(
map(
(members) =>
members.get(userId)?.rawDisplayName || userId,
),
),
),
mxcAvatarUrl$:
matrixMemberMetadataStore.createAvatarUrlBehavior$(
scope,
userId,
),
pickupState$,
intent,
}),
),
map(([media]) => media ?? null),
),
),
distinctUntilChanged(shallowEquals),
tap((ringingMedia) => {
if (ringingMedia.length > 1)
// Warn that UI may do something unexpected in this case
logger.warn(
`Ringing more than one participant is not supported (ringing ${ringingMedia.map((vm) => vm.userId).join(", ")})`,
);
}),
startWith(null),
),
);
@@ -866,14 +875,15 @@ export function createCallViewModel$(
* multiple devices.
*/
const participantCount$ = scope.behavior(
matrixLivekitMembers$.pipe(map((ms) => ms.value.length)),
matrixLivekitMembers$.pipe(map((ms) => ms.length)),
);
const leaveSoundEffect$ = combineLatest([callPickupState$, userMedia$]).pipe(
// Until the call is successful, do not play a leave sound.
// If callPickupState$ is null, then we always play the sound as it will not conflict with a decline sound.
skipWhile(([c]) => c !== null && c !== "success"),
map(([, userMedia]) => userMedia),
const keyRotationSuppressed$ = createKeyRotationSuppressed$(
scope,
matrixRTCSession,
);
const leaveSoundEffect$ = userMedia$.pipe(
pairwise(),
filter(
([prev, current]) =>
@@ -882,6 +892,9 @@ export function createCallViewModel$(
),
map(() => {}),
throttleTime(THROTTLE_SOUND_EFFECT_MS),
// Avoid doubling up on any auto-leave sounds (e.g. the decline sound),
// which are handled elsewhere
takeUntil(autoLeave$),
);
const userHangup$ = new Subject<void>();
@@ -960,13 +973,13 @@ export function createCallViewModel$(
bins.sort(([, bin1], [, bin2]) => bin1 - bin2).map(([m]) => m),
);
}),
distinctUntilChanged(shallowEquals),
distinctUntilChanged(shallowArrayEquals),
),
);
/**
* Local user media suitable for displaying in a PiP (undefined if not found
* or if user prefers to not see themselves).
* Local user media suitable for displaying in a PiP (undefined if not found,
* video is muted, or if user prefers to not see themselves).
*/
const localUserMediaForPip$ = scope.behavior<
LocalUserMediaViewModel | undefined
@@ -978,8 +991,10 @@ export function createCallViewModel$(
m.type === "user" && m.local,
);
if (!localUserMedia) return of(undefined);
return localUserMedia.alwaysShow$.pipe(
map((alwaysShow) => (alwaysShow ? localUserMedia : undefined)),
return combineLatest(
[localUserMedia.videoEnabled$, localUserMedia.alwaysShow$],
(videoEnabled, alwaysShow) =>
videoEnabled && alwaysShow ? localUserMedia : undefined,
);
}),
),
@@ -991,8 +1006,8 @@ export function createCallViewModel$(
}>(
ringingMedia$.pipe(
switchMap((ringingMedia) => {
if (ringingMedia.length > 0)
return of({ spotlight: ringingMedia, pip$: localUserMediaForPip$ });
if (ringingMedia !== null)
return of({ spotlight: [ringingMedia], pip$: localUserMediaForPip$ });
// [lotus #4] Route the screenshare/spotlight computation through the
// manual-spotlight wrapper. This is byte-for-byte upstream behaviour
@@ -1013,7 +1028,7 @@ export function createCallViewModel$(
const spotlight$ = scope.behavior<MediaViewModel[]>(
spotlightAndPip$.pipe(
map(({ spotlight }) => spotlight),
distinctUntilChanged<MediaViewModel[]>(shallowEquals),
distinctUntilChanged<MediaViewModel[]>(shallowArrayEquals),
),
);
@@ -1067,7 +1082,7 @@ export function createCallViewModel$(
spotlightExpandedToggle$,
);
const { setGridMode, gridMode$ } = createLayoutModeSwitch(
const layoutSwitchVm = createLayoutSwitchViewModel(
scope,
windowMode$,
hasRemoteScreenShares$,
@@ -1119,60 +1134,60 @@ export function createCallViewModel$(
),
);
const oneOnOneLayoutMedia$: Observable<{
const oneOnOneLayoutMedia$: Behavior<{
local: LocalUserMediaViewModel;
remote: UserMediaViewModel | RingingMediaViewModel;
} | null> = combineLatest([userMedia$, screenShares$]).pipe(
switchMap(([userMedia, screenShares]) => {
// One-on-one layout only supports 2 user media, no screen shares
if (userMedia.length <= 2 && screenShares.length === 0) {
const local = userMedia.find(
(vm): vm is WrappedUserMediaViewModel & LocalUserMediaViewModel =>
vm.type === "user" && vm.local,
);
if (local !== undefined) {
const remote = userMedia.find(
(vm): vm is WrappedUserMediaViewModel & RemoteUserMediaViewModel =>
vm.type === "user" && !vm.local,
} | null> = scope.behavior(
combineLatest([userMedia$, screenShares$]).pipe(
switchMap(([userMedia, screenShares]) => {
// One-on-one layout only supports 2 user media, no screen shares
if (userMedia.length <= 2 && screenShares.length === 0) {
const local = userMedia.find(
(vm): vm is WrappedUserMediaViewModel & LocalUserMediaViewModel =>
vm.type === "user" && vm.local,
);
if (remote !== undefined) return of({ local, remote });
// If there's no other user media in the call (could still happen in
// this branch due to the duplicate tiles option), we could possibly
// show ringing media instead
if (userMedia.length === 1)
return ringingMedia$.pipe(
map((ringingMedia) => {
return ringingMedia.length === 1
? {
local,
remote: ringingMedia[0],
}
: null;
}),
if (local !== undefined) {
const remote = userMedia.find(
(
vm,
): vm is WrappedUserMediaViewModel & RemoteUserMediaViewModel =>
vm.type === "user" && !vm.local,
);
}
}
return of(null);
}),
if (remote !== undefined) return of({ local, remote });
// If there's no other user media in the call (could still happen in
// this branch due to the duplicate tiles option), we could possibly
// show ringing media instead
if (userMedia.length === 1)
return ringingMedia$.pipe(
map(
(ringingMedia) =>
ringingMedia && { local, remote: ringingMedia },
),
);
}
}
return of(null);
}),
),
);
const oneOnOneLandscapeLayoutMedia$: Observable<OneOnOneLandscapeLayoutMedia | null> =
const oneOnOneDesktopLayoutMedia$: Observable<OneOnOneDesktopLayoutMedia | null> =
oneOnOneLayoutMedia$.pipe(
map((media) => {
if (media === null) return null;
return media.remote.type === "ringing"
? {
type: "one-on-one-landscape" as const,
type: "one-on-one-desktop" as const,
edgeToEdge: false,
spotlight: media.local,
pip: media.remote,
}
: {
type: "one-on-one-landscape" as const,
type: "one-on-one-desktop" as const,
edgeToEdge: false,
spotlight: media.remote,
pip: media.local,
@@ -1180,13 +1195,13 @@ export function createCallViewModel$(
}),
);
const oneOnOnePortraitLayoutMedia$: Observable<OneOnOnePortraitLayoutMedia | null> =
const oneOnOneMobileLayoutMedia$: Observable<OneOnOneMobileLayoutMedia | null> =
oneOnOneLayoutMedia$.pipe(
switchMap((media) => {
if (media === null) return of(null);
return media.local.videoEnabled$.pipe(
map((videoEnabled) => ({
type: "one-on-one-portrait" as const,
type: "one-on-one-mobile" as const,
edgeToEdge: true as const,
spotlight: media.remote,
pip: videoEnabled ? media.local : undefined,
@@ -1203,6 +1218,34 @@ export function createCallViewModel$(
})),
);
spotlight$
.pipe(
switchMap((media) => {
let layout;
const pipMedia = media[0];
if (pipMedia === undefined) return of(undefined);
switch (pipMedia.type) {
case "user":
layout = pipMedia.videoOrientation$;
break;
case "ringing":
layout = of("landscape" as const);
break;
case "screen share":
layout = of("landscape" as const);
break;
}
return layout;
}),
distinctUntilChanged(),
scope.bind(),
)
.subscribe((orientation) => {
if (orientation === undefined) return;
logger.info("controls api pip orientation updated:", orientation);
window.controls.onPipMediaOrientationUpdate?.(orientation);
});
/**
* The media to be used to produce a layout.
*/
@@ -1211,11 +1254,11 @@ export function createCallViewModel$(
switchMap((windowMode) => {
switch (windowMode) {
case "normal":
return gridMode$.pipe(
switchMap((gridMode) => {
switch (gridMode) {
return layoutSwitchVm.layout$.pipe(
switchMap((layout) => {
switch (layout) {
case "grid":
return oneOnOneLandscapeLayoutMedia$.pipe(
return oneOnOneDesktopLayoutMedia$.pipe(
switchMap((oneOnOne) =>
oneOnOne === null ? gridLayoutMedia$ : of(oneOnOne),
),
@@ -1232,7 +1275,7 @@ export function createCallViewModel$(
}),
);
case "narrow":
return oneOnOnePortraitLayoutMedia$.pipe(
return oneOnOneMobileLayoutMedia$.pipe(
switchMap((oneOnOne) =>
oneOnOne === null
? combineLatest([grid$, spotlight$], (grid, spotlight) =>
@@ -1245,17 +1288,23 @@ export function createCallViewModel$(
),
);
case "flat":
return gridMode$.pipe(
switchMap((gridMode) => {
switch (gridMode) {
case "grid":
// Yes, grid mode actually gets you a "spotlight" layout in
// this window mode.
return spotlightLandscapeLayoutMedia$(true);
case "spotlight":
return spotlightExpandedLayoutMedia$(true);
}
}),
return oneOnOneMobileLayoutMedia$.pipe(
switchMap((oneOnOne) =>
oneOnOne === null
? layoutSwitchVm.layout$.pipe(
switchMap((layout) => {
switch (layout) {
case "grid":
// Yes, grid mode actually gets you a "spotlight" layout in
// this window mode.
return spotlightLandscapeLayoutMedia$(true);
case "spotlight":
return spotlightExpandedLayoutMedia$(true);
}
}),
)
: of(oneOnOne),
),
);
case "pip":
return pipLayoutMedia$;
@@ -1283,8 +1332,8 @@ export function createCallViewModel$(
// indicators. And in one-on-one layout there's no question as to who is
// speaking.
case "spotlight-expanded":
case "one-on-one-landscape":
case "one-on-one-portrait":
case "one-on-one-desktop":
case "one-on-one-mobile":
return false;
default:
return true;
@@ -1296,7 +1345,7 @@ export function createCallViewModel$(
const showNameTags$ = scope.behavior<boolean>(
layoutMedia$.pipe(
switchMap((l) =>
l.type === "pip" || l.type === "one-on-one-portrait"
l.type === "pip" || l.type === "one-on-one-mobile"
? matrixRoomMembers$.pipe(
map(
(members) =>
@@ -1340,6 +1389,22 @@ export function createCallViewModel$(
layoutMedia$.pipe(map(({ edgeToEdge }) => edgeToEdge)),
);
// Only show the layout switch in cases where it has an effect on the layout
const showLayoutSwitch$ = windowMode$.pipe(
switchMap((windowMode) => {
switch (windowMode) {
case "normal":
return of(true);
case "flat":
return oneOnOneLayoutMedia$.pipe(
map((oneOnOne) => oneOnOne === null),
);
default:
return of(false);
}
}),
);
const screenTap$ = new Subject<void>();
const controlsTap$ = new Subject<void>();
const screenHover$ = new Subject<void>();
@@ -1412,6 +1477,11 @@ export function createCallViewModel$(
map((naturallyShowFooter) => naturallyShowFooter && showFooterUrlParams),
),
);
const showModals$ = scope.behavior(
windowMode$.pipe(map((mode) => mode !== "pip")),
);
const settingsOpen$ = new BehaviorSubject(false);
const setSettingsOpen$ = constant((open: boolean) => {
settingsOpen$.next(open);
@@ -1421,7 +1491,7 @@ export function createCallViewModel$(
windowMode$.pipe(
switchMap((mode) => {
// In small windows the header would be too obstructive
if (mode === "pip" || mode === "flat") return of(false);
if (mode === "pip") return of(false);
// In edge-to-edge layouts, couple the visibility of the header
// to that of the footer
return edgeToEdge$.pipe(
@@ -1461,7 +1531,7 @@ export function createCallViewModel$(
// There is a cyclical dependency here: the layout algorithms want to know
// which tiles are on screen, but to know which tiles are on screen we have to
// first render a layout. To deal with this we assume initially that no tiles
// first render a layout. To deal with this we assume initially that all tiles
// are visible, and loop the data back into the layouts with a Subject.
const visibleTiles$ = new Subject<number>();
const setVisibleTiles = (value: number): void => visibleTiles$.next(value);
@@ -1469,7 +1539,7 @@ export function createCallViewModel$(
const layoutInternals$ = scope.behavior<LayoutScanState & { layout: Layout }>(
combineLatest([
layoutMedia$,
visibleTiles$.pipe(startWith(0), distinctUntilChanged()),
visibleTiles$.pipe(startWith(Infinity), distinctUntilChanged()),
]).pipe(
scan<
[LayoutMedia, number],
@@ -1479,6 +1549,8 @@ export function createCallViewModel$(
({ tiles: prevTiles }, [media, visibleTiles]) => {
let layout: Layout;
let newTiles: TileStore;
let pip: GridTileViewModel | undefined;
let overflowing = false;
switch (media.type) {
case "grid":
case "spotlight-landscape":
@@ -1490,6 +1562,7 @@ export function createCallViewModel$(
setVisibleTiles,
prevTiles,
);
overflowing = newTiles.gridTiles.length > visibleTiles;
break;
case "spotlight-expanded":
[layout, newTiles] = spotlightExpandedLayout(
@@ -1498,29 +1571,35 @@ export function createCallViewModel$(
prevTiles,
);
break;
case "one-on-one-landscape":
[layout, newTiles] = oneOnOneLandscapeLayout(
case "one-on-one-desktop":
[layout, newTiles] = oneOnOneDesktopLayout(
media,
landscapePipAlignment$,
prevTiles,
);
pip = layout.pip;
break;
case "one-on-one-portrait":
[layout, newTiles] = oneOnOnePortraitLayout(
case "one-on-one-mobile":
[layout, newTiles] = oneOnOneMobileLayout(
media,
portraitPipSize$,
portraitPipAlignment$,
prevTiles,
);
pip = layout.pip;
break;
case "pip":
[layout, newTiles] = pipLayout(media, prevTiles);
break;
}
return { layout, tiles: newTiles };
for (const tile of newTiles.gridTiles) {
tile.setShowOutline(tile === pip);
}
return { layout, overflowing, tiles: newTiles };
},
{ layout: null, tiles: TileStore.empty() },
{ layout: null, overflowing: false, tiles: TileStore.empty() },
),
),
);
@@ -1529,7 +1608,15 @@ export function createCallViewModel$(
* The layout of tiles in the call interface.
*/
const layout$ = scope.behavior<Layout>(
layoutInternals$.pipe(map(({ layout }) => layout)),
layoutInternals$.pipe(
map(({ layout }) => layout),
// Drop redundant layout updates before they would hit React.
distinctUntilChanged<Layout>(layoutShallowEquals),
),
);
const overflowing$ = scope.behavior<boolean>(
layoutInternals$.pipe(map(({ overflowing }) => overflowing)),
);
/**
@@ -1701,9 +1788,9 @@ export function createCallViewModel$(
return {
autoLeave$: autoLeave$,
ringing$: scope.behavior(
callPickupState$.pipe(map((state) => state === "ringing")),
),
ringingVm$: ringingMedia$,
ringingStatusLocation:
urlParams.header === HeaderStyle.AppBar ? "app_bar" : "tile",
leave$: leave$,
hangup: (): void => userHangup$.next(),
join: localMembership.requestJoinAndPublish,
@@ -1736,6 +1823,7 @@ export function createCallViewModel$(
allConnections$,
participantCount$: participantCount$,
userMedia$,
keyRotationSuppressed$: keyRotationSuppressed$,
handsRaised$: handsRaised$,
reactions$: reactions$,
joinSoundEffect$: joinSoundEffect$,
@@ -1747,12 +1835,13 @@ export function createCallViewModel$(
spotlightExpanded$: spotlightExpanded$,
toggleSpotlightExpanded$: toggleSpotlightExpanded$,
gridMode$: gridMode$,
setGridMode: setGridMode,
layoutSwitchVm$: scope.behavior(
showLayoutSwitch$.pipe(map((show) => (show ? layoutSwitchVm : null))),
),
layout$: layout$,
localMatrixLivekitMember$,
matrixLivekitMembers$: scope.behavior(
matrixLivekitMembers$.pipe(
remoteMatrixLivekitMembers$: scope.behavior(
remoteMatrixLivekitMembers$.pipe(
map((members) => members.value),
tap((v) => {
const listForLogs = v
@@ -1773,9 +1862,11 @@ export function createCallViewModel$(
showNameTags$,
showHeader$: showHeader$,
showFooter$: showFooter$,
showModals$,
settingsOpen$: settingsOpen$,
setSettingsOpen$: setSettingsOpen$,
edgeToEdge$,
overflowing$,
earpieceMode$: earpieceMode$,
audioOutputSwitcher$: audioOutputSwitcher$,
reconnecting$: localMembership.reconnecting$,
@@ -1787,6 +1878,7 @@ export function createCallViewModel$(
function getE2eeKeyProvider(
e2eeSystem: EncryptionSystem,
rtcSession: MatrixRTCSession,
logger: Logger,
): BaseKeyProvider | undefined {
if (e2eeSystem.kind === E2eeType.NONE) return undefined;