Commit Graph
167 Commits
Author SHA1 Message Date
Lotus CIandClaude Opus 5 e5d5f13923 fix(lotus): focus_participant works in grid/1:1, clears on leave, keeps PiP, pins by device
- A non-null pin forces layout "spotlight" and remembers the displaced
  mode; clearing restores it only if the user hasn't switched since;
  gridLayoutMedia$ surfaces the pinned item for narrow mode (#3).
- Pin clears when the user is gone for 5 s or on leave$ (#16).
- Screenshare branch keeps pip$ = auto speaker unless it IS the pinned
  user (#29).
- Payload accepts an optional media id (userId:deviceId) and prefers it;
  userId-only picks the speaking device (#30).
18 unit tests.

Fixes #3
Fixes #16
Fixes #29
Fixes #30

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-13 01:22:20 -04:00
Lotus CIandClaude Fable 5.1 c6f9727870 merge: upstream v0.25.0 into lotus
Merge upstream element-hq/element-call tag v0.25.0 into the Lotus fork
(previous base: v0.20.1; actual merge-base v0.20.1-rc.1). Every Lotus
feature and all six io.lotus.* widget actions are preserved. Version
bumped to 0.25.0-lotus.1.

Conflict files and how each Lotus hunk was re-expressed:

* src/state/CallViewModel/remoteMembers/ConnectionFactory.ts
  Upstream moved echoCancellation/noiseSuppression/autoGainControl from
  constructor params (fed by URL params) to persisted Settings
  (settings.ts) with a developer-settings UI. The cinny host still drives
  these per call via URL params (noiseSuppression=false /
  autoGainControl=false when the in-source ML denoiser is active, so the
  model gets a raw mic) - taking upstream verbatim would silently break
  the ML denoise tier. Re-wired as AND semantics in generateRoomOption():
  a constraint is enabled only if BOTH the Setting and the URL param
  allow it. Params default to true, so with no params this is
  byte-for-byte upstream behaviour. Upstream's own echoCancellation /
  noiseSuppression URL params (still parsed but dead in v0.25.0) work
  again as a side effect. Lotus autoGainControl URL param kept in
  UrlParams.ts (auto-merged, unchanged).

* src/state/CallViewModel/remoteMembers/ECConnectionFactory.test.ts
  Took upstream (tests now drive via Settings). The lost Lotus coverage
  is restored in a NEW colocated file src/lotus/lotusAudioConstraints.test.ts
  (3 tests) so the upstream test file stays pristine. Verified the new
  test fails against pure-upstream ConnectionFactory and passes with the
  re-wiring.

* src/state/CallViewModel/CallViewModel.ts
  Three small hunks: kept both the Lotus `userMedia$` interface member
  and upstream's new `keyRotationSuppressed$`; dropped the three Lotus
  audio constructor args (mechanism removed upstream, see above); kept
  both in the returned object. The [lotus #4] overrideSpotlight$ routing,
  manualSpotlightUserId$ and setManualSpotlight auto-merged; verified
  against upstream's changed ringingMedia$ (now single-or-null instead
  of array) - the merge correctly took upstream's outer branch and the
  inner screenShares$/spotlightSpeaker$ logic that lotusSpotlight.ts
  mirrors is unchanged upstream.

* src/index.css
  Kept both: Lotus lotus-transparent / lotus-theme blocks and upstream's
  new body[data-background="gradient"]::before full-viewport gradient.
  The naive merge swallowed the closing brace of body.lotus-theme -
  restored. Added a rule hiding the new gradient pseudo-element under
  body.lotus-transparent, since it would otherwise paint over the
  transparent body and hide the host wallpaper.

* src/components/CallFooterViewModel.tsx, src/components/CallFooter.stories.tsx
  No Lotus content - pure upstream-vs-upstream conflicts caused by the
  merge base being v0.20.1-rc.1. Took upstream (layoutMode ->
  layoutSwitchVm; setLayoutMode removed). No Lotus code uses
  setGridMode/layoutMode.

Non-conflicting but reviewed:

* src/widget.ts auto-merged cleanly. Upstream's removal of .well-known
  transport advertisement and the new RTC-transport capability request
  did not touch the action registration loop the LOTUS_TO_WIDGET_ACTIONS
  spread and widget.lazyActions ride on - nothing to re-wire.
* src/room/InCallView.tsx, src/useAudioContext.tsx, src/useTheme.ts,
  src/tile/MediaView.tsx(+.module.css), src/UrlParams.ts(+test),
  all *.module.css and .gitea/workflows/ci.yml auto-merged; each diff
  against v0.25.0 was checked to equal the original Lotus hunk.
* src/button/Button.module.css: the merge appended an exact duplicate
  of upstream's `.rotate`/`@keyframes spin` block (rc.1 merge-base
  artefact) - reset to upstream verbatim.
* src/grid/OneOnOnePortraitLayout.module.css was renamed upstream to
  OneOnOneMobileLayout.module.css; git followed the rename and the Lotus
  safe-area PiP inset fix applies there (the --content-inset-* vars it
  uses still exist upstream).

Tooling changes inherited from upstream that affect the fork:

* eslint + prettier were replaced by oxlint + oxfmt (`pnpm lint:oxlint`,
  `pnpm format:check`). oxlint flagged 10 issues, all in src/lotus/*:
  8x no-meaningless-void-operator (dropped the `void` before void-typed
  widget transport.reply / callbacks - no behaviour change), 1x
  consistent-type-imports (lotusWidget.ts: `import type`), and 2x
  unicorn/no-useless-spread in lotusAudioInject.ts which are FALSE
  POSITIVES - `[...activeClips]` is a required defensive copy because
  abort() deletes from the Set during iteration; suppressed with an
  explanatory eslint-disable-next-line. oxfmt reformatted 7 Lotus
  touched files (whitespace only).
* packageManager bumped by upstream to pnpm@11.21.0, which requires
  Node >= 22.13 (uses node:sqlite). Node 20 cannot run it; pnpm 10.33
  cannot read the new lockfile either (matrix-js-sdk is now a git
  dependency on develop, using a version-union pnpm 10 rejects). Fork CI
  already uses Node 24 (.node-version), so CI is unaffected.
* matrix-js-sdk is now github:matrix-org/matrix-js-sdk#develop (pinned
  by commit in pnpm-lock.yaml).

Lotus behaviour NOT preserved: none found.

Verification (Node 24.11.1, pnpm 11.21.0): pnpm install --frozen-lockfile
OK (lockfile taken from upstream unchanged, no regeneration needed);
tsc clean; oxlint clean; oxfmt --check clean; knip exit 0 (2 config
hints in upstream knip.ts only); vitest unit 84 files / 627 passed /
9 skipped; build:embedded OK, staged to embedded/web/dist (44M), all
six io.lotus.* action strings present in the bundle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-09-12 11:35:57 -04:00
Timo K. b0d098e065 Merge branch 'main' into toger5/add-key-rotation-participant-limit 2026-08-28 15:27:43 +02:00
Robin ede29bdf1b Remove MatrixRTC legacy mode
This is the mode in which we sent membership events with the 'oldest membership' transport selection algorithm, which stopped being the default back in version 0.21.0. Users will no longer be able to select this mode in developer settings, and admins will no longer be able to select legacy mode through the config either. The app will still continue to support *receiving* membership events with the 'oldest membership' transport selection algorithm from others, however.
2026-08-26 19:51:12 +02:00
Robin 1c03ad335c Hide reactions interface in PiP mode 2026-08-19 17:44:45 +02:00
Robin a1b37bb3c9 Performance: Ignore redundant layout updates
When someone starts or stops speaking, the layout will often be recomputed only to find out that there is ultimately no layout change. We can ignore these redundant updates to avoid re-rendering the InCallView and Grid components, which are relatively slow.

For that specific, common case, this reduces JS CPU usage by as much as 70% in my testing.
2026-08-18 08:42:59 +02:00
Timo K. 0aa9d94826 add key rotation debug information 2026-08-17 11:17:22 +02:00
Ryan Emmick f7cf674152 Merge remote-tracking branch 'upstream/livekit' into livekit 2026-08-05 11:55:38 -05:00
Robin 5a96d1769e Only show layout switch when it has an effect on the layout 2026-07-28 15:06:15 +02:00
Robin a443a55bec Factor out a layout switch view + view model
So it can easily be shown and hidden wholesale.
2026-07-28 15:06:15 +02:00
Robin aa6732da19 Use an actual one-on-one layout in landscape on mobile 2026-07-28 11:53:26 +02:00
Robin 0b21cff020 Rename one-on-one layouts to be orientation-agnostic 2026-07-27 18:28:52 +02:00
Robin 32e30dd4f5 Merge branch 'livekit' into mobile-gradient 2026-07-09 12:56:28 +02:00
Timo K. 2b84c587f2 lint 2026-07-07 19:45:22 +02:00
Timo K. 3cc396f42c fix getChild before rageshake setup 2026-07-07 19:05:10 +02:00
Lotus CIandClaude Opus 4.8 bb3fb7e573 refactor(lotus-spotlight): extract manual-override into src/lotus/lotusSpotlight.ts
Remove the rebase hazard from CallViewModel: upstream's spotlightSpeaker$
auto-selection had been renamed to autoSpotlightSpeaker$ and an inline
manual-override + screenshare-coexistence block was spliced into
spotlightAndPip$. Both diverge from upstream and would conflict on every
rebase.

Restore spotlightSpeaker$ to its byte-for-byte upstream form and move the
[lotus #4] override into a pure wrapper, overrideSpotlight$(), invoked at a
single call point in spotlightAndPip$. Behaviour is unchanged: identical to
upstream while manualSpotlightUserId$ is null (the default), and preserves the
"pin a participant" and "focus camera during screenshare" (#4 / A5) rules when
the host sends io.lotus.focus_participant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 23:56:03 -04:00
Lotus CIandClaude Opus 4.8 940d71da92 feat(lotus-denoise): AGC off for ML tier + init/build leak hardening
CI / Build embedded bundle (push) Successful in 2m34s
CI / Publish to Gitea npm registry (push) Has been skipped
AEC/AGC audit fix + two hardening items from the engine review.

- Add an `autoGainControl` capture param (UrlParams -> CallViewModel ->
  ConnectionFactory audioCaptureDefaults), mirroring echoCancellation/
  noiseSuppression. Defaults true (unchanged); the host sets it false only for
  the ML tier so the browser's auto gain control doesn't fight the in-source ML
  denoiser (pumping). Echo cancellation stays on. Tests cover the URL parse and
  the audioCaptureDefaults wiring.
- L1: init() now closes the owned AudioContext on a build failure (was orphaned;
  browsers cap live contexts, so repeated failures could exhaust them).
- L2: buildGraph() disposes its partially-built nodes on failure (disposeGraph
  previously only cleaned the prior graph).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 00:46:39 -04:00
Robin 900d0abd27 Let page background shine through the footer
By visually separating it from other page content with an additional gradient background only when necessary.
2026-06-30 16:29:20 +02:00
Lotus CIandClaude Opus 4.8 2ab9427c09 lotus(#4): pin camera into spotlight during screenshare (A5)
CI / Build embedded bundle (push) Successful in 51s
CI / Publish to Gitea npm registry (push) Has been skipped
Review found the manual pin only chose among cameras when no screenshare
was active; during a screenshare the screenshare won the spotlight and the
pinned camera was demoted to an ignored PiP. Apply the override at the
spotlightAndPip$ level: when a pin is explicitly set, surface that camera
in the spotlight alongside the shared screen. No manual pin = unchanged.

Note: a pin persists if the pinned user briefly leaves and rejoins; the
host clears it via focus_participant{userId:null} (by design).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 23:21:14 -04:00
Lotus CIandClaude Opus 4.8 9d7784b9bc lotus(#4): focus-participant spotlight via widget action
Adds io.lotus.focus_participant (toWidget): the host can pin a participant
to the spotlight by Matrix user id (or clear with userId:null), via a
manual override injected into CallViewModel.spotlightSpeaker$. Replaces
cinny's fragile DOM .click() tile-selector focus hack. Extracts the action
enum into lotusActions.ts (no circular import) and allow-lists Lotus
toWidget actions in initializeWidget. Additive: no-op unless the host
sends the action.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 23:10:16 -04:00
Lotus CIandClaude Opus 4.8 444af5f7ac lotus(#2): stream per-participant call state over widget API
CI / Build embedded bundle (push) Successful in 1m45s
CI / Publish to Gitea npm registry (push) Has been skipped
Opt-in (lotusCallState=1) bridge that emits io.lotus.call_state with each
participant's speaking/audio/video state, so the Lotus host can drive
speaking rings / mute badges / PiP from real events instead of scraping
EC's rendered DOM. Exposes vm.userMedia$ on the public CallViewModel.
Additive: no-op without the flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-29 23:05:20 -04:00
Robin 8ab7f89e36 Hide local tile from mobile voice calls in landscape orientation
To match the designs. Just like in portrait, it wouldn't contribute anything meaningful.
2026-06-25 20:29:24 +02:00
Johannes Marbach 3fb81032e0 Merge pull request #4054 from element-hq/johannes/pip-border
Add PiP border in 1on1 layouts
2026-06-23 12:38:24 +02:00
Johannes Marbach 93fae5dd67 Add PiP border in 1on1 layouts
Fixes: #4008
2026-06-23 08:56:41 +02:00
Robin 504888b359 Show the subtitle of the app bar on small screens
Previously we were hiding the entire app bar on mobile phones in landscape orientation. However now that the app bar supports a small 'subtitle' element, we should show only the subtitle in this case to match the designs.

The subtitle still hides on tap, just like the footer.
2026-06-22 18:21:17 +02:00
Robin a456e47796 Show ringing status even on spotlight tiles (except in app bar mode) 2026-06-22 12:55:37 +02:00
Robin 22ff0d34b7 Merge branch 'livekit' into header-subtitle 2026-06-22 11:35:52 +02:00
Robin e11c04ac87 Move ringing status indicator to header on mobile
On mobile, the ringing status indicator is supposed to display in the header rather than on a tile. The exact layout differs between Android and iOS. To get it right I had to refactor AppBar to use CSS grid templates.

(Also, I changed my mind about the exact ringing data I needed out of CallViewModel - sorry. A little move of the ringtone audio renderer into its own component was necessary to accommodate that.)
2026-06-18 17:41:35 +02:00
Robin e77d143ce1 Clarify which Matrix-LiveKit members are remote
It was rather confusing that matrixLivekitMembers$ gives you objects of type RemoteMatrixLivekitMembers and yet the *local* member would often be among these. I've attempted to clear this up. To my knowledge this wasn't creating any bugs.
2026-06-18 12:10:32 +02:00
Robin 6af72f4c90 Format 2026-06-18 11:42:02 +02:00
Robin 8cda9777a5 Merge branch 'livekit' into ringing-intent 2026-06-18 06:41:17 +02:00
Robin 3ef3ebe897 Merge pull request #4036 from element-hq/metadata-leak
Fix a minor resource leak with display names and avatars
2026-06-12 10:05:50 +02:00
Timo 0c284ed75b Merge pull request #4004 from element-hq/toger5/controls-api-onPipMediaOrientation
Controls api `onPipMediaOrientation`
2026-06-12 16:03:35 +08:00
Ryan Emmick 08d9982981 Merge upstream/livekit into branch 2026-06-11 22:11:21 -05:00
Robin 96a9115ee7 Fix a minor resource leak with display names and avatars
I noticed that calls to createDisplayNameBehavior$ and createAvatarUrlBehavior$ were technically leaking resources since they reused the ObservableScope from their outer scope, which in practice lasts for the entire lifetime of the CallViewModel. This would not have had any noticeable effect unless you had other participants leave and rejoin the same call many thousands of times.
2026-06-11 12:21:07 +02:00
Robin 2ac6cdeb46 Improve internal model of ringing, expose ringing intent to call UI
I found our code's internal model of ringing a little overgrown (it had superfluous states like 'unknown') and difficult to extend with metadata or callbacks relating to ring attempts. By modeling ringing instead as a stream of ring attempts, where each attempt has an intent, a recipient, and an eventual outcome (accept/decline/timeout), I find it more natural to work with.

This makes room for a future 'try again' callback to allow ringing someone again after a timeout, and also forced me to look for a simpler solution to the duplicate leave sound effects. I exposed the intent of the ringing attempt to the call UI so I can later use it in the header.
2026-06-11 12:12:27 +02:00
fkwpandJohannes Marbach b99c8821d3 Add matrix_rtc_mode config option (#4014)
* Move MatrixRTCMode enum from settings.ts to ConfigOptions.ts

* Add matrix_rtc_mode config option

* add matrix_rtc_mode to config.sample.json

* Update src/settings/DeveloperSettingsTab.tsx

Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>

* Update src/settings/DeveloperSettingsTab.test.tsx

Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>

* reviewer comments

---------

Co-authored-by: Johannes Marbach <n0-0ne+github@mailbox.org>
2026-06-05 09:49:43 +00:00
Timo K 308a042076 fix error with no media 2026-06-03 10:43:47 +02:00
Timo K b86909af52 temp add tests 2026-06-02 10:56:34 +02:00
Timo K a6fc710d8c Add onPipMediaOrientationUpdate to controls api. 2026-06-01 16:12:57 +02:00
Timo ee1e8c60e7 Merge pull request #3961 from element-hq/toger5/view-model-call-footer-example
Implement fast switcher (+ ViewModel with snapshot example)
2026-05-20 23:38:00 +08:00
Timo K d9c0de73b7 Merge branch 'livekit' into toger5/view-model-call-footer-example 2026-05-20 16:28:04 +02:00
Robin 8d07d552d7 Ensure landscape layout is always edge-to-edge on mobile
The spotlight landscape layout was edge-to-edge previously, and should remain that way.
2026-05-20 16:11:07 +02:00
Timo K ec3b52f963 del: windowMode$ from public callViewModel
replace with edgeToEdge$
2026-05-20 15:00:28 +02:00
Valere FedronicandRobin 2fb5de3ba2 review: eqeqe
Co-authored-by: Robin <robin@robin.town>
2026-05-20 14:50:56 +02:00
Timo K c9fa586db1 merge fixups 2026-05-20 14:25:40 +02:00
Timo K 3301d2fe03 Merge branch 'livekit' into toger5/view-model-call-footer-example 2026-05-20 14:25:25 +02:00
Valere 265781ea5e fix(regression): control buttons should be hidden on mobile PIP 2026-05-20 13:08:51 +02:00
Timo K 88f660e43f review 2026-05-15 18:37:25 +02:00
Robin 9ad5d02e31 Merge branch 'livekit' into one-on-one-portrait 2026-05-15 11:28:47 +02:00