[research] Per-participant volume: is it remembered across calls, and where is it stored? #36

Closed
opened 2026-09-17 13:27:08 -04:00 by jared · 3 comments
Owner

Jared believes per-user volume is already persisted; confirm before building anything.

Questions (file:line evidence)

  1. Where does EC v0.25.0 keep the per-participant volume (setVolume on the remote audio track / MediaViewModel)? Is it a Setting (persisted localStorage) keyed by user or device id, or React state that dies with the call?
  2. If persisted: keyed how (userId vs userId:deviceId), and does it survive a page reload / a fork republish?
  3. Interaction with the Lotus deafen path (io.lotus.set_deafen uses setAudioEnabled$ / setVolume — does deafen clobber a stored per-user volume on undeafen)?
  4. Is the volume slider reachable in the fork's mobile UI?

Deliverable

A write-up here. If it is already persisted and keyed sensibly → close as "already works". If not → a small follow-up issue with the storage design (local only, never synced).

Jared believes per-user volume is already persisted; confirm before building anything. ### Questions (file:line evidence) 1. Where does EC v0.25.0 keep the per-participant volume (`setVolume` on the remote audio track / `MediaViewModel`)? Is it a Setting (persisted `localStorage`) keyed by user or device id, or React state that dies with the call? 2. If persisted: keyed how (userId vs `userId:deviceId`), and does it survive a page reload / a fork republish? 3. Interaction with the Lotus deafen path (`io.lotus.set_deafen` uses `setAudioEnabled$` / setVolume — does deafen clobber a stored per-user volume on undeafen)? 4. Is the volume slider reachable in the fork's mobile UI? ### Deliverable A write-up here. If it is already persisted and keyed sensibly → close as "already works". If not → a small follow-up issue with the storage design (local only, never synced).
jared added this to the Features 2026-Q4 milestone 2026-09-17 13:27:08 -04:00
jared added the priority: mediumarea: livekitresearch labels 2026-09-17 13:27:08 -04:00
jared self-assigned this 2026-09-17 13:27:08 -04:00
Author
Owner

Findings

Per-participant volume is NOT persisted today. It's pure in-memory RxJS state, scoped to the lifetime of a single call's tile, not a Setting written to localStorage.

1. Where the volume state actually lives

  • src/state/VolumeControls.ts:44-101 (createVolumeControls) — the volume/mute state is an RxJS Behavior built with accumulate({ volume: 1, committedVolume: 1 }, …) over three Subjects (toggleMuted$, adjustVolume$, commitVolume$). Every instance starts at volume: 1 — there is no read from storage anywhere in this function.
  • src/state/media/RemoteUserMediaViewModel.ts:39-56 (createRemoteUserMedia) — wires createVolumeControls's sink$ straight to LiveKit's RemoteParticipant.setVolume(volume) (line 53). No lookup/keying by userId or userId:deviceId happens here either.
  • src/state/media/WrappedUserMediaViewModel.ts:94-114 (createWrappedUserMedia) just forwards the scope it's given into createRemoteUserMedia; it doesn't add persistence.
  • src/state/CallViewModel/CallViewModel.ts:751-807 (userMedia$) is where that scope is actually created, via generateItems(..., mediaId, ...) with mediaId = \${userId}:${membership$.value.deviceId}`(line 765).generateItemsallocates a **new**ObservableScopeper distinct key and tears it down when the item disappears from the list (participant leaves the call / call ends). SinceCallViewModelitself is instantiated fresh per call, and the volumeBehaviorlives inside that per-item scope, the volume resets to1` (unmuted, full volume) on every new tile — including a rejoin within the same call, and certainly across separate calls.
  • src/settings/settings.ts:16-54 — the Setting<T> class is the fork's actual localStorage-persistence mechanism (this.key = \matrix-setting-${key}`;line 21,localStorage.setItem(...)line 54). Grepping the file for anything volume-related turns up exactly one hit:soundEffectVolumeatsrc/settings/settings.ts:132-133, a single **global** number (reaction/sound-effect volume), unrelated to per-participant playback volume. There is no Setting` for per-user/per-device volume anywhere in the tree.

Conclusion for Q1/Q2: it's plain React/RxJS state that dies with the call (in fact dies with the tile). Nothing is keyed by userId or userId:deviceId for this purpose, nothing survives a page reload, and there's nothing that would survive (or need to survive) a fork republish — because there is no stored key at all today.

2. Interaction with the Lotus deafen path

src/lotus/lotusDeafen.ts is actually where this exact clobbering hazard is already documented and was already fixed for the deafen case:

  • src/lotus/lotusDeafen.ts:35-41 (doc comment): "This deliberately does NOT use RemoteParticipant.setVolume for deafen any more: EC's per-participant volume slider / per-tile mute writes the same volumeMap from createVolumeControls... whenever its sink$ re-emits ... so a setVolume(0) deafen was silently undone for anyone joining while deafened, and an undeafen setVolume(1) clobbered the user's own per-tile volume/mute state."
  • Instead, deafen/undeafen now drives a global mute via setAudioEnabled$ (imported from ../controls, used at src/lotus/lotusDeafen.ts:18,93,106), which feeds muteAllAudio$ and is applied at the LivekitRoomAudioRenderer level — it never touches RemoteParticipant.setVolume or the per-tile VolumeControls state at all for the mic/deafen path. So today: no clobbering of per-user volume from the main deafen action.
  • The one remaining narrow exception: screenshareAudioMuted (a separate host control) still calls p.setVolume(0/1, Track.Source.ScreenShareAudio) directly at src/lotus/lotusDeafen.ts:116,119, bypassing VolumeControls entirely. The module's own comment at lines 50-53 calls this out as a known, accepted limitation: EC's screenshare-volume slider (src/state/media/RemoteScreenShareViewModel.ts:52) writes to the same LiveKit track and "a user who moves that slider while screenshare audio is host-muted wins" — but this is scoped to screenshare audio only, not the mic volume this issue is about.

3. Mobile reachability of the volume slider

Yes, it's reachable from a tap, not just right-click/hover:

  • src/tile/GridTile.tsx:355-409 (RemoteUserMediaTile) renders a <Slider> bound to vm.playbackVolume$ / vm.adjustPlaybackVolume / vm.commitPlaybackVolume inside a menuStart fragment (lines 383-405), alongside a "mute for me" ToggleMenuItem (lines 385-390).
  • That menu content is passed into UserMediaTile (src/tile/GridTile.tsx:237-266), which wraps it in both a <ContextMenu> (right-click/long-press) and an explicit <Menu trigger={menuTrigger}> button rendered as the tile's primaryButton (src/tile/GridTile.tsx:161-215) — an aria-label="common.options" "..." button. ContextMenu is passed hasAccessibleAlternative (line 261) specifically so the same menu is reachable via that tappable button, not only via a gesture a touchscreen may not deliver the same way.
  • Minor caveat (not the core answer, but worth flagging): the button's default CSS opacity is 0 outside of @media (hover)/focus/open state (src/tile/MediaView.module.css:280-306), so it's not visibly rendered on a stock mobile layout — it wants a visible-without-hover affordance for touch, but the wiring (component + accessible alternative) is there today, it's a small polish gap rather than a missing feature.

Verdict

Not persisted → proposed storage design (local only, never synced).

  • Add a new Setting<Record<string, number>> (e.g. perUserVolume, next to soundEffectVolume at src/settings/settings.ts:132), stored under matrix-setting-perUserVolume in localStorage per the existing Setting mechanism — local per-browser-profile, never round-tripped through room state or the widget API, satisfying "never synced".
  • Key by bare Matrix userId, not userId:deviceId. A volume preference is about the person, not their call-membership device id (which is realistically a per-session/reconnect identifier, not a stable hardware id) — keying by device would silently reset on every reconnect, defeating the point of the feature.
  • Hook the persistence into commitPlaybackVolume (write-through) and seed the accumulate initial state in createVolumeControls (src/state/VolumeControls.ts:54) from the stored map for that userId instead of the hardcoded 1, threading userId down from RemoteUserMediaViewModel/WrappedUserMediaViewModel.
  • Because this hooks the same commit path the slider already uses (not a low-level setVolume interceptor), it does not touch the Lotus deafen path — deafen no longer calls VolumeControls/setVolume for mic audio (see above), so no extra clobbering risk is introduced. The screenshare-mute sub-path remains a pre-existing, separately-scoped limitation.
  • Should add basic hygiene: cap map size / prune entries for users not seen recently, since localStorage is unbounded growth otherwise.
  • Effort estimate: small, roughly 0.5-1 day — one new Setting, a few lines threading userId into VolumeControls's init + commit, plus tests mirroring the existing coverage in src/state/media/MediaViewModel.test.ts:40-150 (which already exercises volume/mute/commit behavior thoroughly and would need a persistence-seed case added).
## Findings **Per-participant volume is NOT persisted today.** It's pure in-memory RxJS state, scoped to the lifetime of a single call's tile, not a `Setting` written to `localStorage`. ### 1. Where the volume state actually lives - `src/state/VolumeControls.ts:44-101` (`createVolumeControls`) — the volume/mute state is an RxJS `Behavior` built with `accumulate({ volume: 1, committedVolume: 1 }, …)` over three `Subject`s (`toggleMuted$`, `adjustVolume$`, `commitVolume$`). Every instance starts at `volume: 1` — there is no read from storage anywhere in this function. - `src/state/media/RemoteUserMediaViewModel.ts:39-56` (`createRemoteUserMedia`) — wires `createVolumeControls`'s `sink$` straight to LiveKit's `RemoteParticipant.setVolume(volume)` (line 53). No lookup/keying by `userId` or `userId:deviceId` happens here either. - `src/state/media/WrappedUserMediaViewModel.ts:94-114` (`createWrappedUserMedia`) just forwards the `scope` it's given into `createRemoteUserMedia`; it doesn't add persistence. - `src/state/CallViewModel/CallViewModel.ts:751-807` (`userMedia$`) is where that `scope` is actually created, via `generateItems(..., mediaId, ...)` with `mediaId = \`${userId}:${membership$.value.deviceId}\`` (line 765). `generateItems` allocates a **new** `ObservableScope` per distinct key and tears it down when the item disappears from the list (participant leaves the call / call ends). Since `CallViewModel` itself is instantiated fresh per call, and the volume `Behavior` lives inside that per-item scope, the volume resets to `1` (unmuted, full volume) on every new tile — including a rejoin within the *same* call, and certainly across separate calls. - `src/settings/settings.ts:16-54` — the `Setting<T>` class *is* the fork's actual localStorage-persistence mechanism (`this.key = \`matrix-setting-${key}\`;` line 21, `localStorage.setItem(...)` line 54). Grepping the file for anything volume-related turns up exactly one hit: `soundEffectVolume` at `src/settings/settings.ts:132-133`, a single **global** number (reaction/sound-effect volume), unrelated to per-participant playback volume. There is no `Setting` for per-user/per-device volume anywhere in the tree. **Conclusion for Q1/Q2:** it's plain React/RxJS state that dies with the call (in fact dies with the tile). Nothing is keyed by `userId` or `userId:deviceId` for this purpose, nothing survives a page reload, and there's nothing that would survive (or need to survive) a fork republish — because there is no stored key at all today. ### 2. Interaction with the Lotus deafen path `src/lotus/lotusDeafen.ts` is actually where this exact clobbering hazard is already documented and was already fixed for the deafen case: - `src/lotus/lotusDeafen.ts:35-41` (doc comment): *"This deliberately does NOT use `RemoteParticipant.setVolume` for deafen any more: EC's per-participant volume slider / per-tile mute writes the same `volumeMap` from `createVolumeControls`... whenever its `sink$` re-emits ... so a `setVolume(0)` deafen was silently undone for anyone joining while deafened, and an undeafen `setVolume(1)` clobbered the user's own per-tile volume/mute state."* - Instead, deafen/undeafen now drives a **global** mute via `setAudioEnabled$` (imported from `../controls`, used at `src/lotus/lotusDeafen.ts:18,93,106`), which feeds `muteAllAudio$` and is applied at the `LivekitRoomAudioRenderer` level — it never touches `RemoteParticipant.setVolume` or the per-tile `VolumeControls` state at all for the mic/deafen path. So today: **no clobbering** of per-user volume from the main deafen action. - The one remaining narrow exception: `screenshareAudioMuted` (a separate host control) still calls `p.setVolume(0/1, Track.Source.ScreenShareAudio)` directly at `src/lotus/lotusDeafen.ts:116,119`, bypassing `VolumeControls` entirely. The module's own comment at lines 50-53 calls this out as a known, accepted limitation: EC's screenshare-volume slider (`src/state/media/RemoteScreenShareViewModel.ts:52`) writes to the same LiveKit track and "a user who moves that slider while screenshare audio is host-muted wins" — but this is scoped to screenshare audio only, not the mic volume this issue is about. ### 3. Mobile reachability of the volume slider Yes, it's reachable from a tap, not just right-click/hover: - `src/tile/GridTile.tsx:355-409` (`RemoteUserMediaTile`) renders a `<Slider>` bound to `vm.playbackVolume$` / `vm.adjustPlaybackVolume` / `vm.commitPlaybackVolume` inside a `menuStart` fragment (lines 383-405), alongside a "mute for me" `ToggleMenuItem` (lines 385-390). - That `menu` content is passed into `UserMediaTile` (`src/tile/GridTile.tsx:237-266`), which wraps it in both a `<ContextMenu>` (right-click/long-press) **and** an explicit `<Menu trigger={menuTrigger}>` button rendered as the tile's `primaryButton` (`src/tile/GridTile.tsx:161-215`) — an `aria-label="common.options"` "..." button. `ContextMenu` is passed `hasAccessibleAlternative` (line 261) specifically so the same menu is reachable via that tappable button, not only via a gesture a touchscreen may not deliver the same way. - Minor caveat (not the core answer, but worth flagging): the button's default CSS opacity is 0 outside of `@media (hover)`/focus/open state (`src/tile/MediaView.module.css:280-306`), so it's not visibly rendered on a stock mobile layout — it wants a `visible-without-hover` affordance for touch, but the wiring (component + accessible alternative) is there today, it's a small polish gap rather than a missing feature. ## Verdict **Not persisted → proposed storage design (local only, never synced).** - Add a new `Setting<Record<string, number>>` (e.g. `perUserVolume`, next to `soundEffectVolume` at `src/settings/settings.ts:132`), stored under `matrix-setting-perUserVolume` in `localStorage` per the existing `Setting` mechanism — local per-browser-profile, never round-tripped through room state or the widget API, satisfying "never synced". - **Key by bare Matrix `userId`, not `userId:deviceId`.** A volume preference is about the person, not their call-membership device id (which is realistically a per-session/reconnect identifier, not a stable hardware id) — keying by device would silently reset on every reconnect, defeating the point of the feature. - Hook the persistence into `commitPlaybackVolume` (write-through) and seed the `accumulate` initial state in `createVolumeControls` (`src/state/VolumeControls.ts:54`) from the stored map for that `userId` instead of the hardcoded `1`, threading `userId` down from `RemoteUserMediaViewModel`/`WrappedUserMediaViewModel`. - Because this hooks the same commit path the slider already uses (not a low-level `setVolume` interceptor), it does **not** touch the Lotus deafen path — deafen no longer calls `VolumeControls`/`setVolume` for mic audio (see above), so no extra clobbering risk is introduced. The screenshare-mute sub-path remains a pre-existing, separately-scoped limitation. - Should add basic hygiene: cap map size / prune entries for users not seen recently, since `localStorage` is unbounded growth otherwise. - **Effort estimate: small, roughly 0.5-1 day** — one new `Setting`, a few lines threading `userId` into `VolumeControls`'s init + commit, plus tests mirroring the existing coverage in `src/state/media/MediaViewModel.test.ts:40-150` (which already exercises volume/mute/commit behavior thoroughly and would need a persistence-seed case added).
Author
Owner

Findings (2026-09-20, fork at 0.25.0-lotus.11)

1. Where the per-participant volume lives. src/state/VolumeControls.ts:13-36 — createVolumeControls builds playbackVolume$ with accumulate({ volume: 1, committedVolume: 1 }, …) inside the media view model's ObservableScope. The sink is RemoteParticipant.setVolume(volume) (state/media/RemoteUserMediaViewModel.ts:53) and setVolume(volume, Track.Source.ScreenShareAudio) for screenshares (RemoteScreenShareViewModel.ts:52). It is observable state scoped to the view model — no Setting, no localStorage, nothing keyed by user or device. It dies with the tile's view model, i.e. when that participant leaves or the call ends.

2. Persistence. None. Jared's belief is wrong for the fork and upstream: a per-user slider you drag resets to 1.0 on the next call, on a page reload, and even if the same participant reconnects mid-call (the remote view model is recreated → accumulate restarts at 1). Republishing the fork changes nothing here.

3. Deafen interaction. Safe: the Lotus deafen path (lotusDeafen.ts:29-42) deliberately uses setAudioEnabled$ (the renderer's muted prop) instead of setVolume, precisely so an undeafen doesn't clobber per-tile volumes; screenshare-audio-mute likewise goes through the renderer (lotusScreenshareAudio.ts). Within a call the slider value survives deafen/undeafen.

4. Mobile. The slider is in the tile's overflow menu (GridTile.tsx:363-374, SpotlightTile.tsx:331-343) and it is reachable on touch — the menu is the same component; the range input is finger-sized enough but not great.

Recommendation

Not "already works". Small follow-up (fork only, local only, never synced): a Setting<Record<string, number>> per-user-volume in settings.ts keyed by userId (not device — you want "Bob is loud" to stick when Bob switches devices), written on commitPlaybackVolume when the value ≠ 1, read as the accumulate seed in createRemoteUserMedia, capped at ~50 entries LRU. Screenshare audio is not remembered (it's per-share). ~40 lines + a test. Opened as a separate issue only if you want it — say the word.

## Findings (2026-09-20, fork at 0.25.0-lotus.11) **1. Where the per-participant volume lives.** `src/state/VolumeControls.ts:13-36` — `createVolumeControls` builds `playbackVolume$` with `accumulate({ volume: 1, committedVolume: 1 }, …)` inside the media view model's `ObservableScope`. The sink is `RemoteParticipant.setVolume(volume)` (`state/media/RemoteUserMediaViewModel.ts:53`) and `setVolume(volume, Track.Source.ScreenShareAudio)` for screenshares (`RemoteScreenShareViewModel.ts:52`). It is **observable state scoped to the view model** — no `Setting`, no `localStorage`, nothing keyed by user or device. It dies with the tile's view model, i.e. when that participant leaves or the call ends. **2. Persistence.** None. Jared's belief is wrong for the fork *and* upstream: a per-user slider you drag resets to 1.0 on the next call, on a page reload, and even if the same participant reconnects mid-call (the remote view model is recreated → `accumulate` restarts at 1). Republishing the fork changes nothing here. **3. Deafen interaction.** Safe: the Lotus deafen path (`lotusDeafen.ts:29-42`) deliberately uses `setAudioEnabled$` (the renderer's `muted` prop) instead of `setVolume`, precisely so an undeafen doesn't clobber per-tile volumes; screenshare-audio-mute likewise goes through the renderer (`lotusScreenshareAudio.ts`). Within a call the slider value survives deafen/undeafen. **4. Mobile.** The slider is in the tile's overflow menu (`GridTile.tsx:363-374`, `SpotlightTile.tsx:331-343`) and it is reachable on touch — the menu is the same component; the range input is finger-sized enough but not great. ## Recommendation Not "already works". Small follow-up (fork only, local only, never synced): a `Setting<Record<string, number>>` `per-user-volume` in `settings.ts` keyed by **userId** (not device — you want "Bob is loud" to stick when Bob switches devices), written on `commitPlaybackVolume` when the value ≠ 1, read as the `accumulate` seed in `createRemoteUserMedia`, capped at ~50 entries LRU. Screenshare audio is *not* remembered (it's per-share). ~40 lines + a test. Opened as a separate issue only if you want it — say the word.
Author
Owner

Done: the volume slider is remembered per person (v0.25.0-lotus.15, cinny e91b5fe1)

  • What's remembered: the per-participant volume, keyed by Matrix user id, so "Bob is loud" sticks when Bob switches devices. It survives the next call, a page reload, and Bob reconnecting mid-call.
  • Where it's stored: localStorage in this browser only, under lotus-per-user-volume. It is never synced to the server.
  • Limits: setting someone back to 100 % forgets them. At most 50 people are kept, and the least recently set is dropped first. Anything in storage that isn't a valid volume is ignored.
  • Screenshare audio is not remembered; it stays per-share, as recommended above.
  • Bug fixed along the way (also present upstream): with the keyboard, the slider commits before its last change reaches the view model. The committed volume, which is also what un-mute returns to, was one step behind, and pressing End to return to 100 % never registered. commitPlaybackVolume now takes the slider's committed value.

Verified in a two-person local call (Synapse + LiveKit):

  • Alice sets Bob to 70 % with the keyboard: Bob's audio element is at 0.70 and the stored value is 0.7.
  • After Alice leaves and rejoins, the slider and the audio are both at 0.70.
  • Pressing End removes the entry.

Tests: the memory module (LRU, reset, junk in storage), plus restore-remembered and commit-with-value in MediaViewModel.test.ts. The state, tile, room and lotus suites pass on Node 22 (410 tests).

(The first CI run for lotus.15 failed only on oxfmt; nothing was published. The formatting is fixed in 44023a4c, and the tag was re-pointed to it before it published.)

## Done: the volume slider is remembered per person (`v0.25.0-lotus.15`, cinny `e91b5fe1`) - **What's remembered:** the per-participant volume, keyed by **Matrix user id**, so "Bob is loud" sticks when Bob switches devices. It survives the next call, a page reload, and Bob reconnecting mid-call. - **Where it's stored:** localStorage in this browser only, under `lotus-per-user-volume`. It is never synced to the server. - **Limits:** setting someone back to 100 % forgets them. At most 50 people are kept, and the least recently set is dropped first. Anything in storage that isn't a valid volume is ignored. - **Screenshare audio is not remembered**; it stays per-share, as recommended above. - **Bug fixed along the way (also present upstream):** with the keyboard, the slider commits *before* its last change reaches the view model. The committed volume, which is also what un-mute returns to, was one step behind, and pressing End to return to 100 % never registered. `commitPlaybackVolume` now takes the slider's committed value. **Verified** in a two-person local call (Synapse + LiveKit): - Alice sets Bob to 70 % with the keyboard: Bob's audio element is at 0.70 and the stored value is 0.7. - After Alice leaves and rejoins, the slider and the audio are both at 0.70. - Pressing End removes the entry. **Tests:** the memory module (LRU, reset, junk in storage), plus restore-remembered and commit-with-value in `MediaViewModel.test.ts`. The state, tile, room and lotus suites pass on Node 22 (410 tests). (The first CI run for `lotus.15` failed only on `oxfmt`; nothing was published. The formatting is fixed in `44023a4c`, and the tag was re-pointed to it before it published.)
jared closed this issue 2026-09-25 10:09:26 -04:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: LotusGuild/element-call#36