When the voice-limit guard (or any JWT service) answers 403 with a reason —
"This voice channel is full.", "You don't have permission to …" — EC wrapped it
in FailToGetOpenIdToken and the user saw the generic error page with
OPEN_ID_ERROR. A 403 MatrixError with a non-empty `error` now throws
SFUTokenRefusedError ("Can't join this call" + the server's sentence), so the
reason is the description. Other failures are unchanged. Unit-tested.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
"Mute Screenshare Audio" (io.lotus.set_deafen screenshareAudioMuted) used
RemoteParticipant.setVolume(0, ScreenShareAudio). EC's own createVolumeControls
writes volume 1 through the same setter the moment a new screenshare media item
resolves, so when the sharer stopped and re-shared (or a late joiner shared)
the audio came back at full volume while the host button still said
"Unmute Screenshare Audio". Reproduced on the local calls stack with two
headless clients: after a re-share the screen_share_audio element read vol=1.
Now the flag is a global behavior (muteScreenshareAudio$) that
LivekitRoomAudioRenderer turns into the `muted` prop of every
Track.Source.ScreenShareAudio element — the exact mechanism deafen already uses
(pub.setEnabled(false): the server stops sending). Verified via the
RemoteTrackPublication behind each <audio>: the re-published track (new sid)
mounts with enabled=false while muted and re-enables on unmute; deafen +
undeafen leaves it muted; teardown resets the flag so the next call starts
clean. Unit tests updated; renderer test asserts only ScreenShareAudio
elements get muted by the new prop.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
audioCaptureDefaults set no channelCount, so Firefox captured a 2-channel
audio interface (Scarlett Solo, one XLR mic on input 1) as stereo and,
with browser audio processing off (the ML denoise tier), published it as
is: peers heard the speaker in the left ear only. Chrome downmixes such
captures itself, which is why it only showed on Firefox. Request
channelCount: 1 for mic capture; screenshare audio is captured separately
and is unaffected. Tested. Bumps to 0.25.0-lotus.3.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
- lotusTransparent without lotusTheme warns and applies the theme anyway;
name tags/header/footer get a subtle text-shadow + backdrop blur under
body.lotus-transparent (#21).
- Landscape filmstrip shrink requires (pointer: coarse) so a short
desktop/PiP window isn't reflowed as a phone (#32).
- lotusParam re-reads window.location on every call (#33).
Fixes#21Fixes#32Fixes#33
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
- Injection is gated on localParticipant.isMicrophoneEnabled and replies
{ played:false, reason:"muted" } (host UI follow-up in cinny) (#13).
- One lazily created module-level AudioContext/destination for all
clips, ref-counted and closed on last teardown; per clip only a
BufferSource + Gain. The replace-mode race handling is preserved and
three latent dangling-placeholder paths are closed (#14).
Fixes#13Fixes#14
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
The same maxBitrate was written into every encoding, so a 1.5 Mbps cap
could mean 4.5 Mbps aggregate. Distribute proportionally to the layers'
existing ratios (floor, so never over). Tested with a 3-layer case.
Fixes#12
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Lotus toWidget actions with no handler used to sit in the LazyEventEmitter
backlog forever (host timed out; stale replay on remount). They now get
an immediate {} reply. Adds RequestState and DenoiseState to the enum
(fromWidget) with a test pinning the toWidget/fromWidget split.
Fixes#18
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
- .lotusDecoration 50cqmin -> 62cqmin (cinny's inset ratio); onError
hides a broken image (#4).
- display:none under prefers-reduced-motion, matching the host (#19).
- safeImageUrl only accepts ALLOWED_DECORATION_ORIGINS (the decorations
CDN) plus blob: (#28).
- Roster is no longer wiped on last teardown; the handler sends
io.lotus.request_state on (re)registration so the host can re-push
decorations and the pin (#17 — host half in cinny).
Fixes#4Fixes#19Fixes#28Fixes#17
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
- A non-null pin forces layout "spotlight" and remembers the displaced
mode; clearing restores it only if the user hasn't switched since;
gridLayoutMedia$ surfaces the pinned item for narrow mode (#3).
- Pin clears when the user is gone for 5 s or on leave$ (#16).
- Screenshare branch keeps pip$ = auto speaker unless it IS the pinned
user (#29).
- Payload accepts an optional media id (userId:deviceId) and prefers it;
userId-only picks the speaking device (#30).
18 unit tests.
Fixes#3Fixes#16Fixes#29Fixes#30
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
- Assets (context, worklets, wasm, DFN core) are prepared as soon as the
flag is seen, so init() under LiveKit's trackChangeLock only wires
already-loaded pieces; resume timeout 3 s -> 500 ms (#7).
- init failure retries once with rnnoise; success/failure is reported to
the host as io.lotus.denoise_state so the UI can reflect reality (#8).
- Mic TrackMuted/TrackUnmuted suspend/resume the processor's context so
no inference runs on silence (#9).
- Every node is explicit mono; the dry path gets a per-model DelayNode so
the floor mix no longer comb-filters (#24, #25).
- DTLN/DFN dynamic imports are typed and their exports asserted at load,
feeding the #8 fallback instead of failing silently (#26).
Unit-tested (13 tests across the two files).
Fixes#7Fixes#8Fixes#9Fixes#24Fixes#25Fixes#26
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Gitea reserves secret names beginning with GITEA_, so the
`GITEA_NPM_TOKEN` this workflow has referenced since the fork's first
commit could never be created — which is why CI publishing never worked
and the June release was pushed by hand. Rename to NPM_PUBLISH_TOKEN
(org secret, write:package).
Refs #22
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Run #1861 built fine but `npm publish` refused: "You must specify a tag
using --tag when publishing a prerelease version" (npm 11 rule). Add
`--tag lotus`; cinny pins exact versions so the dist-tag is only a
namespace.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Run #1859: install, tsc, oxlint, knip and oxfmt all passed; the unit
suite then failed 1/648 — DeveloperSettingsTab's snapshot expects the
version to read "dev" when VITE_APP_VERSION is unset, but the workflow
exported it (= "lotus") at workflow level, so it leaked into the tests.
Set it on the two build steps only, which is the only place it matters.
Reproduced locally: the test fails with the variable set and passes
without it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Run #1858 got past install, tsc and oxlint; knip then failed with 1188
"unused files" — all under .pnpm-store/, because the Gitea runner keeps
pnpm's content store inside the checkout, and it now contains the
from-source matrix-js-sdk tree. knip honours .gitignore, so ignoring the
store directory (verified locally with a synthetic .pnpm-store) is enough.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Run #1855: `npm install -g pnpm@…` failed with EEXIST because the runner's
persistent hostedtoolcache Node install still carries the corepack `pnpm`
shim from earlier `corepack enable` runs, and npm won't overwrite it.
Disable/remove the shim first, then install, and print the version so the
log shows which pnpm actually ran.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Run #1854 failed in `pnpm install --frozen-lockfile`: matrix-js-sdk is a
git dependency (upstream pins matrix-org/matrix-js-sdk#develop) that pnpm
must build from source, and that source's devEngines pins pnpm 11.9.0.
Under `corepack enable` the nested `pnpm install` runs the 11.21.0 shim,
which refuses to switch versions ("pnpm does not switch versions when
running under corepack") and the prepare step aborts. Reproduced locally
with a fresh store; with a real pnpm binary the nested install downloads
11.9.0 itself and succeeds.
Both jobs now `npm install -g pnpm@<version>` with the version read from
package.json's packageManager field.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
The workflow only built and smoke-checked that dist/index.html existed,
so a type error, lint violation or failing test could be tagged straight
to a published npm release. Run `pnpm lint` (tsc + oxlint + knip),
`pnpm format:check` (oxfmt) and `pnpm test:unit --run` before the build,
all hard. Add the same cancel-in-progress concurrency group cinny's CI
uses so rapid pushes to lotus don't queue redundant full builds.
Also fixes the one tsc error the v0.25.0 merge surfaced in
lotusDecorations (lazyActions.off now returns the emitter), so the new
gate is green from its first run.
Fixes#5Fixes#34
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
`pnpm lint:eslint` and `pnpm prettier:check` were never run by fork CI (#5),
and lotus HEAD failed both: a missing return type in lotusDecorations, a
type-only import in lotusWidget, and formatting drift in lotusAudioInject,
lotusFocus, lotusWidget and .gitea/workflows/ci.yml. No behaviour change.
Both gates now pass clean so they can be turned on.
Refs #5
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Six fix(lotus) commits reached `lotus` while the upstream merge was in
progress (deafen via global output mute, denoise race/fallback fixes,
strictOriginCheck on the widget transport, call_state standalone skip,
set_quality null-clears-cap, plus four new src/lotus/*.test.ts files).
Merged them on top of the v0.25.0 sync so this branch is a superset of
current `lotus`.
One conflict, src/lotus/lotusDenoiseProcessor.ts: the sync commit had
dropped a meaningless `void` (oxlint no-meaningless-void-operator) on a
line the incoming commit also touched; kept the void-less form so oxlint
stays clean.
Verification after the fold-in (Node 24.11.1 / pnpm 11.21.0): tsc clean;
oxlint clean; oxfmt --check clean; knip exit 0; vitest unit 88 files /
639 passed / 9 skipped; build:embedded OK, staged to embedded/web/dist,
all six io.lotus.* actions present in the bundle.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Merge upstream element-hq/element-call tag v0.25.0 into the Lotus fork
(previous base: v0.20.1; actual merge-base v0.20.1-rc.1). Every Lotus
feature and all six io.lotus.* widget actions are preserved. Version
bumped to 0.25.0-lotus.1.
Conflict files and how each Lotus hunk was re-expressed:
* src/state/CallViewModel/remoteMembers/ConnectionFactory.ts
Upstream moved echoCancellation/noiseSuppression/autoGainControl from
constructor params (fed by URL params) to persisted Settings
(settings.ts) with a developer-settings UI. The cinny host still drives
these per call via URL params (noiseSuppression=false /
autoGainControl=false when the in-source ML denoiser is active, so the
model gets a raw mic) - taking upstream verbatim would silently break
the ML denoise tier. Re-wired as AND semantics in generateRoomOption():
a constraint is enabled only if BOTH the Setting and the URL param
allow it. Params default to true, so with no params this is
byte-for-byte upstream behaviour. Upstream's own echoCancellation /
noiseSuppression URL params (still parsed but dead in v0.25.0) work
again as a side effect. Lotus autoGainControl URL param kept in
UrlParams.ts (auto-merged, unchanged).
* src/state/CallViewModel/remoteMembers/ECConnectionFactory.test.ts
Took upstream (tests now drive via Settings). The lost Lotus coverage
is restored in a NEW colocated file src/lotus/lotusAudioConstraints.test.ts
(3 tests) so the upstream test file stays pristine. Verified the new
test fails against pure-upstream ConnectionFactory and passes with the
re-wiring.
* src/state/CallViewModel/CallViewModel.ts
Three small hunks: kept both the Lotus `userMedia$` interface member
and upstream's new `keyRotationSuppressed$`; dropped the three Lotus
audio constructor args (mechanism removed upstream, see above); kept
both in the returned object. The [lotus #4] overrideSpotlight$ routing,
manualSpotlightUserId$ and setManualSpotlight auto-merged; verified
against upstream's changed ringingMedia$ (now single-or-null instead
of array) - the merge correctly took upstream's outer branch and the
inner screenShares$/spotlightSpeaker$ logic that lotusSpotlight.ts
mirrors is unchanged upstream.
* src/index.css
Kept both: Lotus lotus-transparent / lotus-theme blocks and upstream's
new body[data-background="gradient"]::before full-viewport gradient.
The naive merge swallowed the closing brace of body.lotus-theme -
restored. Added a rule hiding the new gradient pseudo-element under
body.lotus-transparent, since it would otherwise paint over the
transparent body and hide the host wallpaper.
* src/components/CallFooterViewModel.tsx, src/components/CallFooter.stories.tsx
No Lotus content - pure upstream-vs-upstream conflicts caused by the
merge base being v0.20.1-rc.1. Took upstream (layoutMode ->
layoutSwitchVm; setLayoutMode removed). No Lotus code uses
setGridMode/layoutMode.
Non-conflicting but reviewed:
* src/widget.ts auto-merged cleanly. Upstream's removal of .well-known
transport advertisement and the new RTC-transport capability request
did not touch the action registration loop the LOTUS_TO_WIDGET_ACTIONS
spread and widget.lazyActions ride on - nothing to re-wire.
* src/room/InCallView.tsx, src/useAudioContext.tsx, src/useTheme.ts,
src/tile/MediaView.tsx(+.module.css), src/UrlParams.ts(+test),
all *.module.css and .gitea/workflows/ci.yml auto-merged; each diff
against v0.25.0 was checked to equal the original Lotus hunk.
* src/button/Button.module.css: the merge appended an exact duplicate
of upstream's `.rotate`/`@keyframes spin` block (rc.1 merge-base
artefact) - reset to upstream verbatim.
* src/grid/OneOnOnePortraitLayout.module.css was renamed upstream to
OneOnOneMobileLayout.module.css; git followed the rename and the Lotus
safe-area PiP inset fix applies there (the --content-inset-* vars it
uses still exist upstream).
Tooling changes inherited from upstream that affect the fork:
* eslint + prettier were replaced by oxlint + oxfmt (`pnpm lint:oxlint`,
`pnpm format:check`). oxlint flagged 10 issues, all in src/lotus/*:
8x no-meaningless-void-operator (dropped the `void` before void-typed
widget transport.reply / callbacks - no behaviour change), 1x
consistent-type-imports (lotusWidget.ts: `import type`), and 2x
unicorn/no-useless-spread in lotusAudioInject.ts which are FALSE
POSITIVES - `[...activeClips]` is a required defensive copy because
abort() deletes from the Set during iteration; suppressed with an
explanatory eslint-disable-next-line. oxfmt reformatted 7 Lotus
touched files (whitespace only).
* packageManager bumped by upstream to pnpm@11.21.0, which requires
Node >= 22.13 (uses node:sqlite). Node 20 cannot run it; pnpm 10.33
cannot read the new lockfile either (matrix-js-sdk is now a git
dependency on develop, using a version-union pnpm 10 rejects). Fork CI
already uses Node 24 (.node-version), so CI is unaffected.
* matrix-js-sdk is now github:matrix-org/matrix-js-sdk#develop (pinned
by commit in pnpm-lock.yaml).
Lotus behaviour NOT preserved: none found.
Verification (Node 24.11.1, pnpm 11.21.0): pnpm install --frozen-lockfile
OK (lockfile taken from upstream unchanged, no regeneration needed);
tsc clean; oxlint clean; oxfmt --check clean; knip exit 0 (2 config
hints in upstream knip.ts only); vitest unit 84 files / 627 passed /
9 skipped; build:embedded OK, staged to embedded/web/dist (44M), all
six io.lotus.* action strings present in the bundle.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
A null cap only removed the key from the sticky settings and applyToRoom
then skipped that sender, leaving the previously written maxBitrate /
maxFramerate live on the RTCRtpSender — contrary to the host contract
documented in cinny's CallControl. Track which keys this module wrote per
sender and write them back as undefined on clear. Unit-tested.
Fixes#11
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
startLotusCallState gated only on the URL flag, so a standalone (non-widget)
load built the whole per-member pipeline and stringified it at up to 8 Hz
for a host that doesn't exist. Add the same `if (!widget)` guard the sibling
modules use. Also move distinctUntilChanged ahead of throttleTime so an
unchanged value no longer spends a throttle window (first half of #20).
Fixes#31
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
matrix-widget-api's PostmessageTransport accepts toWidget actions from any
origin by default; the fork added actions that inject audio, deafen, retune
the encoder and render images. EC is served same-origin with the cinny
host, so globalThis.origin === parentOrigin and the strict check passes.
Fixes#15
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
apply() guarded on mic.getProcessor(), which is only set after setProcessor()
resolves (after the whole wasm/model load), so mic-published followed by
camera-published constructed two processors — two AudioContexts, two model
loads, double lock hold time. Track the in-flight processor per room, skip
apply() while one is pending, and destroy a pending processor if the module
is torn down before setProcessor resolves. Unit-tested.
Fixes#10
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
On a failed restart() the processor set processedTrack to the LiveKit-owned
input track. LiveKit's internalStopProcessor() stops processedTrack and then
republishes _mediaStreamTrack — the same object — so any later
stopProcessor()/teardown killed the live mic for the rest of the session.
Leave processedTrack undefined so LiveKit falls through to its own track.
Unit-tested.
Fixes#2
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Deafen wrote RemoteParticipant.setVolume(0, Microphone), which EC's own
createVolumeControls overwrote with the tile volume every time sink$
re-emitted (every join / re-resolution) — so anyone joining while you were
deafened was audible — and undeafen blanket-wrote 1 to every participant,
clobbering per-tile volume/mute. The ParticipantConnected listeners also
hung off livekitRoomItems$, which is empty while alone, so the first joiner
could be heard briefly.
Deafen now drives setAudioEnabled$ -> muteAllAudio$, the `muted` prop
InCallView already passes to every audio renderer, which also silences
Track.Source.Unknown soundboard audio (the known P6-2 gap). Undeafen
restores the user's own output state and never touches the mute-all
setting. Re-applying the same state is a no-op so resendForkState() is
safe. Screenshare-audio-only mute keeps setVolume(ScreenShareAudio) but
only restores participants it muted itself. Rooms come from
allConnections$ like the sibling modules. Unit-tested.
Fixes#1
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
3-agent survey of the in-call UI at phone width + 2-agent review of the
staged diff. All changes are mobile-gated CSS (mobile-first @media on
Compound tokens), so desktop rendering is provably unchanged.
- CallFooter: wrap the control row and tighten gap/padding at <=500px so
the buttons (incl. the destructive hangup) can never be clipped by the
grid's overflow-x:hidden. A loudspeaker button was added to the bar
without a shed rule, pushing 6-7 lg buttons past the viewport at
320-500px.
- OneOnOnePortraitLayout: give the 1:1 self-view the same safe-area-aware
inset as SpotlightExpandedLayout so it clears the home indicator /
floating footer (was a bare 16px inset). Phone-only layout.
- GridTile: enlarge the always-visible camera-flip control to a 44px touch
target on coarse pointers (was ~28px).
- ReactionToggleButton: enlarge reaction-picker emoji buttons to 44px at
<=420px (were ~32px); five still fit a 360px drawer row.
- Tabs: scroll the horizontal tab row on the inline axis (overflow-x) so
the Settings tabs don't clip in a phone drawer.
- SpotlightLandscapeLayout: shrink the 180px filmstrip rail to 132px on
short landscape phones (max-height:400px) so the spotlight isn't a sliver.
Gates: tsc 0, prettier clean, affected vitest pass. No TS changed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- EC1: lotusQuality — track + clearTimeout the 500ms settle re-apply per room
(was leaking a timer that fired on torn-down rooms).
- EC2/EC3: lotusQuality + lotusAudioInject drive off vm.allConnections$ instead
of the remote-gated livekitRoomItems$ (were no-ops when alone), matching
lotusDenoise.
- EC4: lotusDecorations resets its roster to {} on teardown so a decoration from
a previous call can't render on a shared user in the next one.
- EC5: hoisted a stable useSyncExternalStore subscribe fn (was re-subscribing
every tile render).
- EC6: lotusFocus only sets the spotlight when the userId field is present
(a partial payload no longer clears the pin).
tsc clean. Needs a republish to ship.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New toWidget action { deafened, screenshareAudioMuted } that sets each remote
RemoteParticipant.setVolume per source (Microphone + ScreenShareAudio), applied
to existing participants + re-applied to late joiners via
RoomEvent.ParticipantConnected (subscribed through vm.livekitRoomItems$). Closure-
scoped state, matching the sibling lotus modules; the cinny host re-sends on join
so a fresh call never inherits stale deafen state.
Replaces cinny's brittle iframe-DOM <audio>.muted hack (which broke on EC
re-render / late tracks). Folded into unpublished 0.20.1-lotus.2.
Note: injected/soundboard audio (Track.Source.Unknown) is not silenced — the
livekit-client setVolume type only accepts Microphone|ScreenShareAudio.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The in-source ML denoiser is gated on lotusDenoiseSource (see lotusDenoise.ts
gate), not the legacy build-time shim flag lotusDenoise=ml. Correct the two
stale references (lotusDenoise.ts JSDoc + InCallView.tsx comment) to
lotusDenoiseSource=1.
Bump the embedded package to 0.20.1-lotus.2 for the next publish and update the
CI comment: the checked-in version now tracks the intended release, while a
pushed tag still wins (npm version "$TAG" overwrites at publish time).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
playInjectedClip only registered its cleanup (and thus became abortable by a
later clip's replace-mode loop) AFTER publishing. Two inject actions fired in
quick succession could both pass their fetch/decode/publish awaits before
either was registered, so both tracks got published.
Register a synchronous placeholder abort BEFORE the first await: it aborts the
in-flight fetch and flips an `aborted` flag checked after every await, so a
newer clip cancels the older one during the vulnerable window. The real
cleanup replaces the placeholder once the track is live, and if we were
superseded mid-publish we tear the just-published track down immediately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the rebase hazard from CallViewModel: upstream's spotlightSpeaker$
auto-selection had been renamed to autoSpotlightSpeaker$ and an inline
manual-override + screenshare-coexistence block was spliced into
spotlightAndPip$. Both diverge from upstream and would conflict on every
rebase.
Restore spotlightSpeaker$ to its byte-for-byte upstream form and move the
[lotus #4] override into a pure wrapper, overrideSpotlight$(), invoked at a
single call point in spotlightAndPip$. Behaviour is unchanged: identical to
upstream while manualSpotlightUserId$ is null (the default), and preserves the
"pin a participant" and "focus camera during screenshare" (#4 / A5) rules when
the host sends io.lotus.focus_participant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
playInjectedClip now stops any in-flight clip (via the existing idempotent
cleanup) before starting a new one, so rapid taps replace rather than
overlap/stack — no track leak. The host also debounces the button.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AEC/AGC audit fix + two hardening items from the engine review.
- Add an `autoGainControl` capture param (UrlParams -> CallViewModel ->
ConnectionFactory audioCaptureDefaults), mirroring echoCancellation/
noiseSuppression. Defaults true (unchanged); the host sets it false only for
the ML tier so the browser's auto gain control doesn't fight the in-source ML
denoiser (pumping). Echo cancellation stays on. Tests cover the URL parse and
the audioCaptureDefaults wiring.
- L1: init() now closes the owned AudioContext on a build failure (was orphaned;
browsers cap live contexts, so repeated failures could exhaust them).
- L2: buildGraph() disposes its partially-built nodes on failure (disposeGraph
previously only cleaned the prior graph).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track-B audio-quality changes to reduce the "robotic/underwater" artifact.
- Dry/wet attenuation floor (default 0.15 ≈ -16 dB) blends a little of the raw
mic under the denoised signal so suppression can't fully collapse the noise
floor between words (the main cause of the RNNoise "underwater"/pumping
sound). Applied ONLY to the low-latency flat models (RNNoise/Speex); DTLN/DFN
add algorithmic latency that would comb-filter an undelayed dry mix, so they
rely on their own level instead. Tunable via `lotusDenoiseFloor`.
- Noise gate now runs AFTER the ML model, not before — gating the raw signal
fed hard-zeroed frames into the model and tuned the threshold on pre-denoise
levels.
- DeepFilterNet 3 noiseReductionLevel 80 -> 60: full strength was the main
"over-processed" contributor; 60 keeps voice natural.
Defaults are conservative and tunable; final values are meant to be dialed in
with real-call A/B listening.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track-A robustness fixes from the engine review; no quality/model changes.
- H1: auto-resume the AudioContext on `statechange` if it suspends mid-call
(mobile backgrounding / audio interruption). Previously the dest node emitted
digital silence with no recovery — a silent mute of the sender.
- H2: `resumeCtx()` races `resume()` against a timeout. A suspended context can
only resume on a user gesture; the action can arrive via postMessage, so a
bare `await resume()` inside LiveKit's track-change lock could hang and
deadlock all later mute/unmute/device-switch. Now it proceeds and the H1
watcher heals it.
- M1: don't cache a REJECTED wasm fetch — a transient blip during a reconnect
used to permanently disable denoise for the session. Evict on failure.
- M2: activate denoise off `allConnections$` (local participant's connections)
instead of `livekitRoomItems$`, which excludes the local participant and only
surfaces rooms with a remote member — so denoise now also runs when you're
alone and no longer couples to a remote-render concern.
- Context lifecycle: `closeContext()` removes the state watcher before closing;
`ensureContext()` closes a half-initialised context on any failure (no leak).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
audioOutputId comes from `useMediaDevices().audioOutput.selected$?.id`, which is
undefined until a device is selected. On the Tauri desktop webview the observable
emits undefined first, so setSinkId(undefined) threw "The provided value is not
of type 'AudioSinkOptions'" on every call join (repeatedly). Guard on a string
(the default device is the empty string, still valid).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host already sets lotusDenoise=ml and injects its getUserMedia shim;
reusing that flag would double-process audio the moment this fork ships.
Gate the in-source engine on lotusDenoiseSource=1 instead, so the fork is
inert on deploy and the host cuts over explicitly (set the flag + drop the
shim) when ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review found native dynamic import() of the DTLN/DeepFilterNet ESM
resolves "./denoise/…" against the bundled JS chunk's URL (-> /assets/…)
not the document, so those two models 404'd and silently fell back to raw
mic in the default config. Resolve the asset base to an absolute
same-origin href against the document; addModule()/fetch() accept absolute
too, so all three load paths stay consistent. (rnnoise/speex were
unaffected since addModule resolves against the document.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Faithful port of cinny's proven pipeline into the TrackProcessor, closing
protocol gap F3 (host offers rnnoise/speex/dtln/deepfilternet; only the
first two existed in-source).
- Fix real bug: gate worklet registers as "noise-gate" (hyphenated), not
"noiseGate" — the gated path would have failed to construct the node.
- Per-model sample rate: DTLN runs at 16kHz, others 48kHz (worklets don't
resample); verify the context actually got the rate, else fall back.
- resume() a suspended context (host postMessage isn't a gesture).
- DTLN via dynamic-imported @workadventure helper (bypassUntilReady);
DeepFilterNet via dynamic-imported ESM + DeepFilterNet3Core pointed at
the self-hosted base. Same-origin base (kept from the C1 fix) makes these
dynamic imports safe.
- Prefer SIMD rnnoise.wasm with non-SIMD fallback; cache wasm per URL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Denoise deep-review (CRITICAL): restart() read opts.audioContext, which
LiveKit does NOT pass on restart — so reconnect (the A7 scenario) and mic
device-switch threw after stopping the old track, leaving the mic SILENT
(A7 reintroduced). Fix:
- Processor owns a dedicated 48kHz AudioContext (sapphi worklets require
48kHz; H1), reused across restart, closed on destroy.
- restart() never throws and never leaves a stopped track on the sender:
builds the new graph first, then disposes the old; on failure degrades
to RAW mic audio rather than silence.
- Cache wasm per URL (no re-fetch each reconnect); gate threshold default
-45 and accept an explicit 0 (M2); document the cross-repo asset contract.
Protocol audit:
- Non-silent warning when an unsupported denoise model (dtln/deepfilternet)
is requested instead of silent rnnoise fallback (F3).
- Correct the call_state enum comment (immediate error-reply, not 10s) (F2).
Build/CI audit:
- Stamp VITE_APP_VERSION in CI; document the vX.Y.Z-lotus.N version scheme.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Holistic security audit findings:
- C1 (CRITICAL): force lotusDenoiseBase to same-origin before it reaches
audioWorklet.addModule()/fetch — a crafted call-link param could
otherwise load attacker JS/WASM as a worklet processing the live mic.
Non-same-origin/malformed values fall back to bundled ./denoise/.
- H1 (HIGH): gate audio-inject behind explicit lotusAudioInject=1 (still
acks the action so no transport hang) — it publishes under the local
user's identity, so it must not be silently armed for every call.
- M1 (MED): cap the decoration roster at 512 entries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements RNNoise/Speex noise suppression as a LiveKit audio
TrackProcessor attached to the local mic track, replacing the host's
build-time getUserMedia monkeypatch. Because EC re-attaches the processor
on every (re)publish (LocalTrackPublished), denoise now survives EC's
mid-call reconnect — the root cause of A7 "mic dead after reconnect".
Reuses the worklet/wasm assets already shipped under ./denoise/ (no new EC
dependency); model/gate configured via lotusDenoise/lotusModel/lotusGate
URL params. Additive: no-op unless lotusDenoise=ml.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review found in-call tiles use MediaView->Avatar, not TileAvatar, so the
decoration never rendered in-call (CRITICAL). Move the overlay into
MediaView, gated on the avatar's own visibility (!(video && videoEnabled))
so it never floats over live video; revert the TileAvatar changes.
Also ref-count the io.lotus.decorations registration (one shared handler,
no double-reply) and stop clearing the map on teardown so a transient
remount doesn't drop decorations (HIGH/MED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.decorations (toWidget): the host pushes a userId->image-URL
map and EC overlays the profile decoration on each tile avatar
(TileAvatar), keyed by userId, with a useSyncExternalStore-backed store.
Makes A6 first-class in-call instead of absent. URLs are validated
https/blob. Additive: no-op unless the host sends decorations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Apply the encoding patch to ALL simulcast layers, not just encodings[0]:
screenshare publishes simulcast (VP8), so the full-res layer (the real
bandwidth hog) was left uncapped (CRITICAL).
- Re-apply on TrackUnmuted/restart + a 500ms settle, since LiveKit's
refreshSenderEncodings() overwrites our caps on replaceTrack (device/
source switch, processor toggle) without firing LocalTrackPublished (HIGH).
- Clamp values to sane ranges so a typo can't brick the encoder (MED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds body.lotus-transparent (lotusTransparent=1) so the host wallpaper
shows through the call natively, retiring cinny's injected
`html,body{background:none!important}` hack; and body.lotus-theme
(lotusTheme=1) as a source-level Compound-token override hook for the
Lotus/TDS palette (driven by the existing setTheme channel / URL flags).
Additive: no-op without the flags.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- resume() the AudioContext (host postMessage isn't a gesture) so the clip
isn't silent; warn if it stays suspended (HIGH).
- Close the AudioContext on decode failure (no context leak) (MED).
- Abort in-flight clips on teardown (unmount/vm-change/leave) so audio
doesn't keep blasting to peers (MED).
- Stop the cloned MediaStreamTrack when a room publish fails (MED).
- Validate url is https/blob and fetch with credentials:omit, mode:cors
(MED security).
- Guard against NaN clip duration; fix stale enum doc comment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.set_quality (toWidget): caps mic audio bitrate and
screenshare bitrate/framerate via RTCRtpSender.setParameters (no
republish). Settings are sticky and re-applied on LocalTrackPublished so
they survive mute/unmute and reconnects. These encoding controls lived in
EC's module scope, unreachable from the host against the prebuilt bundle.
Additive: no-op unless the host sends the action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review found the manual pin only chose among cameras when no screenshare
was active; during a screenshare the screenshare won the spotlight and the
pinned camera was demoted to an ignored PiP. Apply the override at the
spotlightAndPip$ level: when a pin is explicitly set, surface that camera
in the spotlight alongside the shared screen. No manual pin = unchanged.
Note: a pin persists if the pinned user briefly leaves and rejoins; the
host clears it via focus_participant{userId:null} (by design).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.inject_audio (toWidget): mixes a soundboard clip into the
call so other participants hear it. Publishes the clip as a separate
Unknown-source LiveKit track (rendered by MatrixAudioRenderer) rather than
splicing into the mic track, so the denoise pipeline is untouched; the
track is unpublished when the clip ends (with a 30s safety cap). This is
the real call-audio injection that was impossible against the prebuilt EC
bundle. Additive: no-op unless the host sends the action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Document that io.lotus.call_state is request/response and the host must
ack it (cinny listenAction replies {}) to avoid 10s-timeout churn (H1).
- Throttle 150ms -> 250ms to reduce widget traffic (M1).
- lotusParam: hash fragment wins over query, matching EC's ParamParser (L1).
- Fix the misleading "opaque" id comment; id is userId:deviceId (L2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.focus_participant (toWidget): the host can pin a participant
to the spotlight by Matrix user id (or clear with userId:null), via a
manual override injected into CallViewModel.spotlightSpeaker$. Replaces
cinny's fragile DOM .click() tile-selector focus hack. Extracts the action
enum into lotusActions.ts (no circular import) and allow-lists Lotus
toWidget actions in initializeWidget. Additive: no-op unless the host
sends the action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Opt-in (lotusCallState=1) bridge that emits io.lotus.call_state with each
participant's speaking/audio/video state, so the Lotus host can drive
speaking rings / mute badges / PiP from real events instead of scraping
EC's rendered DOM. Exposes vm.userMedia$ on the public CallViewModel.
Additive: no-op without the flag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
upload-artifact@v4 needs GitHub's artifact backend, which the Gitea
act_runner doesn't implement — it failed the build job with exit 1 even
though build:embedded + smoke-check passed. The publish job rebuilds from
source, so the artifact was only a convenience.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build:embedded outputs to repo-root dist/; the publish job and smoke-check
expected embedded/web/dist (the publish template's files entry), which was
never created in CI — so a tagged publish would fail its smoke-check or ship
an empty tarball. Copy dist -> embedded/web/dist in both jobs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- embedded/web/package.json: publish as @lotusguild/element-call-embedded
from our Gitea fork (repository URL updated).
- .gitea/workflows/ci.yml: build the embedded bundle on PR/push to lotus;
publish to the Gitea npm registry on a v* tag. Linux-only (web bundle).
Based on upstream element-call v0.20.1 (2d74c48).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Lotus CIandClaude Opus 5