3-agent survey of the in-call UI at phone width + 2-agent review of the
staged diff. All changes are mobile-gated CSS (mobile-first @media on
Compound tokens), so desktop rendering is provably unchanged.
- CallFooter: wrap the control row and tighten gap/padding at <=500px so
the buttons (incl. the destructive hangup) can never be clipped by the
grid's overflow-x:hidden. A loudspeaker button was added to the bar
without a shed rule, pushing 6-7 lg buttons past the viewport at
320-500px.
- OneOnOnePortraitLayout: give the 1:1 self-view the same safe-area-aware
inset as SpotlightExpandedLayout so it clears the home indicator /
floating footer (was a bare 16px inset). Phone-only layout.
- GridTile: enlarge the always-visible camera-flip control to a 44px touch
target on coarse pointers (was ~28px).
- ReactionToggleButton: enlarge reaction-picker emoji buttons to 44px at
<=420px (were ~32px); five still fit a 360px drawer row.
- Tabs: scroll the horizontal tab row on the inline axis (overflow-x) so
the Settings tabs don't clip in a phone drawer.
- SpotlightLandscapeLayout: shrink the 180px filmstrip rail to 132px on
short landscape phones (max-height:400px) so the spotlight isn't a sliver.
Gates: tsc 0, prettier clean, affected vitest pass. No TS changed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- EC1: lotusQuality — track + clearTimeout the 500ms settle re-apply per room
(was leaking a timer that fired on torn-down rooms).
- EC2/EC3: lotusQuality + lotusAudioInject drive off vm.allConnections$ instead
of the remote-gated livekitRoomItems$ (were no-ops when alone), matching
lotusDenoise.
- EC4: lotusDecorations resets its roster to {} on teardown so a decoration from
a previous call can't render on a shared user in the next one.
- EC5: hoisted a stable useSyncExternalStore subscribe fn (was re-subscribing
every tile render).
- EC6: lotusFocus only sets the spotlight when the userId field is present
(a partial payload no longer clears the pin).
tsc clean. Needs a republish to ship.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New toWidget action { deafened, screenshareAudioMuted } that sets each remote
RemoteParticipant.setVolume per source (Microphone + ScreenShareAudio), applied
to existing participants + re-applied to late joiners via
RoomEvent.ParticipantConnected (subscribed through vm.livekitRoomItems$). Closure-
scoped state, matching the sibling lotus modules; the cinny host re-sends on join
so a fresh call never inherits stale deafen state.
Replaces cinny's brittle iframe-DOM <audio>.muted hack (which broke on EC
re-render / late tracks). Folded into unpublished 0.20.1-lotus.2.
Note: injected/soundboard audio (Track.Source.Unknown) is not silenced — the
livekit-client setVolume type only accepts Microphone|ScreenShareAudio.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The in-source ML denoiser is gated on lotusDenoiseSource (see lotusDenoise.ts
gate), not the legacy build-time shim flag lotusDenoise=ml. Correct the two
stale references (lotusDenoise.ts JSDoc + InCallView.tsx comment) to
lotusDenoiseSource=1.
Bump the embedded package to 0.20.1-lotus.2 for the next publish and update the
CI comment: the checked-in version now tracks the intended release, while a
pushed tag still wins (npm version "$TAG" overwrites at publish time).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
playInjectedClip only registered its cleanup (and thus became abortable by a
later clip's replace-mode loop) AFTER publishing. Two inject actions fired in
quick succession could both pass their fetch/decode/publish awaits before
either was registered, so both tracks got published.
Register a synchronous placeholder abort BEFORE the first await: it aborts the
in-flight fetch and flips an `aborted` flag checked after every await, so a
newer clip cancels the older one during the vulnerable window. The real
cleanup replaces the placeholder once the track is live, and if we were
superseded mid-publish we tear the just-published track down immediately.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the rebase hazard from CallViewModel: upstream's spotlightSpeaker$
auto-selection had been renamed to autoSpotlightSpeaker$ and an inline
manual-override + screenshare-coexistence block was spliced into
spotlightAndPip$. Both diverge from upstream and would conflict on every
rebase.
Restore spotlightSpeaker$ to its byte-for-byte upstream form and move the
[lotus #4] override into a pure wrapper, overrideSpotlight$(), invoked at a
single call point in spotlightAndPip$. Behaviour is unchanged: identical to
upstream while manualSpotlightUserId$ is null (the default), and preserves the
"pin a participant" and "focus camera during screenshare" (#4 / A5) rules when
the host sends io.lotus.focus_participant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
playInjectedClip now stops any in-flight clip (via the existing idempotent
cleanup) before starting a new one, so rapid taps replace rather than
overlap/stack — no track leak. The host also debounces the button.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AEC/AGC audit fix + two hardening items from the engine review.
- Add an `autoGainControl` capture param (UrlParams -> CallViewModel ->
ConnectionFactory audioCaptureDefaults), mirroring echoCancellation/
noiseSuppression. Defaults true (unchanged); the host sets it false only for
the ML tier so the browser's auto gain control doesn't fight the in-source ML
denoiser (pumping). Echo cancellation stays on. Tests cover the URL parse and
the audioCaptureDefaults wiring.
- L1: init() now closes the owned AudioContext on a build failure (was orphaned;
browsers cap live contexts, so repeated failures could exhaust them).
- L2: buildGraph() disposes its partially-built nodes on failure (disposeGraph
previously only cleaned the prior graph).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track-B audio-quality changes to reduce the "robotic/underwater" artifact.
- Dry/wet attenuation floor (default 0.15 ≈ -16 dB) blends a little of the raw
mic under the denoised signal so suppression can't fully collapse the noise
floor between words (the main cause of the RNNoise "underwater"/pumping
sound). Applied ONLY to the low-latency flat models (RNNoise/Speex); DTLN/DFN
add algorithmic latency that would comb-filter an undelayed dry mix, so they
rely on their own level instead. Tunable via `lotusDenoiseFloor`.
- Noise gate now runs AFTER the ML model, not before — gating the raw signal
fed hard-zeroed frames into the model and tuned the threshold on pre-denoise
levels.
- DeepFilterNet 3 noiseReductionLevel 80 -> 60: full strength was the main
"over-processed" contributor; 60 keeps voice natural.
Defaults are conservative and tunable; final values are meant to be dialed in
with real-call A/B listening.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Track-A robustness fixes from the engine review; no quality/model changes.
- H1: auto-resume the AudioContext on `statechange` if it suspends mid-call
(mobile backgrounding / audio interruption). Previously the dest node emitted
digital silence with no recovery — a silent mute of the sender.
- H2: `resumeCtx()` races `resume()` against a timeout. A suspended context can
only resume on a user gesture; the action can arrive via postMessage, so a
bare `await resume()` inside LiveKit's track-change lock could hang and
deadlock all later mute/unmute/device-switch. Now it proceeds and the H1
watcher heals it.
- M1: don't cache a REJECTED wasm fetch — a transient blip during a reconnect
used to permanently disable denoise for the session. Evict on failure.
- M2: activate denoise off `allConnections$` (local participant's connections)
instead of `livekitRoomItems$`, which excludes the local participant and only
surfaces rooms with a remote member — so denoise now also runs when you're
alone and no longer couples to a remote-render concern.
- Context lifecycle: `closeContext()` removes the state watcher before closing;
`ensureContext()` closes a half-initialised context on any failure (no leak).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
audioOutputId comes from `useMediaDevices().audioOutput.selected$?.id`, which is
undefined until a device is selected. On the Tauri desktop webview the observable
emits undefined first, so setSinkId(undefined) threw "The provided value is not
of type 'AudioSinkOptions'" on every call join (repeatedly). Guard on a string
(the default device is the empty string, still valid).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host already sets lotusDenoise=ml and injects its getUserMedia shim;
reusing that flag would double-process audio the moment this fork ships.
Gate the in-source engine on lotusDenoiseSource=1 instead, so the fork is
inert on deploy and the host cuts over explicitly (set the flag + drop the
shim) when ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review found native dynamic import() of the DTLN/DeepFilterNet ESM
resolves "./denoise/…" against the bundled JS chunk's URL (-> /assets/…)
not the document, so those two models 404'd and silently fell back to raw
mic in the default config. Resolve the asset base to an absolute
same-origin href against the document; addModule()/fetch() accept absolute
too, so all three load paths stay consistent. (rnnoise/speex were
unaffected since addModule resolves against the document.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Faithful port of cinny's proven pipeline into the TrackProcessor, closing
protocol gap F3 (host offers rnnoise/speex/dtln/deepfilternet; only the
first two existed in-source).
- Fix real bug: gate worklet registers as "noise-gate" (hyphenated), not
"noiseGate" — the gated path would have failed to construct the node.
- Per-model sample rate: DTLN runs at 16kHz, others 48kHz (worklets don't
resample); verify the context actually got the rate, else fall back.
- resume() a suspended context (host postMessage isn't a gesture).
- DTLN via dynamic-imported @workadventure helper (bypassUntilReady);
DeepFilterNet via dynamic-imported ESM + DeepFilterNet3Core pointed at
the self-hosted base. Same-origin base (kept from the C1 fix) makes these
dynamic imports safe.
- Prefer SIMD rnnoise.wasm with non-SIMD fallback; cache wasm per URL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Denoise deep-review (CRITICAL): restart() read opts.audioContext, which
LiveKit does NOT pass on restart — so reconnect (the A7 scenario) and mic
device-switch threw after stopping the old track, leaving the mic SILENT
(A7 reintroduced). Fix:
- Processor owns a dedicated 48kHz AudioContext (sapphi worklets require
48kHz; H1), reused across restart, closed on destroy.
- restart() never throws and never leaves a stopped track on the sender:
builds the new graph first, then disposes the old; on failure degrades
to RAW mic audio rather than silence.
- Cache wasm per URL (no re-fetch each reconnect); gate threshold default
-45 and accept an explicit 0 (M2); document the cross-repo asset contract.
Protocol audit:
- Non-silent warning when an unsupported denoise model (dtln/deepfilternet)
is requested instead of silent rnnoise fallback (F3).
- Correct the call_state enum comment (immediate error-reply, not 10s) (F2).
Build/CI audit:
- Stamp VITE_APP_VERSION in CI; document the vX.Y.Z-lotus.N version scheme.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Holistic security audit findings:
- C1 (CRITICAL): force lotusDenoiseBase to same-origin before it reaches
audioWorklet.addModule()/fetch — a crafted call-link param could
otherwise load attacker JS/WASM as a worklet processing the live mic.
Non-same-origin/malformed values fall back to bundled ./denoise/.
- H1 (HIGH): gate audio-inject behind explicit lotusAudioInject=1 (still
acks the action so no transport hang) — it publishes under the local
user's identity, so it must not be silently armed for every call.
- M1 (MED): cap the decoration roster at 512 entries.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements RNNoise/Speex noise suppression as a LiveKit audio
TrackProcessor attached to the local mic track, replacing the host's
build-time getUserMedia monkeypatch. Because EC re-attaches the processor
on every (re)publish (LocalTrackPublished), denoise now survives EC's
mid-call reconnect — the root cause of A7 "mic dead after reconnect".
Reuses the worklet/wasm assets already shipped under ./denoise/ (no new EC
dependency); model/gate configured via lotusDenoise/lotusModel/lotusGate
URL params. Additive: no-op unless lotusDenoise=ml.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review found in-call tiles use MediaView->Avatar, not TileAvatar, so the
decoration never rendered in-call (CRITICAL). Move the overlay into
MediaView, gated on the avatar's own visibility (!(video && videoEnabled))
so it never floats over live video; revert the TileAvatar changes.
Also ref-count the io.lotus.decorations registration (one shared handler,
no double-reply) and stop clearing the map on teardown so a transient
remount doesn't drop decorations (HIGH/MED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.decorations (toWidget): the host pushes a userId->image-URL
map and EC overlays the profile decoration on each tile avatar
(TileAvatar), keyed by userId, with a useSyncExternalStore-backed store.
Makes A6 first-class in-call instead of absent. URLs are validated
https/blob. Additive: no-op unless the host sends decorations.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Apply the encoding patch to ALL simulcast layers, not just encodings[0]:
screenshare publishes simulcast (VP8), so the full-res layer (the real
bandwidth hog) was left uncapped (CRITICAL).
- Re-apply on TrackUnmuted/restart + a 500ms settle, since LiveKit's
refreshSenderEncodings() overwrites our caps on replaceTrack (device/
source switch, processor toggle) without firing LocalTrackPublished (HIGH).
- Clamp values to sane ranges so a typo can't brick the encoder (MED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds body.lotus-transparent (lotusTransparent=1) so the host wallpaper
shows through the call natively, retiring cinny's injected
`html,body{background:none!important}` hack; and body.lotus-theme
(lotusTheme=1) as a source-level Compound-token override hook for the
Lotus/TDS palette (driven by the existing setTheme channel / URL flags).
Additive: no-op without the flags.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- resume() the AudioContext (host postMessage isn't a gesture) so the clip
isn't silent; warn if it stays suspended (HIGH).
- Close the AudioContext on decode failure (no context leak) (MED).
- Abort in-flight clips on teardown (unmount/vm-change/leave) so audio
doesn't keep blasting to peers (MED).
- Stop the cloned MediaStreamTrack when a room publish fails (MED).
- Validate url is https/blob and fetch with credentials:omit, mode:cors
(MED security).
- Guard against NaN clip duration; fix stale enum doc comment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.set_quality (toWidget): caps mic audio bitrate and
screenshare bitrate/framerate via RTCRtpSender.setParameters (no
republish). Settings are sticky and re-applied on LocalTrackPublished so
they survive mute/unmute and reconnects. These encoding controls lived in
EC's module scope, unreachable from the host against the prebuilt bundle.
Additive: no-op unless the host sends the action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review found the manual pin only chose among cameras when no screenshare
was active; during a screenshare the screenshare won the spotlight and the
pinned camera was demoted to an ignored PiP. Apply the override at the
spotlightAndPip$ level: when a pin is explicitly set, surface that camera
in the spotlight alongside the shared screen. No manual pin = unchanged.
Note: a pin persists if the pinned user briefly leaves and rejoins; the
host clears it via focus_participant{userId:null} (by design).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.inject_audio (toWidget): mixes a soundboard clip into the
call so other participants hear it. Publishes the clip as a separate
Unknown-source LiveKit track (rendered by MatrixAudioRenderer) rather than
splicing into the mic track, so the denoise pipeline is untouched; the
track is unpublished when the clip ends (with a 30s safety cap). This is
the real call-audio injection that was impossible against the prebuilt EC
bundle. Additive: no-op unless the host sends the action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Document that io.lotus.call_state is request/response and the host must
ack it (cinny listenAction replies {}) to avoid 10s-timeout churn (H1).
- Throttle 150ms -> 250ms to reduce widget traffic (M1).
- lotusParam: hash fragment wins over query, matching EC's ParamParser (L1).
- Fix the misleading "opaque" id comment; id is userId:deviceId (L2).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds io.lotus.focus_participant (toWidget): the host can pin a participant
to the spotlight by Matrix user id (or clear with userId:null), via a
manual override injected into CallViewModel.spotlightSpeaker$. Replaces
cinny's fragile DOM .click() tile-selector focus hack. Extracts the action
enum into lotusActions.ts (no circular import) and allow-lists Lotus
toWidget actions in initializeWidget. Additive: no-op unless the host
sends the action.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Opt-in (lotusCallState=1) bridge that emits io.lotus.call_state with each
participant's speaking/audio/video state, so the Lotus host can drive
speaking rings / mute badges / PiP from real events instead of scraping
EC's rendered DOM. Exposes vm.userMedia$ on the public CallViewModel.
Additive: no-op without the flag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
upload-artifact@v4 needs GitHub's artifact backend, which the Gitea
act_runner doesn't implement — it failed the build job with exit 1 even
though build:embedded + smoke-check passed. The publish job rebuilds from
source, so the artifact was only a convenience.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build:embedded outputs to repo-root dist/; the publish job and smoke-check
expected embedded/web/dist (the publish template's files entry), which was
never created in CI — so a tagged publish would fail its smoke-check or ship
an empty tarball. Copy dist -> embedded/web/dist in both jobs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- embedded/web/package.json: publish as @lotusguild/element-call-embedded
from our Gitea fork (repository URL updated).
- .gitea/workflows/ci.yml: build the embedded bundle on PR/push to lotus;
publish to the Gitea npm registry on a v* tag. Linux-only (web bundle).
Based on upstream element-call v0.20.1 (2d74c48).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
