feat(lotus): detailed call summary for the host's Details card (cinny #253)

io.lotus.call_summary gains an optional `details`: the other Matrix users
seen and the peak in the call; how long your connection, each other
person's, and everyone at once (likely the server) was poor/lost; RTT
median/p95, outbound and inbound packet loss and jitter (sampled every 5 s
from LiveKit sender/receiver stats); and your talk, mute and screenshare
time. People already present when you connect are recorded on Connected
(LiveKit sends no ParticipantConnected for them), and joins are idempotent
per device. Older hosts ignore the field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5

LOTUS.md

@@ -37,13 +37,13 @@ checked against cinny's `lotus` branch.
 | ---------------------------------- | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- |
 | `io.lotus.call_state`              | call → host | in-call per-participant speaking / mute state. NOTE: matrix-widget-api `transport.send` is request/response — the host MUST register a handler that replies/acks each one (cinny's `listenAction` does, replying `{}`). If the host has no handler, ClientWidgetApi immediately error-replies "unsupported from-widget action", so the data is silently dropped and each throttled send rejects (caught) — functional miss + log churn, not a hang. | yes                        |
 | `io.lotus.focus_participant`       | host → call | pin/spotlight (or clear, with userId=null) a participant.                                                                                                                                                                                                                                                                                                                                                                                           | yes                        |
-| `io.lotus.inject_audio`            | host → call | play an audio clip into the call as a separate published track.                                                                                                                                                                                                                                                                                                                                                                                     | yes                        |
+| `io.lotus.inject_audio`            | host → call | play an audio clip into the call: as a separate published track, or mixed into the microphone where the token only allows listed sources (rooms with a call policy, #47). Replies `{ played: false, reason: "muted" \| "not_permitted" }` when it can't.                                                                                                                                                                                            | yes                        |
 | `io.lotus.set_quality`             | host → call | set audio/screenshare encoding quality (bitrate/framerate).                                                                                                                                                                                                                                                                                                                                                                                         | yes                        |
 | `io.lotus.decorations`             | host → call | per-user avatar-decoration image URLs for in-call tiles.                                                                                                                                                                                                                                                                                                                                                                                            | yes                        |
 | `io.lotus.set_deafen`              | host → call | deafen remote audio (and optionally mute screenshare audio).                                                                                                                                                                                                                                                                                                                                                                                        | yes                        |
 | `io.lotus.request_state`           | call → host | sent on (re)registration of a lotus toWidget handler (e.g. decorations) so the host can re-push state that would otherwise only be sent on change (decorations roster, the current focus pin) after an EC-side reconnect/remount. cinny's `resendForkState()` should respond to this the same way it responds to a fresh join.                                                                                                                      | yes                        |
 | `io.lotus.denoise_state`           | call → host | real state of the in-source denoise engine — `{ active: boolean, model: string, error?: string }` — sent once the processor attaches (or after the rnnoise fallback also fails), so the host toggle can reflect reality rather than the requested state.                                                                                                                                                                                            | yes                        |
-| `io.lotus.call_summary`            | call → host | one-shot end-of-call readout for the local participant — `{ durationMs, reconnects, poorMs, verdict }` (#143) — sent on SFU disconnect or in-call teardown, whichever comes first.                                                                                                                                                                                                                                                                  | yes                        |
+| `io.lotus.call_summary`            | call → host | one-shot end-of-call readout `{ durationMs, reconnects, poorMs, verdict, details? }` (#143); `details` (cinny #253): participants, peak, per-person poor time, everyone-poor time, RTT/loss/jitter, your talk/mute/screenshare time.                                                                                                                                                                                                                | yes                        |
 | `io.lotus.set_audio_output`        | host → call | select the audio output device `{ deviceId }` (#119).                                                                                                                                                                                                                                                                                                                                                                                               | yes                        |
 | `io.lotus.audio_output_state`      | call → host | the currently selected output `{ deviceId }` (#119).                                                                                                                                                                                                                                                                                                                                                                                                | cinny #417                 |
 | `io.lotus.screenshare_audio_state` | call → host | `{ present }`: whether any remote screenshare audio is being received, sent on change (cinny #277).                                                                                                                                                                                                                                                                                                                                                 | cinny #277                 |

embedded/web/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lotusguild/element-call-embedded",
-  "version": "0.25.0-lotus.26",
+  "version": "0.25.0-lotus.28",
   "files": [
     "README.md",
     "LICENSE-AGPL-3.0",

src/lotus/lotusAudioInject.test.ts

@@ -13,6 +13,7 @@ import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
 import {
   canPublishSoundboard,
   MAX_INJECT_BYTES,
+  soundboardMode,
   parseInjectSource,
   startLotusAudioInject,
 } from "./lotusAudioInject";
@@ -94,6 +95,10 @@ class FakeAudioContext {
     connect: vi.fn((n: unknown) => n),
     disconnect: vi.fn(),
   }));
+  public createMediaStreamSource = vi.fn(() => ({
+    connect: vi.fn((n: unknown) => n),
+    disconnect: vi.fn(),
+  }));
   public createBufferSource = vi.fn(() => ({
     buffer: undefined as AudioBuffer | undefined,
     connect: vi.fn((n: unknown) => n),
@@ -116,6 +121,12 @@ beforeEach(() => {
   // leave a real timer pending.
   vi.useFakeTimers();
   vi.stubGlobal("AudioContext", FakeAudioContext);
+  vi.stubGlobal(
+    "MediaStream",
+    class {
+      public constructor(public tracks: unknown[]) {}
+    },
+  );
   vi.stubGlobal(
     "fetch",
     vi.fn().mockResolvedValue({
@@ -273,3 +284,127 @@ test("canPublishSoundboard: refused by a microphone-only grant (EC #47)", () =>
   ).toBe(true);
   expect(canPublishSoundboard({ canPublish: false })).toBe(false);
 });
+
+test("soundboardMode: own track when unrestricted, the mic under a call policy (EC #47)", () => {
+  expect(soundboardMode(undefined)).toBe("track");
+  expect(soundboardMode({ canPublish: true, canPublishSources: [] })).toBe(
+    "track",
+  );
+  expect(soundboardMode({ canPublish: true, canPublishSources: [2, 0] })).toBe(
+    "track",
+  );
+  expect(soundboardMode({ canPublish: true, canPublishSources: [2] })).toBe(
+    "mic",
+  );
+  expect(
+    soundboardMode({ canPublish: true, canPublishSources: [2, 3, 4] }),
+  ).toBe("mic");
+  expect(
+    soundboardMode({ canPublish: true, canPublishSources: [3] }),
+  ).toBeNull();
+  expect(soundboardMode({ canPublish: false })).toBeNull();
+});
+
+/** A room whose token only allows the microphone, with a published mic track. */
+function makePolicyRoom(sources: number[]): {
+  room: unknown;
+  sender: { track: MediaStreamTrack; replaceTrack: ReturnType<typeof vi.fn> };
+  original: MediaStreamTrack;
+  micTrack: EventEmitter;
+  publishTrack: ReturnType<typeof vi.fn>;
+} {
+  const original = makeTrack();
+  const sender = {
+    track: original,
+    replaceTrack: vi.fn(async (t: MediaStreamTrack) => {
+      await Promise.resolve();
+      sender.track = t;
+    }),
+  };
+  const micTrack = Object.assign(new EventEmitter(), { sender });
+  const publishTrack = vi.fn();
+  const room = {
+    localParticipant: {
+      isMicrophoneEnabled: true,
+      permissions: { canPublish: true, canPublishSources: sources },
+      publishTrack,
+      unpublishTrack: vi.fn(),
+      getTrackPublication: (): unknown => ({ track: micTrack }),
+    },
+  };
+  return { room, sender, original, micTrack, publishTrack };
+}
+
+test("[EC #47] under a mic-only grant the clip is mixed into the microphone, then the mic is restored", async () => {
+  const { room, sender, original, publishTrack } = makePolicyRoom([2]);
+  const stop = startLotusAudioInject(mockVm([room]));
+
+  send({ url: "https://example.com/clip.mp3" });
+  await flush();
+
+  expect(reply).toHaveBeenCalledWith(expect.anything(), {});
+  expect(publishTrack).not.toHaveBeenCalled();
+  expect(sender.replaceTrack).toHaveBeenCalledTimes(1);
+  const mixed = sender.track;
+  expect(mixed).not.toBe(original);
+  const source = contextInstances[0].createBufferSource.mock.results[0].value;
+  expect(source.start).toHaveBeenCalled();
+
+  source.onended();
+  await flush();
+  expect(sender.replaceTrack).toHaveBeenLastCalledWith(original);
+  expect(sender.track).toBe(original);
+  expect(mixed.stop).toHaveBeenCalled();
+
+  stop();
+});
+
+test("[EC #47] muting mid-clip stops the clip and restores the microphone", async () => {
+  const { room, sender, original, micTrack } = makePolicyRoom([2]);
+  const stop = startLotusAudioInject(mockVm([room]));
+
+  send({ url: "https://example.com/clip.mp3" });
+  await flush();
+  const source = contextInstances[0].createBufferSource.mock.results[0].value;
+
+  micTrack.emit("muted");
+  await flush();
+  expect(source.stop).toHaveBeenCalled();
+  expect(sender.track).toBe(original);
+
+  stop();
+});
+
+test("[EC #47] a track LiveKit swapped in meanwhile is not overwritten on restore", async () => {
+  const { room, sender, original } = makePolicyRoom([2]);
+  const stop = startLotusAudioInject(mockVm([room]));
+
+  send({ url: "https://example.com/clip.mp3" });
+  await flush();
+  const newer = makeTrack();
+  sender.track = newer; // e.g. a device switch while the clip plays
+  const source = contextInstances[0].createBufferSource.mock.results[0].value;
+  source.onended();
+  await flush();
+  expect(sender.track).toBe(newer);
+  expect(sender.replaceTrack).not.toHaveBeenCalledWith(original);
+
+  stop();
+});
+
+test("[EC #47] no microphone grant at all is still refused up front", async () => {
+  const { room, publishTrack, sender } = makePolicyRoom([3]);
+  const stop = startLotusAudioInject(mockVm([room]));
+
+  send({ url: "https://example.com/clip.mp3" });
+  await flush();
+
+  expect(reply).toHaveBeenCalledWith(expect.anything(), {
+    played: false,
+    reason: "not_permitted",
+  });
+  expect(publishTrack).not.toHaveBeenCalled();
+  expect(sender.replaceTrack).not.toHaveBeenCalled();
+
+  stop();
+});

src/lotus/lotusAudioInject.ts

@@ -5,7 +5,12 @@ SPDX-License-Identifier: AGPL-3.0-only OR LicenseRef-Element-Commercial
 Please see LICENSE in the repository root for full details.
 */
 
-import { type Room as LivekitRoom, Track } from "livekit-client";
+import {
+  type LocalTrack,
+  type Room as LivekitRoom,
+  Track,
+  TrackEvent,
+} from "livekit-client";
 import { logger } from "matrix-js-sdk/lib/logger";
 import { type IWidgetApiRequest } from "matrix-widget-api";
 
@@ -52,6 +57,10 @@ function acquireSharedAudio(): {
  * that was impossible against the prebuilt EC bundle (LiveKit's
  * LocalParticipant lived in EC's module scope).
  *
+ * [EC #47] Exception: rooms with a call policy issue tokens that only allow
+ * listed sources, which refuse that separate track. There the clip is mixed
+ * into the published microphone for its duration (`playIntoMicrophone`).
+ *
  * Action data: `{ url?: string, audio?: ArrayBuffer, volume?: number }`.
  * `audio` is the clip's bytes (cinny #43: a host `blob:` URL can't be fetched
  * once EC is served from its own origin, so the host sends the bytes and they
@@ -76,6 +85,25 @@ export function canPublishSoundboard(
   return sources.length === 0 || sources.includes(0);
 }
 
+/**
+ * Pure: how a clip can reach the call under these permissions (EC #47).
+ * - `"track"`: as its own Unknown-source track (unrestricted rooms).
+ * - `"mic"`: mixed into the published microphone track. Rooms with a call
+ *   policy grant only listed sources (microphone, maybe screenshare), which
+ *   refuse the separate track, but the mic is allowed.
+ * - `null`: neither (no publishing at all).
+ */
+export function soundboardMode(
+  perm:
+    | { canPublish?: boolean; canPublishSources?: readonly number[] }
+    | undefined,
+): "track" | "mic" | null {
+  if (canPublishSoundboard(perm)) return "track";
+  if (perm?.canPublish === false) return null;
+  // TrackSource.MICROPHONE = 2
+  return perm?.canPublishSources?.includes(2) ? "mic" : null;
+}
+
 export function startLotusAudioInject(vm: CallViewModel): () => void {
   const w = widget;
   if (!w) return () => undefined;
@@ -132,21 +160,24 @@ export function startLotusAudioInject(vm: CallViewModel): () => void {
       return;
     }
 
-    // [EC #47] Clips publish as an Unknown-source track. Audio-only rooms get
-    // a token limited to the microphone source, where LiveKit refuses that
-    // publish *after* we replied; say so up front instead of failing silently.
-    if (
-      rooms.some((r) => !canPublishSoundboard(r.localParticipant.permissions))
-    ) {
+    // [EC #47] Clips publish as an Unknown-source track, which rooms with a
+    // call policy refuse (their tokens list allowed sources). There the clip
+    // is mixed into the published microphone instead; only when neither is
+    // possible say so up front instead of failing silently.
+    const modes = rooms.map((r) =>
+      soundboardMode(r.localParticipant.permissions),
+    );
+    if (modes.some((m) => m === null)) {
       w.api.transport.reply(ev.detail, {
         played: false,
         reason: "not_permitted",
       });
       return;
     }
+    const mode = modes.includes("mic") ? "mic" : "track";
 
     w.api.transport.reply(ev.detail, {});
-    void playInjectedClip(clip, volume, rooms, activeClips).catch((e) =>
+    void playInjectedClip(clip, volume, rooms, activeClips, mode).catch((e) =>
       logger.warn("[lotus] inject_audio failed", e),
     );
   };
@@ -211,6 +242,7 @@ async function playInjectedClip(
   volume: number,
   rooms: LivekitRoom[],
   activeClips: Set<() => void>,
+  mode: "track" | "mic" = "track",
 ): Promise<void> {
   if (rooms.length === 0) {
     logger.warn("[lotus] inject_audio: no connected rooms");
@@ -295,8 +327,23 @@ async function playInjectedClip(
   gain.gain.value = volume;
   const source = ctx.createBufferSource();
   source.buffer = buffer;
-  source.connect(gain).connect(dest);
+  source.connect(gain);
 
+  if (mode === "mic") {
+    playIntoMicrophone(
+      ctx,
+      source,
+      gain,
+      buffer,
+      rooms,
+      activeClips,
+      placeholder,
+      () => aborted,
+    );
+    return;
+  }
+
+  gain.connect(dest);
   const mst = dest.stream.getAudioTracks()[0];
   if (!mst) {
     source.disconnect();
@@ -375,3 +422,114 @@ async function playInjectedClip(
 
   source.start();
 }
+
+/**
+ * [EC #47] Mix the clip into the published microphone for its duration. For
+ * every room, the track currently going out on the mic's RTCRtpSender (the
+ * denoised one, when noise suppression is on) and the clip are mixed in a
+ * MediaStreamAudioDestinationNode, and the sender switches to the mix with
+ * `replaceTrack`. The server still sees one ordinary microphone track, which
+ * rooms with a call policy allow. When the clip ends the original track goes
+ * back. Muting mid-clip stops the clip (#13: muted means silent), and if
+ * LiveKit swapped its own track in meanwhile (device change, denoise toggle)
+ * that one is left alone.
+ */
+function playIntoMicrophone(
+  ctx: AudioContext,
+  source: AudioBufferSourceNode,
+  gain: GainNode,
+  buffer: AudioBuffer,
+  rooms: LivekitRoom[],
+  activeClips: Set<() => void>,
+  placeholder: () => void,
+  isAborted: () => boolean,
+): void {
+  type Mix = {
+    sender: RTCRtpSender;
+    original: MediaStreamTrack;
+    mixed: MediaStreamTrack;
+    mic: MediaStreamAudioSourceNode;
+    dest: MediaStreamAudioDestinationNode;
+    track: LocalTrack;
+  };
+  const mixes: Mix[] = [];
+  let cleanedUp = false;
+  const cleanup = (): void => {
+    if (cleanedUp) return;
+    cleanedUp = true;
+    activeClips.delete(cleanup);
+    activeClips.delete(placeholder);
+    try {
+      source.stop();
+    } catch {
+      /* not started or already stopped */
+    }
+    source.disconnect();
+    gain.disconnect();
+    for (const m of mixes) {
+      m.track.off(TrackEvent.Muted, cleanup);
+      m.mic.disconnect();
+      // Only undo our own swap: LiveKit may have replaced the track since.
+      if (m.sender.track === m.mixed)
+        void m.sender.replaceTrack(m.original).catch(() => undefined);
+      m.mixed.stop();
+    }
+  };
+
+  for (const room of rooms) {
+    const track = room.localParticipant.getTrackPublication(
+      Track.Source.Microphone,
+    )?.track;
+    const sender = track?.sender;
+    const original = sender?.track;
+    if (!track || !sender || !original) continue;
+    const dest = ctx.createMediaStreamDestination();
+    const mic = ctx.createMediaStreamSource(new MediaStream([original]));
+    mic.connect(dest);
+    gain.connect(dest);
+    const mixed = dest.stream.getAudioTracks()[0];
+    if (!mixed) {
+      mic.disconnect();
+      continue;
+    }
+    mixes.push({ sender, original, mixed, mic, dest, track });
+    track.on(TrackEvent.Muted, cleanup);
+  }
+
+  activeClips.delete(placeholder);
+  activeClips.add(cleanup);
+  if (mixes.length === 0) {
+    logger.warn("[lotus] inject_audio: no published microphone to mix into");
+    cleanup();
+    return;
+  }
+  if (isAborted()) {
+    cleanup();
+    return;
+  }
+
+  // Switch every sender to its mix, then start the clip once they're live so
+  // its start isn't clipped.
+  void Promise.all(
+    mixes.map(async (m) =>
+      m.sender.replaceTrack(m.mixed).catch((e) => {
+        logger.warn("[lotus] inject_audio: replaceTrack failed", e);
+      }),
+    ),
+  ).then(() => {
+    if (cleanedUp || isAborted()) {
+      cleanup();
+      return;
+    }
+    source.onended = cleanup;
+    const durationMs = Number.isFinite(buffer.duration)
+      ? buffer.duration * 1000 + 500
+      : MAX_CLIP_MS;
+    const guard = setTimeout(
+      cleanup,
+      Math.min(MAX_CLIP_MS, Math.max(0, durationMs)),
+    );
+    source.addEventListener("ended", () => clearTimeout(guard));
+    source.start();
+  });
+}

src/lotus/lotusCallSummary.test.ts

@@ -8,7 +8,11 @@ Please see LICENSE in the repository root for full details.
 import { ConnectionQuality } from "livekit-client";
 import { describe, expect, it } from "vitest";
 
-import { CallQualityTracker } from "./lotusCallSummary";
+import {
+  CallDetailsTracker,
+  CallQualityTracker,
+  matrixUserOf,
+} from "./lotusCallSummary";
 
 const clock = (): { now: () => number; tick: (ms: number) => void } => {
   let t = 1_000_000;
@@ -91,3 +95,92 @@ describe("CallQualityTracker", () => {
     });
   });
 });
+
+describe("[cinny #253] CallDetailsTracker", () => {
+  it("participants, peak, and devices of one person counted once", () => {
+    const c = clock();
+    const d = new CallDetailsTracker(c.now);
+    d.remoteJoined("@bob:x:DEV1");
+    d.remoteJoined("@bob:x:DEV2");
+    d.remoteJoined("@carol:x:D");
+    d.remoteLeft("@carol:x:D");
+    d.remoteJoined("@dave:x:D");
+    const r = d.details();
+    expect(r.participants).toEqual(["@bob:x", "@carol:x", "@dave:x"]);
+    expect(r.peakParticipants).toBe(3); // you + bob + carol (dave came after carol left)
+  });
+
+  it("joining is idempotent per device (seeding everyone present on connect)", () => {
+    const d = new CallDetailsTracker();
+    d.remoteJoined("@bob:x:DEV1");
+    d.remoteJoined("@bob:x:DEV1"); // seen again by the on-connect seed
+    d.remoteLeft("@bob:x:DEV1"); // one leave: bob is gone
+    d.remoteJoined("@carol:x:D");
+    d.remoteJoined("@dave:x:D"); // peak is you + carol + dave, not + a ghost bob
+    d.setQuality("@bob:x:DEV1", ConnectionQuality.Poor);
+    expect(d.details().participants).toEqual(["@bob:x", "@carol:x", "@dave:x"]);
+    expect(d.details().peakParticipants).toBe(3);
+  });
+
+  it("poor time per person, and everyone-at-once only while all are poor", () => {
+    const c = clock();
+    const d = new CallDetailsTracker(c.now);
+    d.remoteJoined("@bob:x:D");
+    d.setQuality(undefined, ConnectionQuality.Poor); // you poor
+    c.tick(10_000);
+    d.setQuality("@bob:x:D", ConnectionQuality.Lost); // now everyone
+    c.tick(5_000);
+    d.setQuality(undefined, ConnectionQuality.Good);
+    c.tick(3_000);
+    d.setQuality("@bob:x:D", ConnectionQuality.Excellent);
+    const q = d.details().quality;
+    expect(q.youPoorMs).toBe(15_000);
+    expect(q.othersPoorMs).toEqual({ "@bob:x": 8_000 });
+    expect(q.everyonePoorMs).toBe(5_000);
+  });
+
+  it("network numbers: RTT median/p95, loss from the latest counters, jitter", () => {
+    const d = new CallDetailsTracker();
+    for (const rtt of [0.02, 0.03, 0.04, 0.05, 0.3]) d.addRtt(rtt);
+    d.addRtt(Number.NaN);
+    d.setSent("mic", 900, 5);
+    d.setSent("mic", 995, 5); // cumulative: latest wins
+    d.setReceived("bob", 490, 10);
+    d.addJitter(0.004);
+    d.addJitter(0.006);
+    const n = d.details().network;
+    expect(n.rttMedianMs).toBe(40);
+    expect(n.rttP95Ms).toBe(300);
+    expect(n.outboundLossPct).toBe(0.5);
+    expect(n.inboundLossPct).toBe(2);
+    expect(n.jitterMs).toBe(5);
+  });
+
+  it("no samples: network numbers are absent, not zero", () => {
+    expect(new CallDetailsTracker().details().network).toEqual({});
+  });
+
+  it("talk, mute and screenshare time", () => {
+    const c = clock();
+    const d = new CallDetailsTracker(c.now);
+    d.setMuted(true);
+    c.tick(4_000);
+    d.setMuted(false);
+    d.setSpeaking(true);
+    c.tick(2_000);
+    d.setSpeaking(false);
+    d.setScreensharing(true);
+    c.tick(6_000); // still sharing when summarised
+    expect(d.details().activity).toEqual({
+      speakingMs: 2_000,
+      mutedMs: 4_000,
+      screenshareMs: 6_000,
+    });
+  });
+
+  it("matrixUserOf reduces a LiveKit identity to the Matrix user", () => {
+    expect(matrixUserOf("@bob:example.org:ABCDEF")).toBe("@bob:example.org");
+    expect(matrixUserOf("@bob:example.org")).toBe("@bob:example.org");
+    expect(matrixUserOf("hashed-federated-id")).toBe("hashed-federated-id");
+  });
+});

src/lotus/lotusCallSummary.ts

@@ -7,9 +7,16 @@ Please see LICENSE in the repository root for full details.
 
 import {
   ConnectionQuality,
+  type LocalAudioTrack,
+  type LocalTrackPublication,
   type Participant,
+  ParticipantEvent,
+  type RemoteAudioTrack,
+  type RemoteParticipant,
   type Room as LivekitRoom,
   RoomEvent,
+  Track,
+  type TrackPublication,
 } from "livekit-client";
 
 import { type CallViewModel } from "../state/CallViewModel/CallViewModel";
@@ -27,6 +34,241 @@ export interface CallSummary {
   /** Time the local connection quality was reported poor/lost, ms. */
   poorMs: number;
   verdict: CallQualityVerdict;
+  /** [cinny #253] The breakdown behind the host's "Details" card. */
+  details?: CallSummaryDetails;
+}
+
+/** [cinny #253] Everything the host's post-call Details card shows. */
+export interface CallSummaryDetails {
+  /** Other Matrix users seen in the call (`@user:server`), first-seen order. */
+  participants: string[];
+  /** Most people in the call at once, you included. */
+  peakParticipants: number;
+  quality: {
+    /** Your connection reported poor/lost, ms. */
+    youPoorMs: number;
+    /** Each other person's connection reported poor/lost, ms (only > 0). */
+    othersPoorMs: Record<string, number>;
+    /** Everyone, you included, poor at the same time (points at the server). */
+    everyonePoorMs: number;
+  };
+  network: {
+    rttMedianMs?: number;
+    rttP95Ms?: number;
+    /** Your microphone's packets the SFU reported lost, %. */
+    outboundLossPct?: number;
+    /** Packets lost on the audio you received, %. */
+    inboundLossPct?: number;
+    jitterMs?: number;
+  };
+  activity: { speakingMs: number; mutedMs: number; screenshareMs: number };
+}
+
+/** Time spent in an on/off state, e.g. muted. */
+class Stopwatch {
+  private since: number | undefined;
+
+  private total = 0;
+
+  public set(on: boolean, now: number): void {
+    if (on && this.since === undefined) this.since = now;
+    if (!on && this.since !== undefined) {
+      this.total += now - this.since;
+      this.since = undefined;
+    }
+  }
+
+  public get on(): boolean {
+    return this.since !== undefined;
+  }
+
+  public ms(now: number): number {
+    return this.total + (this.since === undefined ? 0 : now - this.since);
+  }
+}
+
+/** `@user:server:DEVICE` (a LiveKit identity) → `@user:server`. */
+export function matrixUserOf(identity: string): string {
+  if (!identity.startsWith("@")) return identity;
+  const first = identity.indexOf(":");
+  const second = first === -1 ? -1 : identity.indexOf(":", first + 1);
+  return second === -1 ? identity : identity.slice(0, second);
+}
+
+const percentile = (sorted: number[], p: number): number =>
+  sorted[Math.min(sorted.length - 1, Math.floor(p * sorted.length))];
+
+/**
+ * [cinny #253] Who was in the call, whose connection was poor and when,
+ * network numbers, and your own activity. Fed by LiveKit events and periodic
+ * stats samples; nothing leaves the frame except the final summary, sent to
+ * the host only.
+ */
+export class CallDetailsTracker {
+  // user -> identities (devices) currently in the call
+  private readonly devices = new Map<string, Set<string>>();
+
+  private readonly seen: string[] = [];
+
+  private peak = 1;
+
+  private readonly poor = new Map<string, Stopwatch>();
+
+  private readonly youPoor = new Stopwatch();
+
+  private readonly everyonePoor = new Stopwatch();
+
+  private readonly rtts: number[] = [];
+
+  private readonly jitters: number[] = [];
+
+  private readonly sent = new Map<string, { packets: number; lost: number }>();
+
+  private readonly received = new Map<
+    string,
+    { packets: number; lost: number }
+  >();
+
+  private readonly speaking = new Stopwatch();
+
+  private readonly muted = new Stopwatch();
+
+  private readonly sharing = new Stopwatch();
+
+  public constructor(private readonly now: () => number = () => Date.now()) {}
+
+  /** Idempotent per device: safe to call for everyone already present too. */
+  public remoteJoined(identity: string): void {
+    const user = matrixUserOf(identity);
+    let ids = this.devices.get(user);
+    if (!ids) this.devices.set(user, (ids = new Set()));
+    ids.add(identity);
+    if (!this.seen.includes(user)) this.seen.push(user);
+    this.peak = Math.max(this.peak, this.devices.size + 1);
+    this.updateEveryone();
+  }
+
+  public remoteLeft(identity: string): void {
+    const user = matrixUserOf(identity);
+    const ids = this.devices.get(user);
+    ids?.delete(identity);
+    if (!ids || ids.size === 0) {
+      this.devices.delete(user);
+      this.poor.get(user)?.set(false, this.now());
+    }
+    this.updateEveryone();
+  }
+
+  /** `identity` undefined = the local participant. */
+  public setQuality(
+    identity: string | undefined,
+    quality: ConnectionQuality,
+  ): void {
+    if (quality === ConnectionQuality.Unknown) return;
+    const bad =
+      quality === ConnectionQuality.Poor || quality === ConnectionQuality.Lost;
+    if (identity === undefined) this.youPoor.set(bad, this.now());
+    else {
+      const user = matrixUserOf(identity);
+      let sw = this.poor.get(user);
+      if (!sw) this.poor.set(user, (sw = new Stopwatch()));
+      sw.set(bad, this.now());
+    }
+    this.updateEveryone();
+  }
+
+  private updateEveryone(): void {
+    const others = [...this.devices.keys()];
+    const all =
+      this.youPoor.on &&
+      others.length > 0 &&
+      others.every((u) => this.poor.get(u)?.on);
+    this.everyonePoor.set(all, this.now());
+  }
+
+  public addRtt(seconds: number | undefined): void {
+    if (typeof seconds === "number" && Number.isFinite(seconds) && seconds >= 0)
+      this.rtts.push(seconds * 1000);
+  }
+
+  public addJitter(seconds: number | undefined): void {
+    if (typeof seconds === "number" && Number.isFinite(seconds) && seconds >= 0)
+      this.jitters.push(seconds * 1000);
+  }
+
+  /** Cumulative counters for one sent/received track (latest value wins). */
+  public setSent(key: string, packets?: number, lost?: number): void {
+    if (typeof packets === "number")
+      this.sent.set(key, { packets, lost: Math.max(0, lost ?? 0) });
+  }
+
+  public setReceived(key: string, packets?: number, lost?: number): void {
+    if (typeof packets === "number")
+      this.received.set(key, { packets, lost: Math.max(0, lost ?? 0) });
+  }
+
+  public setSpeaking(on: boolean): void {
+    this.speaking.set(on, this.now());
+  }
+
+  public setMuted(on: boolean): void {
+    this.muted.set(on, this.now());
+  }
+
+  public setScreensharing(on: boolean): void {
+    this.sharing.set(on, this.now());
+  }
+
+  public details(): CallSummaryDetails {
+    const now = this.now();
+    const othersPoorMs: Record<string, number> = {};
+    for (const [user, sw] of this.poor) {
+      const ms = sw.ms(now);
+      if (ms > 0) othersPoorMs[user] = ms;
+    }
+    const lossPct = (
+      m: Map<string, { packets: number; lost: number }>,
+    ): number | undefined => {
+      let packets = 0;
+      let lost = 0;
+      for (const v of m.values()) {
+        packets += v.packets;
+        lost += v.lost;
+      }
+      return packets + lost > 0
+        ? Math.round((lost / (packets + lost)) * 1000) / 10
+        : undefined;
+    };
+    const rtts = [...this.rtts].sort((a, b) => a - b);
+    const network: CallSummaryDetails["network"] = {};
+    if (rtts.length) {
+      network.rttMedianMs = Math.round(percentile(rtts, 0.5));
+      network.rttP95Ms = Math.round(percentile(rtts, 0.95));
+    }
+    const out = lossPct(this.sent);
+    if (out !== undefined) network.outboundLossPct = out;
+    const inn = lossPct(this.received);
+    if (inn !== undefined) network.inboundLossPct = inn;
+    if (this.jitters.length)
+      network.jitterMs = Math.round(
+        this.jitters.reduce((a, b) => a + b, 0) / this.jitters.length,
+      );
+    return {
+      participants: [...this.seen],
+      peakParticipants: this.peak,
+      quality: {
+        youPoorMs: this.youPoor.ms(now),
+        othersPoorMs,
+        everyonePoorMs: this.everyonePoor.ms(now),
+      },
+      network,
+      activity: {
+        speakingMs: this.speaking.ms(now),
+        mutedMs: this.muted.ms(now),
+        screenshareMs: this.sharing.ms(now),
+      },
+    };
+  }
 }
 
 /**
@@ -93,13 +335,48 @@ export class CallQualityTracker {
 export function startLotusCallSummary(vm: CallViewModel): () => void {
   if (!widget) return () => undefined;
   const tracker = new CallQualityTracker();
+  const details = new CallDetailsTracker();
   let sent = false;
   const send = (): void => {
     if (sent) return;
     sent = true;
-    lotusSendToHost(LotusWidgetActions.CallSummary, tracker.summary());
+    clearInterval(sampler);
+    lotusSendToHost(LotusWidgetActions.CallSummary, {
+      ...tracker.summary(),
+      details: details.details(),
+    });
   };
 
+  // [cinny #253] Network numbers: sample sender/receiver stats every 5 s.
+  const sample = async (room: LivekitRoom): Promise<void> => {
+    const mic = room.localParticipant.getTrackPublication(
+      Track.Source.Microphone,
+    )?.track as LocalAudioTrack | undefined;
+    const s = await mic?.getSenderStats?.().catch(() => undefined);
+    if (s) {
+      details.addRtt(s.roundTripTime);
+      details.setSent(`${room.name}:mic`, s.packetsSent, s.packetsLost);
+    }
+    for (const p of room.remoteParticipants.values()) {
+      for (const pub of p.audioTrackPublications.values()) {
+        const t = pub.track as RemoteAudioTrack | undefined;
+        const r = await t?.getReceiverStats?.().catch(() => undefined);
+        if (r) {
+          details.addJitter(r.jitter);
+          details.setReceived(
+            `${room.name}:${p.identity}:${pub.trackSid}`,
+            r.packetsReceived,
+            r.packetsLost,
+          );
+        }
+      }
+    }
+  };
+  const sampler = setInterval(() => {
+    for (const room of listeners.keys())
+      void sample(room).catch(() => undefined);
+  }, 5_000);
+
   const listeners = new Map<LivekitRoom, () => void>();
   const attach = (room: LivekitRoom): void => {
     const onQuality = (
@@ -107,20 +384,69 @@ export function startLotusCallSummary(vm: CallViewModel): () => void {
       participant: Participant,
     ): void => {
       if (participant.isLocal) tracker.setQuality(quality);
+      details.setQuality(
+        participant.isLocal ? undefined : participant.identity,
+        quality,
+      );
+    };
+    // LiveKit doesn't send ParticipantConnected for people already in the
+    // room when we connect, and the mic isn't published yet at attach time:
+    // record both once the connection is up.
+    const onConnected = (): void => {
+      tracker.start();
+      for (const p of room.remoteParticipants.values())
+        details.remoteJoined(p.identity);
+      details.setMuted(!room.localParticipant.isMicrophoneEnabled);
     };
-    const onConnected = (): void => tracker.start();
     const onReconnecting = (): void => tracker.reconnect();
     const onDisconnected = (): void => send();
+    const onJoined = (p: RemoteParticipant): void =>
+      details.remoteJoined(p.identity);
+    const onLeft = (p: RemoteParticipant): void =>
+      details.remoteLeft(p.identity);
+    const isLocalMic = (pub: TrackPublication, p: Participant): boolean =>
+      p.isLocal && pub.source === Track.Source.Microphone;
+    const onMuted = (pub: TrackPublication, p: Participant): void => {
+      if (isLocalMic(pub, p)) details.setMuted(true);
+    };
+    const onUnmuted = (pub: TrackPublication, p: Participant): void => {
+      if (isLocalMic(pub, p)) details.setMuted(false);
+    };
+    const onLocalPublished = (pub: LocalTrackPublication): void => {
+      if (pub.source === Track.Source.ScreenShare)
+        details.setScreensharing(true);
+      if (pub.source === Track.Source.Microphone) details.setMuted(pub.isMuted);
+    };
+    const onLocalUnpublished = (pub: LocalTrackPublication): void => {
+      if (pub.source === Track.Source.ScreenShare)
+        details.setScreensharing(false);
+    };
+    const onSpeaking = (speaking: boolean): void =>
+      details.setSpeaking(speaking);
     room.on(RoomEvent.ConnectionQualityChanged, onQuality);
     room.on(RoomEvent.Connected, onConnected);
     room.on(RoomEvent.Reconnecting, onReconnecting);
     room.on(RoomEvent.Disconnected, onDisconnected);
-    if (room.state === "connected") tracker.start();
+    room.on(RoomEvent.ParticipantConnected, onJoined);
+    room.on(RoomEvent.ParticipantDisconnected, onLeft);
+    room.on(RoomEvent.TrackMuted, onMuted);
+    room.on(RoomEvent.TrackUnmuted, onUnmuted);
+    room.on(RoomEvent.LocalTrackPublished, onLocalPublished);
+    room.on(RoomEvent.LocalTrackUnpublished, onLocalUnpublished);
+    room.localParticipant.on(ParticipantEvent.IsSpeakingChanged, onSpeaking);
+    if (room.state === "connected") onConnected();
     listeners.set(room, () => {
       room.off(RoomEvent.ConnectionQualityChanged, onQuality);
       room.off(RoomEvent.Connected, onConnected);
       room.off(RoomEvent.Reconnecting, onReconnecting);
       room.off(RoomEvent.Disconnected, onDisconnected);
+      room.off(RoomEvent.ParticipantConnected, onJoined);
+      room.off(RoomEvent.ParticipantDisconnected, onLeft);
+      room.off(RoomEvent.TrackMuted, onMuted);
+      room.off(RoomEvent.TrackUnmuted, onUnmuted);
+      room.off(RoomEvent.LocalTrackPublished, onLocalPublished);
+      room.off(RoomEvent.LocalTrackUnpublished, onLocalUnpublished);
+      room.localParticipant.off(ParticipantEvent.IsSpeakingChanged, onSpeaking);
     });
   };
 

src/state/CallViewModel/CallViewModel.test.ts

@@ -451,14 +451,16 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
     });
   });
 
-  test("one-on-one mobile layout shows local tile when video is enabled", () => {
+  // [lotus / cinny #265] The local tile stays on mobile with the camera off
+  // (upstream hid it), so you can see yourself in a voice call.
+  test("one-on-one mobile layout shows the local tile whether or not video is enabled", () => {
     withTestScheduler(({ behavior, schedule, expectObservable }) => {
       // Local participant enables their video, then disables it
       const videoInputMarbles = "    ny--n";
-      // While tile is shown, tap the screen twice
+      // Tap the screen twice
       const tapScreenInputMarbles = "--aa-";
-      // Layout should show local tile, make it small, enlarge it again, then hide it
-      const expectedLayoutMarbles = "abcba";
+      // The local tile is shown throughout; taps make it small, then large again
+      const expectedLayoutMarbles = "b-cb";
 
       withCallViewModel(
         {
@@ -477,12 +479,6 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
           expectObservable(summarizeLayout$(vm.layout$)).toBe(
             expectedLayoutMarbles,
             {
-              a: {
-                type: "one-on-one-mobile",
-                spotlight: [`${aliceId}:0`],
-                pip: undefined,
-                pipSize: "lg",
-              },
               b: {
                 type: "one-on-one-mobile",
                 spotlight: [`${aliceId}:0`],
@@ -518,7 +514,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
             a: {
               type: "one-on-one-mobile",
               spotlight: [`${aliceId}:0`],
-              pip: undefined,
+              pip: `${localId}:0`,
               pipSize: "lg",
             },
           });
@@ -566,7 +562,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
               a: {
                 type: "one-on-one-mobile",
                 spotlight: [`${aliceId}:0`],
-                pip: undefined,
+                pip: `${localId}:0`,
                 pipSize: "sm",
               },
               b: {
@@ -765,7 +761,7 @@ describe.each(modes)("CallViewModel (%s mode)", (mode) => {
                 // This is the expected one-on-one layout for a narrow window
                 type: "one-on-one-mobile",
                 spotlight: [`${aliceId}:0`],
-                pip: undefined,
+                pip: `${localId}:0`,
                 pipSize: "lg",
               },
               b: {

src/state/CallViewModel/CallViewModel.ts

@@ -1220,19 +1220,22 @@ export function createCallViewModel$(
       }),
     );
 
+  // [lotus / cinny #265] Upstream drops the local PiP on mobile while your
+  // camera is off; in Lotus voice rooms that is almost always, so on a phone
+  // you never saw yourself (or your own speaking ring). Show it as desktop
+  // does: the tile falls back to your avatar.
   const oneOnOneMobileLayoutMedia$: Observable<OneOnOneMobileLayoutMedia | null> =
     oneOnOneLayoutMedia$.pipe(
-      switchMap((media) => {
-        if (media === null) return of(null);
-        return media.local.videoEnabled$.pipe(
-          map((videoEnabled) => ({
-            type: "one-on-one-mobile" as const,
-            edgeToEdge: true as const,
-            spotlight: media.remote,
-            pip: videoEnabled ? media.local : undefined,
-          })),
-        );
-      }),
+      map((media) =>
+        media === null
+          ? null
+          : {
+              type: "one-on-one-mobile" as const,
+              edgeToEdge: true as const,
+              spotlight: media.remote,
+              pip: media.local,
+            },
+      ),
     );
 
   const pipLayoutMedia$: Observable<LayoutMedia> = spotlight$.pipe(