Voice call overlay like Discord (show speakers over games): needs in-depth planning #25

New Issue

2026-09-28T19:26:52-04:00

jared commented

2026-09-28 19:26:52 -04:00

Feature request: voice call overlay (like Discord's in-game overlay)

Status: needs in-depth planning before any code. This issue captures the goal, known constraints and the questions a plan has to answer.

Goal

While in a Lotus voice call and playing a game (or using any other full-screen app), show a small always-on-top overlay:

who is in the call and who is speaking (a speaking ring on avatars)
your own mute / deafen / push-to-talk state
optionally, incoming messages and notifications, and quick controls

The user should never have to alt-tab to see who is talking.

What already exists to build on

Call state: the web app already gets live participants, speaking and mute state from the Element Call fork (io.lotus.call_state, useCallSpeakers, the call status bar).
Global hotkeys: PTT and deafen work while a game has focus (#2, src-tauri/src/native/hotkeys.rs, Windows).
Native call integrations: tray, taskbar thumbnail buttons and media keys already reflect call state, so desktop-native call UI has precedent.

Approaches to evaluate

Separate overlay window (Tauri 2 multi-window): transparent, borderless, always on top, hidden from the taskbar, and click-through (set_ignore_cursor_events), fed call state from the main window over Tauri events.
- No code injection, so no anti-cheat risk.
- Only covers games in borderless windowed mode; exclusive-fullscreen games usually draw over topmost windows (to verify).
In-process overlay injection into the game's renderer (DirectX / Vulkan / OpenGL hooks), which is how classic game overlays work.
- Works in exclusive fullscreen.
- Anti-cheat systems (EAC, BattlEye, Vanguard…) can flag or ban it, and it's a large, security-sensitive native surface. Likely out of scope, but the plan should state that explicitly and why.
A compact "mini call window": a small, movable, always-on-top window you place yourself, with no click-through. The simplest fallback, and useful on its own.

Platform questions

Windows: transparency and click-through behaviour in WebView2, focus stealing (the overlay must never take focus from the game), multiple monitors and DPI.
Linux: under Wayland, apps generally can't make themselves always-on-top or choose their own position. Some compositors offer layer-shell or window rules; X11 and XWayland are more permissive. Pair this with the NVIDIA/Wayland rendering findings from #23/#24.
macOS: no desktop build today; out of scope unless that changes.

Data and privacy

The overlay window shouldn't need its own Matrix session. The main window passes it a minimal model: display names, avatars (as image bytes, since media is authenticated), and speaking/mute flags.
No message content in the overlay unless the user turns that on, since screenshots and streams would capture it.

UX questions to settle

Layout (vertical list or row of avatars), position (corner snapping or free drag), size and opacity.
Show everyone, or only people currently speaking?
A hotkey to toggle the overlay, and one to make it interactive (click-through off) for quick controls.
Per-game or per-app enable/disable? Auto-show only while in a call and a full-screen app has focus?
Performance budget: near-zero CPU/GPU when idle, and it must not cost game FPS.

Planning deliverables (before implementation)

Prototype approach 1 on Windows: a transparent click-through topmost window over a borderless game and over an exclusive-fullscreen game; measure CPU/GPU cost.
Decide on injection (approach 2) with a written anti-cheat risk assessment. Default: no.
Linux feasibility matrix: KDE Wayland, GNOME Wayland, X11, with and without NVIDIA.
UX mock-ups: overlay states (idle, speaking, muted, deafened, PTT held), settings screen.
Data contract between the main window and the overlay (events, payloads, update rate).
Phased plan: e.g. v1 mini call window → v2 click-through overlay on Windows → v3 Linux where supported.

Acceptance criteria (draft, for v1+)

In a call, the overlay shows participants and speaking state within ~200 ms of a change, with no focus stealing and no visible FPS impact in a borderless game.
It can be turned off entirely; settings persist; it disappears when you leave the call.

## Feature request: voice call overlay (like Discord's in-game overlay) **Status:** needs in-depth planning before any code. This issue captures the goal, known constraints and the questions a plan has to answer. ### Goal While in a Lotus voice call and playing a game (or using any other full-screen app), show a small always-on-top overlay: - who is in the call and **who is speaking** (a speaking ring on avatars) - your own **mute / deafen / push-to-talk** state - optionally, incoming messages and notifications, and quick controls The user should never have to alt-tab to see who is talking. ### What already exists to build on - **Call state:** the web app already gets live participants, speaking and mute state from the Element Call fork (`io.lotus.call_state`, `useCallSpeakers`, the call status bar). - **Global hotkeys:** PTT and deafen work while a game has focus (#2, `src-tauri/src/native/hotkeys.rs`, Windows). - **Native call integrations:** tray, taskbar thumbnail buttons and media keys already reflect call state, so desktop-native call UI has precedent. ### Approaches to evaluate 1. **Separate overlay window** (Tauri 2 multi-window): transparent, borderless, always on top, hidden from the taskbar, and click-through (`set_ignore_cursor_events`), fed call state from the main window over Tauri events. - No code injection, so no anti-cheat risk. - Only covers games in **borderless windowed** mode; exclusive-fullscreen games usually draw over topmost windows (to verify). 2. **In-process overlay injection** into the game's renderer (DirectX / Vulkan / OpenGL hooks), which is how classic game overlays work. - Works in exclusive fullscreen. - Anti-cheat systems (EAC, BattlEye, Vanguard…) can flag or ban it, and it's a large, security-sensitive native surface. Likely out of scope, but the plan should state that explicitly and why. 3. **A compact "mini call window"**: a small, movable, always-on-top window you place yourself, with no click-through. The simplest fallback, and useful on its own. ### Platform questions - **Windows:** transparency and click-through behaviour in WebView2, focus stealing (the overlay must never take focus from the game), multiple monitors and DPI. - **Linux:** under Wayland, apps generally can't make themselves always-on-top or choose their own position. Some compositors offer layer-shell or window rules; X11 and XWayland are more permissive. Pair this with the NVIDIA/Wayland rendering findings from #23/#24. - **macOS:** no desktop build today; out of scope unless that changes. ### Data and privacy - The overlay window shouldn't need its own Matrix session. The main window passes it a minimal model: display names, avatars (as image bytes, since media is authenticated), and speaking/mute flags. - No message content in the overlay unless the user turns that on, since screenshots and streams would capture it. ### UX questions to settle - Layout (vertical list or row of avatars), position (corner snapping or free drag), size and opacity. - Show everyone, or only people currently speaking? - A hotkey to toggle the overlay, and one to make it interactive (click-through off) for quick controls. - Per-game or per-app enable/disable? Auto-show only while in a call and a full-screen app has focus? - Performance budget: near-zero CPU/GPU when idle, and it must not cost game FPS. ### Planning deliverables (before implementation) - [ ] Prototype approach 1 on Windows: a transparent click-through topmost window over a borderless game and over an exclusive-fullscreen game; measure CPU/GPU cost. - [ ] Decide on injection (approach 2) with a written anti-cheat risk assessment. Default: no. - [ ] Linux feasibility matrix: KDE Wayland, GNOME Wayland, X11, with and without NVIDIA. - [ ] UX mock-ups: overlay states (idle, speaking, muted, deafened, PTT held), settings screen. - [ ] Data contract between the main window and the overlay (events, payloads, update rate). - [ ] Phased plan: e.g. v1 mini call window → v2 click-through overlay on Windows → v3 Linux where supported. ### Acceptance criteria (draft, for v1+) - In a call, the overlay shows participants and speaking state within ~200 ms of a change, with no focus stealing and no visible FPS impact in a borderless game. - It can be turned off entirely; settings persist; it disappears when you leave the call.

jared added the enhancement research needs-human-review area: native labels 2026-09-28 19:26:52 -04:00

jared commented

2026-11-09 17:06:32 -05:00

Plan: in-game voice overlay

Recommendation: build approach 1, a separate click-through overlay window, Windows first. Don't do in-process injection: it needs anti-cheat-sensitive hooks into the game's renderer, and approach 1 covers how most players run games today (borderless or "fullscreen optimised"). Linux waits for in-app calls (#21).

What we can reuse

Call state is already in the web app.
- CallEmbed.getLotusParticipants() / onLotusCallState() (cinny src/app/plugins/call/CallEmbed.ts) give { userId, speaking, audioEnabled, videoEnabled } per participant, live from the Element Call fork (io.lotus.call_state).
- useCallControlState gives your own mute/deafen, and pttActiveAtom gives PTT.
A web → native bridge for call state exists. useTauriThumbbar already sends {active, muted, deafened} to set_thumbbar / set_tray_call_state from a single effect. The overlay is one more consumer of the same pattern.
Fullscreen detection exists. native/focus_assist.rs already polls SHQueryUserNotificationState (QUNS_RUNNING_D3D_FULL_SCREEN, QUNS_BUSY), which tells us when a fullscreen app is in front.
Global hotkeys exist. native/hotkeys.rs uses a non-consuming key poll, which can give us an "unlock overlay" key without stealing keys from the game.

Design

1. Overlay window (Rust, new native/overlay.rs)

A second WebviewWindow labelled overlay:
- transparent, no decorations, no shadow;
- always_on_top, skip_taskbar, focusable(false);
- set_ignore_cursor_events(true), so it's click-through.
On Windows, also make sure of WS_EX_NOACTIVATE | WS_EX_TOOLWINDOW and show with SW_SHOWNOACTIVATE. Showing the overlay must never take focus from the game; that's the most important property to test.
It loads a tiny static page bundled with the desktop app (overlay.html, plain JS/CSS, no cinny bundle). It costs one extra WebView2 renderer instead of a second full client.
- Strict CSP: no network, no connect-src.
- Its own capability: event listening only, no commands.
- It never sees the access token.
Rust forwards state with app.emit_to("overlay", "overlay-state", …). The window is created lazily on the first call join and hidden (not destroyed) on leave.

2. State feed (cinny, new useTauriOverlay hook, desktop only)

While a call is joined and the setting is on, it sends set_overlay_state with a compact snapshot:
- self: { muted, deafened, ptt };
- people: [{ userId, name, avatar, speaking, muted }].
Avatars: small data URLs made in the main window from images it has already loaded, because media needs authentication and the overlay must not have the token. The MVP can use coloured initials and add avatars in phase 2.
Update rate: coalesced to about 10 updates a second at most. Speaking flips often, and the overlay only repaints on change.

3. When it shows

Default: in a call, and Lotus is not the foreground window, and the overlay is on in settings. Optionally, only when a fullscreen app is in front (focus_assist state).
It sits in a corner of the monitor the foreground window is on (MonitorFromWindow(GetForegroundWindow())).
Respects DPI and doesn't jump while you play: position is recomputed only when the foreground monitor changes.

4. Settings → Calls → "In-game overlay (beta)"

On/off (off by default while in beta).
Corner (4 options), size (S/M/L), opacity.
Show "speaking only" or "everyone in the call".
Phase 2 adds an "unlock to move" hotkey: the overlay becomes interactive and draggable, and the position is saved per monitor.

Known limits (to say in the UI or docs)

True exclusive fullscreen (older DX9/DX11 titles with exclusive mode) draws over every top-most window, so the overlay won't be visible there. Borderless and fullscreen-optimised (flip-model) games, which is most modern titles, show it.
A visible top-most window can stop the game using "independent flip". That's a small compositing cost and latency. Keeping the overlay small and repainting only on change keeps the cost low. Worth measuring once in a real game.
Linux: Wayland has no standard way for an app to keep its own window on top. KDE window rules or layer-shell could work, but GNOME has no equivalent. On top of that, in-app calls don't work on Linux yet (#21). So Linux comes after #21, X11/KDE first.
Anti-cheat: approach 1 doesn't touch the game process, so there's nothing for EAC, BattlEye or Vanguard to flag. We should still say "uses no game hooks" in the setting's help text.

Phases

MVP (Windows):
- overlay window plus overlay.html;
- useTauriOverlay;
- list of speakers with initials, speaking ring and muted icon;
- your own mute/deafen/PTT badge;
- show-when-unfocused rule, corner setting, off by default.
Avatars, opacity and size, the fullscreen-only option, and the unlock-to-move hotkey with a saved position.
Optional message/notification line (privacy: names only, opt-in). Linux X11/KDE after #21.

Tests

Rust unit tests for the show/hide rule and corner/DPI placement, as pure functions like gpu_workarounds::decide.
cinny vitest for building the snapshot and coalescing updates.
Windows smoke: check the overlay window's extended styles (top-most, transparent, no-activate), and that showing it doesn't change the foreground window. That second check needs the interactive-desktop runner from #19.
Manual: one borderless game and one exclusive-fullscreen game. Check that focus is never stolen, PTT still works, and there's no visible frame-time hit.

Decisions needed before building

MVP shows speaking only or everyone in the call? I suggest everyone, with the speakers highlighted, like Discord.
Show it only over fullscreen apps, or whenever Lotus isn't focused?
OK with off by default while it's in beta?
Should messages and notifications be in scope at all (phase 3), or voice only?

Estimate: the MVP is roughly one PR in each repo (cinny-desktop: window, page, command; cinny: hook and setting).

🤖 Generated with Claude Code

https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA

## Plan: in-game voice overlay **Recommendation:** build approach 1, a separate click-through overlay window, **Windows first**. Don't do in-process injection: it needs anti-cheat-sensitive hooks into the game's renderer, and approach 1 covers how most players run games today (borderless or "fullscreen optimised"). Linux waits for in-app calls (#21). ### What we can reuse - **Call state is already in the web app.** - `CallEmbed.getLotusParticipants()` / `onLotusCallState()` (cinny `src/app/plugins/call/CallEmbed.ts`) give `{ userId, speaking, audioEnabled, videoEnabled }` per participant, live from the Element Call fork (`io.lotus.call_state`). - `useCallControlState` gives your own mute/deafen, and `pttActiveAtom` gives PTT. - **A web → native bridge for call state exists.** `useTauriThumbbar` already sends `{active, muted, deafened}` to `set_thumbbar` / `set_tray_call_state` from a single effect. The overlay is one more consumer of the same pattern. - **Fullscreen detection exists.** `native/focus_assist.rs` already polls `SHQueryUserNotificationState` (`QUNS_RUNNING_D3D_FULL_SCREEN`, `QUNS_BUSY`), which tells us when a fullscreen app is in front. - **Global hotkeys exist.** `native/hotkeys.rs` uses a non-consuming key poll, which can give us an "unlock overlay" key without stealing keys from the game. ### Design **1. Overlay window (Rust, new `native/overlay.rs`)** - A second `WebviewWindow` labelled `overlay`: - transparent, no decorations, no shadow; - `always_on_top`, `skip_taskbar`, `focusable(false)`; - `set_ignore_cursor_events(true)`, so it's click-through. - On Windows, also make sure of `WS_EX_NOACTIVATE | WS_EX_TOOLWINDOW` and show with `SW_SHOWNOACTIVATE`. Showing the overlay must **never take focus from the game**; that's the most important property to test. - It loads a **tiny static page bundled with the desktop app** (`overlay.html`, plain JS/CSS, no cinny bundle). It costs one extra WebView2 renderer instead of a second full client. - Strict CSP: no network, no `connect-src`. - Its own capability: event listening only, no commands. - It never sees the access token. - Rust forwards state with `app.emit_to("overlay", "overlay-state", …)`. The window is created lazily on the first call join and hidden (not destroyed) on leave. **2. State feed (cinny, new `useTauriOverlay` hook, desktop only)** - While a call is joined and the setting is on, it sends `set_overlay_state` with a compact snapshot: - `self: { muted, deafened, ptt }`; - `people: [{ userId, name, avatar, speaking, muted }]`. - **Avatars:** small data URLs made in the main window from images it has already loaded, because media needs authentication and the overlay must not have the token. The MVP can use coloured initials and add avatars in phase 2. - **Update rate:** coalesced to about 10 updates a second at most. Speaking flips often, and the overlay only repaints on change. **3. When it shows** - Default: in a call, **and** Lotus is not the foreground window, **and** the overlay is on in settings. Optionally, only when a fullscreen app is in front (`focus_assist` state). - It sits in a corner of the monitor the foreground window is on (`MonitorFromWindow(GetForegroundWindow())`). - Respects DPI and doesn't jump while you play: position is recomputed only when the foreground monitor changes. **4. Settings → Calls → "In-game overlay (beta)"** - On/off (**off by default** while in beta). - Corner (4 options), size (S/M/L), opacity. - Show "speaking only" or "everyone in the call". - Phase 2 adds an "unlock to move" hotkey: the overlay becomes interactive and draggable, and the position is saved per monitor. ### Known limits (to say in the UI or docs) - **True exclusive fullscreen** (older DX9/DX11 titles with exclusive mode) draws over every top-most window, so the overlay won't be visible there. Borderless and fullscreen-optimised (flip-model) games, which is most modern titles, show it. - **A visible top-most window can stop the game using "independent flip".** That's a small compositing cost and latency. Keeping the overlay small and repainting only on change keeps the cost low. Worth measuring once in a real game. - **Linux:** Wayland has no standard way for an app to keep its own window on top. KDE window rules or layer-shell could work, but GNOME has no equivalent. On top of that, in-app calls don't work on Linux yet (#21). So Linux comes after #21, X11/KDE first. - **Anti-cheat:** approach 1 doesn't touch the game process, so there's nothing for EAC, BattlEye or Vanguard to flag. We should still say "uses no game hooks" in the setting's help text. ### Phases 1. **MVP (Windows):** - overlay window plus `overlay.html`; - `useTauriOverlay`; - list of speakers with initials, speaking ring and muted icon; - your own mute/deafen/PTT badge; - show-when-unfocused rule, corner setting, off by default. 2. Avatars, opacity and size, the fullscreen-only option, and the unlock-to-move hotkey with a saved position. 3. Optional message/notification line (privacy: names only, opt-in). Linux X11/KDE after #21. ### Tests - **Rust unit tests** for the show/hide rule and corner/DPI placement, as pure functions like `gpu_workarounds::decide`. - **cinny vitest** for building the snapshot and coalescing updates. - **Windows smoke:** check the overlay window's extended styles (top-most, transparent, no-activate), and that showing it doesn't change the foreground window. That second check needs the interactive-desktop runner from #19. - **Manual:** one borderless game and one exclusive-fullscreen game. Check that focus is never stolen, PTT still works, and there's no visible frame-time hit. ### Decisions needed before building 1. MVP shows **speaking only** or **everyone in the call**? I suggest everyone, with the speakers highlighted, like Discord. 2. Show it **only over fullscreen apps**, or whenever Lotus isn't focused? 3. OK with **off by default** while it's in beta? 4. Should messages and notifications be in scope at all (phase 3), or voice only? Estimate: the MVP is roughly one PR in each repo (cinny-desktop: window, page, command; cinny: hook and setting). 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA

jared referenced this issue from a commit

2026-11-09 17:24:19 -05:00

feat(overlay): in-game voice overlay window, Windows (#25)

jared referenced this issue

2026-11-09 17:26:24 -05:00

feat(overlay): in-game voice overlay window, Windows (#25) #50

jared referenced this issue from a commit

2026-11-09 21:32:28 -05:00

Merge pull request 'feat(overlay): in-game voice overlay window, Windows (#25)' (#50) from feat/voice-overlay into main

jared commented

2026-11-09 21:33:04 -05:00

Phase 1 (MVP) merged (2026-11-09): #50 (desktop window) and LotusGuild/cinny#488 (setting + feed). It ships in the next desktop release, off by default: Settings → Calls → In-game overlay (beta).

What's verified:

unit tests (desktop 37/37, cinny 25/25);
the Windows-only code type-checks with clippy;
the real Windows (MSVC) build compiles;
the Windows smoke test shows no new failures. Its one failing check, "local server listening", fails identically on unchanged main (run 3145), so it's a runner problem.

Not verified yet (keeping this open for it): a real game on Windows.

Borderless game: the overlay shows in the chosen corner, the game keeps focus, and clicks pass through.
The speaking ring and the mute/deafen/PTT header follow the call.
Focusing Lotus hides it, and hanging up hides it.
The background is transparent (no white or black box).
Exclusive fullscreen: expected not to show.

Phases 2 and 3 (avatars, opacity and size, a fullscreen-only option, an unlock-to-move hotkey, Linux after #21) remain as planned above.

**Phase 1 (MVP) merged** (2026-11-09): #50 (desktop window) and LotusGuild/cinny#488 (setting + feed). It ships in the next desktop release, **off by default**: Settings → Calls → *In-game overlay (beta)*. **What's verified:** - unit tests (desktop 37/37, cinny 25/25); - the Windows-only code type-checks with clippy; - the real Windows (MSVC) build compiles; - the Windows smoke test shows no new failures. Its one failing check, "local server listening", fails identically on unchanged `main` (run 3145), so it's a runner problem. **Not verified yet (keeping this open for it):** a real game on Windows. 1. Borderless game: the overlay shows in the chosen corner, the game **keeps focus**, and clicks pass through. 2. The speaking ring and the mute/deafen/PTT header follow the call. 3. Focusing Lotus hides it, and hanging up hides it. 4. The background is transparent (no white or black box). 5. Exclusive fullscreen: expected not to show. Phases 2 and 3 (avatars, opacity and size, a fullscreen-only option, an unlock-to-move hotkey, Linux after #21) remain as planned above.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: LotusGuild/cinny-desktop#25