LotusGuild/cinny
3
0
Fork 0
Code Issues 1 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

feat(voice): render the waveform on playback with click/drag/keyboard scrubbing

Browse Source 
Voice messages carry an MSC1767 waveform (org.matrix.msc1767.audio.waveform) and the
recorder draws a live one, but AudioContent playback only showed a plain seek bar.
Now the player renders the waveform as bars that fill with the accent (TDS green under
Lotus Terminal) as the clip plays, and the waveform itself is the seek control —
click, drag, or keyboard (arrows +/-5s, Home/End) with role=slider + ARIA value text.

- AudioContent: new optional "waveform" prop + a WaveformSeek sub-component
  (downsamples to 48 bars, mirrors the recorder's bar styling); falls back to the
  plain Range seek bar when there's no waveform.
- Threaded through MAudio (RenderAudioContentProps) so timeline voice messages get it
  automatically; the Media Gallery Audio tab passes it directly.

Improves both the timeline and the new gallery Audio tab at once.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
jared
2026-07-09 21:50:08 -04:00
co-authored by Claude Opus 4.8
parent 2d0c804abc
commit 56863d2649
4 changed files with 160 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LOTUS_FEATURES.md
+6 -3
View File
@@ -845,9 +845,12 @@ player.kick).
- **Accessibility** — `radiogroup`/`radio` (single, with arrow-key roving) or `group`/`checkbox` (multi) semantics, `aria-checked`/`aria-disabled`, winner announced to AT.
- Pure tally/visibility/winner + wire-format parsing live in `utils/poll.ts` (+ `poll.test.ts`, 14 tests incl. the stable/unstable round-trip).
### Voice Message Playback Speed
### Voice Message Playback (waveform + speed)
`AudioContent.tsx` adds a playback speed cycle button to voice message players. Available speeds: `[0.75, 1, 1.5, 2]×`. A `useEffect` sets `audioElement.playbackRate` whenever the speed selection changes.
`AudioContent.tsx` is the shared audio player (timeline + Media Gallery Audio tab):
- **Waveform scrubbing** — voice messages carry an MSC1767 waveform (`org.matrix.msc1767.audio.waveform`, sent by the recorder). Playback renders it as bars that fill with the accent as the clip plays (TDS green under Lotus Terminal), and the waveform **is** the seek control — click, drag, or keyboard (arrows ±5s, Home/End) to scrub (`role="slider"`, ARIA value text). Threaded through `MAudio` (`MsgTypeRenderers.tsx`) + passed directly by the gallery. Regular audio with no waveform keeps the plain seek bar.
- **Playback speed** — a cycle button (`[0.75, 1, 1.5, 2]×`); a `useEffect` sets `audioElement.playbackRate` and re-applies it on (re)load (the browser resets it).
---
@@ -1084,7 +1087,7 @@ A toggle in **Settings → Privacy** switches between sending `m.read` (public r
- Four tabs: **Images**, **Videos**, **Audio**, **Files** (each with a live count)
- **Images/Videos** — a month-grouped grid; tiles decrypt on demand (lazy, near-viewport), open a keyboard-navigable **lightbox** (←/→/Esc, prev/next)
- **Audio** — voice messages + audio files (`m.audio`) with an inline player (reuses `AudioContent`: play/seek/**speed control**; decrypts on play)
- **Audio** — voice messages + audio files (`m.audio`) with an inline player (reuses `AudioContent`: **waveform scrubbing** for voice messages, play/seek/**speed control**; decrypts on play)
- **Files** — name/size/sender rows with download
- **Jump to message** — a "Go to message" action on file rows, audio rows, and in the lightbox navigates the timeline to the source event (`useRoomNavigate`) and closes the drawer
- Encrypted media is decrypted client-side on demand (no lock placeholder); download works for all types