docs: document inline media embeds feature
- README: new bullet for inline media embeds (video/audio/post players + facade). - LOTUS_FEATURES: 'Inline Media Embeds' section (kinds table, facade, provider notes, sandbox, setting, CSP, the fixed YouTube-thumbnail web bug). - LOTUS_TODO: awaiting-verification row (16 providers) + deferred providers (Bandcamp / on.soundcloud / Vimeo event) under the feature backlog + the iframe onError gap. - LOTUS_TESTING: section Q (facade, TikTok, post self-resize + close, new Bluesky/Loom/Kick + toggle + cap). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -767,6 +767,65 @@ Redacted events display "This message has been deleted" along with the redaction
|
||||
|
||||
Generic (non-domain-specific) cards display a Google S2 favicon. Empty or unparseable preview responses are suppressed entirely rather than showing a blank card.
|
||||
|
||||
### Inline Media Embeds
|
||||
|
||||
Media links play/render **in place** instead of opening a browser tab. A pure
|
||||
resolver, `parseMediaEmbed(url, host)` in `src/app/utils/videoEmbed.ts`, maps a
|
||||
URL to `{ provider, kind, embedUrl }`; `MediaEmbedCard` / `TikTokEmbedCard` /
|
||||
`TwitterCard` in `UrlPreviewCard.tsx` render it. Four render `kind`s:
|
||||
|
||||
| kind | shape | providers |
|
||||
| ----------- | ------------------------ | ----------------------------------------------------------------- |
|
||||
| `landscape` | 16:9 video player | YouTube, Vimeo, Dailymotion, Streamable, Twitch, Loom, Kick (live) |
|
||||
| `portrait` | 9:16 video player | YouTube Shorts, TikTok |
|
||||
| `audio` | fixed-height audio player | Spotify, SoundCloud, Apple Music, Tidal |
|
||||
| `rich` | self-resizing post embed | X/Twitter, Instagram, Reddit, Bluesky |
|
||||
|
||||
**Privacy-friendly facade.** The tile first shows the homeserver's cached
|
||||
`og:image` thumbnail + a play button; the third-party `<iframe>` is only mounted
|
||||
on click, so nothing hits Google/Meta/etc. until the user opts in. A **Close**
|
||||
button collapses a playing embed back to the facade, and video players carry a
|
||||
Fullscreen control. Cookie-less/DNT variants are used where offered
|
||||
(`youtube-nocookie.com`, Vimeo `dnt=1`).
|
||||
|
||||
**Provider notes.**
|
||||
|
||||
- **TikTok** — short "copy-link" URLs (`vm.tiktok.com`, `tiktok.com/t/…`) carry no
|
||||
video id and the homeserver preview is bot-walled, so `TikTokEmbedCard`
|
||||
resolves the id client-side via TikTok's CORS-enabled **oEmbed** API on click
|
||||
(`AbortController`-guarded), then plays the `player/v1` embed.
|
||||
- **Reddit / Instagram / Bluesky / X** — post embeds self-size via `postMessage`;
|
||||
`useIframeAutoHeight` listens scoped to each provider's origin **and** our own
|
||||
iframe, parsing each provider's height shape (Instagram `MEASURE`, Reddit
|
||||
`resize.embed`, Twitter `twttr.private.resize`). `redd.it` short links resolve to
|
||||
the subreddit-less `embed.reddit.com/comments/{id}/` route.
|
||||
- **Vimeo** unlisted-video privacy hashes (`vimeo.com/{id}/{hash}`) and
|
||||
channel/group/album forms are parsed; **YouTube** handles `/watch`, `youtu.be`,
|
||||
`/embed`, `/live`, `/shorts`, and `m.`/`music.youtube.com`.
|
||||
|
||||
**Defense-in-depth.** Every embed iframe carries a `sandbox` that omits
|
||||
`allow-top-navigation` (so a compromised embed can't redirect the whole app —
|
||||
phishing guard) on top of the CSP `frame-src` allowlist. Previews are also capped
|
||||
at 6 per message.
|
||||
|
||||
**Setting.** `inlineMediaEmbeds` (Settings → General → "Inline Media Players",
|
||||
default **on**). Off → media links fall back to plain link tiles.
|
||||
|
||||
**Latent web bug fixed along the way.** YouTube thumbnails now come from the
|
||||
homeserver `og:image` instead of `img.youtube.com` — which was silently broken on
|
||||
the web build (nginx `img-src` has no YouTube host) — removing a pre-click Google
|
||||
request as a bonus.
|
||||
|
||||
**CSP.** Desktop Tauri `frame-src` (`cinny-desktop` `tauri.conf.json`) and the web
|
||||
nginx `frame-src` allowlist enumerate every embed host (youtube-nocookie,
|
||||
player.vimeo, geo.dailymotion, streamable, player/clips.twitch, open.spotify,
|
||||
w.soundcloud, embed.music.apple, embed.tidal, www.tiktok + connect-src for its
|
||||
oEmbed, platform.twitter, www.instagram, embed.reddit, embed.bsky.app, www.loom,
|
||||
player.kick).
|
||||
|
||||
**Files:** `src/app/utils/videoEmbed.ts` (resolver + parsers, unit-tested),
|
||||
`src/app/components/url-preview/{UrlPreviewCard,UrlPreview.css}.tsx`.
|
||||
|
||||
### Poll Creation
|
||||
|
||||
- `PollCreator.tsx` creates stable `m.poll.start` events
|
||||
|
||||
@@ -662,6 +662,43 @@ Run the axe DevTools extension (or Lighthouse → Accessibility) on a room view,
|
||||
|
||||
---
|
||||
|
||||
## Q. Inline Media Embeds — video / audio / post players (needs the web deploy live)
|
||||
|
||||
The whole feature is behind **Settings → General → "Inline Media Players"** (default **on**). Everything loads from the homeserver's cached thumbnail first; the third-party player only mounts on **Play**. Test on the **web** build first, then re-check the video ones on **desktop (Tauri)** since the CSP differs. On any failure, grab the **browser console** (F12) — a blocked embed shows as a CSP `frame-src` violation naming the host.
|
||||
|
||||
### Q1. Facade + one of each kind plays in place
|
||||
|
||||
Paste each of these into a room and confirm a media tile (not a plain link) with a thumbnail + play button, and that clicking Play mounts the player **inline**:
|
||||
|
||||
- **16:9 video:** a YouTube `watch` link, a Vimeo link, a Dailymotion link, a Streamable link, a Twitch VOD/clip, a Loom `share` link.
|
||||
- **9:16 portrait:** a YouTube **Shorts** link (renders tall, not letterboxed).
|
||||
- **Audio player:** a Spotify track, a SoundCloud track, an Apple Music album, a Tidal album/track.
|
||||
- **Post embed:** an X/Twitter post, an Instagram post, a Reddit post.
|
||||
|
||||
**Expected:** ✅ tile shows the thumbnail; **no** request to the third party until you press Play (check DevTools → Network); the player then plays inline. ❌ tell me any that stay a plain link, show a blank frame, or hit the network before you click.
|
||||
|
||||
### Q2. TikTok (the tricky one) + portrait fill
|
||||
|
||||
1. Paste a **full** TikTok URL and a **short** copy-link (`vm.tiktok.com/…` or `tiktok.com/t/…`).
|
||||
2. Press Play on each.
|
||||
|
||||
**Expected:** both resolve to a clean **9:16** player that **fills the box** (no big empty band on the right). The short link shows a brief spinner while it resolves via oEmbed, then plays. ❌ tell me if a short link shows only the TikTok logo/♫ and never a play button, or if the player has dead space beside it.
|
||||
|
||||
### Q3. Post self-resize + Close / Fullscreen controls
|
||||
|
||||
1. Play a **Reddit**, **Instagram**, and **X/Twitter** post embed.
|
||||
2. Watch the card height as the embed loads.
|
||||
|
||||
**Expected:** the card **grows to fit** the post (no clipped/scrollbarless content, no giant empty box). A **Close** button (✕) collapses the player back to the thumbnail; video players also show a **⛶ Fullscreen** control that works. Keyboard: Tab to the play button → it shows a visible **focus ring**.
|
||||
|
||||
### Q4. New providers (unverified) + the toggle + the cap
|
||||
|
||||
- **Bluesky / Loom / Kick** — these are freshly added and unverified live. Paste a `bsky.app/profile/…/post/…`, a `loom.com/share/…`, and a live `kick.com/{channel}` link. ✅ good if each plays/renders inline; ❌ if any is a broken frame (for **Bluesky** especially, note whether a **handle** URL resolves or only a DID one does — grab the console).
|
||||
- **Toggle off:** Settings → General → **Inline Media Players** off → every media link reverts to a plain link tile (no player).
|
||||
- **Cap:** paste a message with **8+** media links → at most **6** preview cards render (the rest are suppressed), and the page stays responsive.
|
||||
|
||||
---
|
||||
|
||||
## Priority if you're short on time
|
||||
|
||||
1. **O1 + O2** (threads + per-thread notifications) — the largest new surface; the main-timeline change is user-visible.
|
||||
|
||||
@@ -57,6 +57,7 @@ Built and gate-green; verify per [LOTUS_TESTING.md](./LOTUS_TESTING.md), then gr
|
||||
| Desktop proactive update notifications (P5-40) | J1 |
|
||||
| OIDC/SSO login (P4-6, needs an MSC3861 server — pick mozilla.org on login) | OIDC |
|
||||
| Windows native WinRT toast quick-reply / click-to-open (D6, AUMID) | rich-toast (§backlog) |
|
||||
| Inline media embeds (16 providers: video/audio/post + click-to-play facade) | Q1 / Q2 / Q3 / Q4 |
|
||||
|
||||
---
|
||||
|
||||
@@ -172,6 +173,16 @@ Phase 1 shipped: `io.lotus.set_deafen` (LiveKit-source deafen/screenshare-audio-
|
||||
|
||||
Comprehensive audit of all LOTUS_FEATURES.md features for mobile PWA usability + responsiveness. Method: 44px touch targets, no horizontal overflow, full-screen modals/drawers on mobile, composer not obscured by keyboard.
|
||||
|
||||
### [ ] Inline media embeds — remaining providers (LOW PRIORITY)
|
||||
|
||||
The inline embed system (`videoEmbed.ts`) covers 16 providers; three more were **deliberately deferred** (verified against 2026 docs by review agents):
|
||||
|
||||
- **Bandcamp** (highest-value audio add) — needs an **oEmbed** round-trip: the player URL requires numeric `album`/`track` item ids that aren't in the page URL (`bandcamp.com/oembed` is the resolver; mirror the `TikTokEmbedCard` on-click oEmbed pattern). CSP `frame-src`: `bandcamp.com`. Classify `kind: 'audio'`.
|
||||
- **SoundCloud `on.soundcloud.com` short links** — the `w.soundcloud` widget resolver does **not** follow the redirect; needs the same on-click oEmbed resolve (`soundcloud.com/oembed`, CORS-enabled) to get the canonical URL. (Canonical `soundcloud.com/{user}/{track}` links already work.)
|
||||
- **Vimeo `event/{id}` (live events) + `ondemand/…`** — event embed host is `vimeo.com` (**not** `player.vimeo.com`, so it needs a new CSP `frame-src` host); on-demand is paywalled and doesn't embed for non-purchasers. Low ROI — only do the event case if `vimeo.com` is widened for another reason.
|
||||
|
||||
Also open (from the quality review): a real `onError`/error-state fallback for iframes that fail to load (deleted post / region lock / X login-wall) — cross-origin frames don't fire `onError` reliably, so this needs a load-timeout heuristic; the Close button + badge link are the current escape hatch.
|
||||
|
||||
### Deferred / dropped (decided — kept for context)
|
||||
|
||||
- **[DEFERRED] P5-51** Federated "Identity Contexts" (session isolation) — multi-sprint, touches auth/crypto/storage core; smaller intermediate step = plain multi-account switch. **[DROPPED] P5-52** per-room sync governor — js-sdk can't truly per-room filter `/sync`; only a cosmetic hide. **[DEFERRED] P5-53** local scripting plugin — prefer a declarative automation-rules feature (no arbitrary code). **[DEFERRED] Audit-3** profile banner — MSC4427 open/unmerged; revisit on merge. **[WON'T FIX] P5-50** Windows HW media pipeline (WebRTC decode lives in WebView2; not injectable). **[MOVED] P5-9** LFG → LotusBot `!lfg`.
|
||||
|
||||
@@ -41,6 +41,7 @@ The Lotus Chat logo (`public/res/Lotus.png`) is a derivative work based on the o
|
||||
- Deleted messages show a placeholder instead of disappearing
|
||||
- Code blocks highlight syntax for JS/TS, Python, and Rust
|
||||
- Rich link preview cards for YouTube, GitHub, Twitter/X, Reddit, Spotify, Twitch, Steam, Wikipedia, Discord, npm, Stack Overflow, and IMDb
|
||||
- Inline media embeds — play videos and posts in place instead of opening a browser tab: YouTube/Shorts, Vimeo, Dailymotion, Streamable, Twitch, Loom, and Kick as video players; TikTok, X/Twitter, Instagram, Reddit, and Bluesky as inline posts; Spotify, SoundCloud, Apple Music, and Tidal as a built-in audio player. A privacy-friendly facade shows the homeserver's cached thumbnail and only loads the third-party player when you press play. Toggle at Settings → General → "Inline Media Players" (on by default)
|
||||
|
||||
### Calls & Voice
|
||||
|
||||
|
||||
Reference in New Issue
Block a user