LotusGuild/cinny
3
0
Fork 0
Code Issues 27 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

Inline media embeds: remaining providers (low priority) #200

New Issue
Closed
opened 2026-09-17 23:24:15 -04:00 by jared · 5 comments
jared commented 2026-09-17 23:24:15 -04:00
Owner
Copy Link
Copy Source

Migrated from LOTUS_TODO.md on 2026-09-17 (the file is now reference-only).

[ ] Inline media embeds — remaining providers (LOW PRIORITY)

The inline embed system (videoEmbed.ts) covers 18 providers (16 + Mixcloud/Deezer); three more were deliberately deferred (verified against 2026 docs by review agents):

  • Bandcamp (highest-value audio add) — needs an oEmbed round-trip: the player URL requires numeric album/track item ids that aren't in the page URL (bandcamp.com/oembed is the resolver; mirror the TikTokEmbedCard on-click oEmbed pattern). CSP frame-src: bandcamp.com. Classify kind: 'audio'.
  • SoundCloud on.soundcloud.com short links — the w.soundcloud widget resolver does not follow the redirect; needs the same on-click oEmbed resolve (soundcloud.com/oembed, CORS-enabled) to get the canonical URL. (Canonical soundcloud.com/{user}/{track} links already work.)
  • Vimeo event/{id} (live events) + ondemand/… — event embed host is vimeo.com (not player.vimeo.com, so it needs a new CSP frame-src host); on-demand is paywalled and doesn't embed for non-purchasers. Low ROI — only do the event case if vimeo.com is widened for another reason.

Also open (from the quality review): a real onError/error-state fallback for iframes that fail to load (deleted post / region lock / X login-wall) — cross-origin frames don't fire onError reliably, so this needs a load-timeout heuristic; the Close button + badge link are the current escape hatch.

✅ Steam detailed embed (2026-07, 2-agent review) — ef82650c. store.steampowered.com content URLs get rich cards: app pages → OG capsule header + click-to-play facade → Steam's official /widget/{id} store iframe (live region-aware price / discount % / Buy on Steam, gated by inlineMediaEmbeds); news/announcement → banner + headline + body-preview card; bundle/sub/dlc → OG store card. getSteamTarget/steamWidgetEmbedUrl in videoEmbed.ts (+tests). Grounded in prod CSP (frame-src https: allows the widget with no infra change; images via homeserver mxc; NO client-side Steam API — connect-src + Steam CORS both block it, which is the honest ceiling: no review scores/genres/screenshots client-side). Needs on-device QA: the live widget iframe height/fit (can't render headlessly) — verify the price/Buy stay visible on desktop-wide and phone.

✅ GIF previews now animate + Mixcloud/Deezer embeds (2026-07, 2-agent review) — 4154cae5. Reported live: a media.giphy.com link "shows the gif's image but doesn't play it." Root cause: Synapse's /thumbnail endpoint flattens animated GIFs to a still first frame, and every preview image went through it. GifCard (Giphy/Tenor) + the generic OG card now request the original via /download (mxcUrlToHttp with no width/height) when the preview is a GIF (og:image:type === 'image/gif' or a .gif pathname). Guarded: shouldServeGifOriginal() keeps the frozen thumbnail past a 10 MB matrix:image:size cap, and the generic card's eager <img> gained the loading="lazy" it was the only preview image missing. Also added Mixcloud + Deezer audio embeds, and fixed Deezer podcasts (they live at /show/<id>, not /podcast/<id> — the latter 404s on Deezer's own oEmbed; verified against the live API). Needs on-device QA: confirm a large GIF still animates and doesn't stall the timeline.

✅ Embed bug hunt (2026-07, 3 survey agents + 2-agent review) — f2673eff. Core posture verified sound (iframe sandbox, useIframeAutoHeight postMessage origin+source trust, no XSS/dangerouslySetInnerHTML, rel="noreferrer" on all 21 links, oEmbed no-SSRF, the whole facade→iframe/abort/observer lifecycle). Fixed: Twitch/Kick/SoundCloud/Streamable reserved-path over-match (utility pages rendered as broken players), Vimeo hash over-capture ([0-9a-f]{6,}), Spotify/Steam/Discord/IMDb og:image now via mxcUrlToHttp (was a broken raw mxc:// <img> + a pre-click 3p-request facade bypass), wide class follows the og:url-resolved embed, Twitter host alignment (mobile.twitter.com//statuses/), URL de-dupe.

Deferred / surfaced from the hunt (not fixed — decide before doing):

  • Security-vs-functionality tradeoff (needs a call): drop allow-popups-to-escape-sandbox and/or clipboard-write from EMBED_SANDBOX/allow= on embed iframes — real hardening against a compromised provider (phishing popup / clipboard hijack), but risks breaking a legit provider popup/copy on the trusted major providers we embed. Low marginal value; not shipped blindly.
  • Defense-in-depth: encodeURIComponent the Bluesky authority + Apple Music path/search interpolated into the embed src (not currently exploitable — host is fixed and value comes from URL.pathname; React escapes the attribute).
  • Out of embed scope (real, low-sev): LotusDenoiseFeature (ClientNonUIFeatures.tsx) has a window message listener with no origin/source check → any frame/window can post {type:'lotus-denoise-status', error} and pop a forged "System" toast (text only, no XSS). Validate event.source.
  • Lifecycle Lows (cosmetic/latent): a re-fetch flips a playing embed back to the spinner (latent — url is keyed); auto-height retained across close→reopen; extractEmbedHeight generic .height fallback accepts any allowed-origin message; TweetEmbed theme is a one-time matchMedia snapshot (no live theme switch); host-normalization gaps (vt.tiktok.com misses StackOnMobile, m.instagram.com, www.youtu.be).
_Migrated from `LOTUS_TODO.md` on 2026-09-17 (the file is now reference-only)._ ### [ ] Inline media embeds — remaining providers (LOW PRIORITY) The inline embed system (`videoEmbed.ts`) covers 18 providers (16 + Mixcloud/Deezer); three more were **deliberately deferred** (verified against 2026 docs by review agents): - **Bandcamp** (highest-value audio add) — needs an **oEmbed** round-trip: the player URL requires numeric `album`/`track` item ids that aren't in the page URL (`bandcamp.com/oembed` is the resolver; mirror the `TikTokEmbedCard` on-click oEmbed pattern). CSP `frame-src`: `bandcamp.com`. Classify `kind: 'audio'`. - **SoundCloud `on.soundcloud.com` short links** — the `w.soundcloud` widget resolver does **not** follow the redirect; needs the same on-click oEmbed resolve (`soundcloud.com/oembed`, CORS-enabled) to get the canonical URL. (Canonical `soundcloud.com/{user}/{track}` links already work.) - **Vimeo `event/{id}` (live events) + `ondemand/…`** — event embed host is `vimeo.com` (**not** `player.vimeo.com`, so it needs a new CSP `frame-src` host); on-demand is paywalled and doesn't embed for non-purchasers. Low ROI — only do the event case if `vimeo.com` is widened for another reason. Also open (from the quality review): a real `onError`/error-state fallback for iframes that fail to load (deleted post / region lock / X login-wall) — cross-origin frames don't fire `onError` reliably, so this needs a load-timeout heuristic; the Close button + badge link are the current escape hatch. **✅ Steam detailed embed (2026-07, 2-agent review) — `ef82650c`.** `store.steampowered.com` content URLs get rich cards: **app** pages → OG capsule header + click-to-play facade → Steam's official `/widget/{id}` store iframe (live region-aware price / discount % / Buy on Steam, gated by `inlineMediaEmbeds`); **news/announcement** → banner + headline + body-preview card; **bundle/sub/dlc** → OG store card. `getSteamTarget`/`steamWidgetEmbedUrl` in `videoEmbed.ts` (+tests). Grounded in prod CSP (`frame-src https:` allows the widget with no infra change; images via homeserver `mxc`; NO client-side Steam API — `connect-src` + Steam CORS both block it, which is the honest ceiling: no review scores/genres/screenshots client-side). **Needs on-device QA:** the live widget iframe height/fit (can't render headlessly) — verify the price/Buy stay visible on desktop-wide and phone. **✅ GIF previews now animate + Mixcloud/Deezer embeds (2026-07, 2-agent review) — `4154cae5`.** Reported live: a `media.giphy.com` link "shows the gif's image but doesn't play it." Root cause: **Synapse's `/thumbnail` endpoint flattens animated GIFs to a still first frame**, and every preview image went through it. `GifCard` (Giphy/Tenor) + the generic OG card now request the **original** via `/download` (`mxcUrlToHttp` with no width/height) when the preview is a GIF (`og:image:type === 'image/gif'` or a `.gif` pathname). Guarded: `shouldServeGifOriginal()` keeps the frozen thumbnail past a **10 MB** `matrix:image:size` cap, and the generic card's eager `<img>` gained the `loading="lazy"` it was the only preview image missing. Also added **Mixcloud + Deezer** audio embeds, and fixed Deezer podcasts (they live at `/show/<id>`, **not** `/podcast/<id>` — the latter 404s on Deezer's own oEmbed; verified against the live API). **Needs on-device QA:** confirm a large GIF still animates and doesn't stall the timeline. **✅ Embed bug hunt (2026-07, 3 survey agents + 2-agent review) — `f2673eff`.** Core posture verified **sound** (iframe sandbox, `useIframeAutoHeight` postMessage origin+source trust, no XSS/`dangerouslySetInnerHTML`, `rel="noreferrer"` on all 21 links, oEmbed no-SSRF, the whole facade→iframe/abort/observer lifecycle). Fixed: Twitch/Kick/SoundCloud/Streamable reserved-path over-match (utility pages rendered as broken players), Vimeo hash over-capture (`[0-9a-f]{6,}`), Spotify/Steam/Discord/IMDb `og:image` now via `mxcUrlToHttp` (was a broken raw `mxc://` `<img>` + a pre-click 3p-request facade bypass), `wide` class follows the og:url-resolved embed, Twitter host alignment (`mobile.twitter.com`/`/statuses/`), URL de-dupe. **Deferred / surfaced from the hunt (not fixed — decide before doing):** - **Security-vs-functionality tradeoff (needs a call):** drop `allow-popups-to-escape-sandbox` and/or `clipboard-write` from `EMBED_SANDBOX`/`allow=` on embed iframes — real hardening against a _compromised_ provider (phishing popup / clipboard hijack), but risks breaking a legit provider popup/copy on the trusted major providers we embed. Low marginal value; not shipped blindly. - **Defense-in-depth:** `encodeURIComponent` the Bluesky authority + Apple Music path/search interpolated into the embed `src` (not currently exploitable — host is fixed and value comes from `URL.pathname`; React escapes the attribute). - **Out of embed scope (real, low-sev):** `LotusDenoiseFeature` (`ClientNonUIFeatures.tsx`) has a `window` `message` listener with **no origin/source check** → any frame/window can post `{type:'lotus-denoise-status', error}` and pop a forged **"System"** toast (text only, no XSS). Validate `event.source`. - **Lifecycle Lows (cosmetic/latent):** a re-fetch flips a playing embed back to the spinner (latent — url is keyed); auto-height retained across close→reopen; `extractEmbedHeight` generic `.height` fallback accepts any allowed-origin message; `TweetEmbed` theme is a one-time `matchMedia` snapshot (no live theme switch); host-normalization gaps (`vt.tiktok.com` misses `StackOnMobile`, `m.instagram.com`, `www.youtu.be`).
embed-stalled.png
167 KiB
embed-final.png
162 KiB
embed-stalled.png embed-final.png
jared added this to the Features 2026-Q4 milestone 2026-09-17 23:24:15 -04:00
jared added the enhancementpriority: lowarea: media labels 2026-09-17 23:24:15 -04:00
jared commented 2026-09-23 22:08:02 -04:00
Author
Owner
Copy Link
Copy Source

Bandcamp: not feasible as scoped. Checked 2026-09-23

The plan above assumed a bandcamp.com/oembed resolver. Every way to get the numeric track/album id that EmbeddedPlayer needs is closed:

  • No oEmbed. https://bandcamp.com/oembed?format=json&url=… returns a 404 HTML page.
  • Client-side fetch: Bandcamp sends no Access-Control-Allow-Origin, so the browser can't read the page. It also 403s browser-like User-Agents from our IP (162.192.14.139).
  • Synapse link preview: it can fetch the page (200 from LXC 151 with Synapse's UA), and the page does carry the id, e.g. twitter:player = https://bandcamp.com/EmbeddedPlayer/v=2/track=2026463514/…. But Bandcamp writes it as <meta property="twitter:player">, and Synapse (preview_html.py) only reads property="og:*" and name="twitter:*". There's no og:video URL, only og:video:type/width/height. So the id never reaches the client.

Options if Bandcamp is still wanted:

  1. Desktop only: fetch the page through the Tauri side (no CORS) and read twitter:player. Web users keep the plain link card.
  2. Tiny server helper: an endpoint on LXC 151 that returns twitter:player for a Bandcamp URL. It would need a CSP connect-src entry.
  3. Upstream: a Synapse patch to also read property="twitter:*", which would help every client.

I'd leave Bandcamp parked unless one of those is worth it to you. The rest of this issue (SoundCloud short links, the load-timeout fallback) is unaffected. SoundCloud's oEmbed does send access-control-allow-origin: *, so the on-click resolve for on.soundcloud.com links is feasible client-side.

## Bandcamp: not feasible as scoped. Checked 2026-09-23 The plan above assumed a `bandcamp.com/oembed` resolver. Every way to get the numeric track/album id that `EmbeddedPlayer` needs is closed: - **No oEmbed.** `https://bandcamp.com/oembed?format=json&url=…` returns a 404 HTML page. - **Client-side fetch:** Bandcamp sends no `Access-Control-Allow-Origin`, so the browser can't read the page. It also 403s browser-like User-Agents from our IP (`162.192.14.139`). - **Synapse link preview:** it can fetch the page (200 from LXC 151 with Synapse's UA), and the page does carry the id, e.g. `twitter:player = https://bandcamp.com/EmbeddedPlayer/v=2/track=2026463514/…`. But Bandcamp writes it as `<meta property="twitter:player">`, and Synapse (`preview_html.py`) only reads `property="og:*"` and `name="twitter:*"`. There's no `og:video` URL, only `og:video:type/width/height`. So the id never reaches the client. Options if Bandcamp is still wanted: 1. **Desktop only:** fetch the page through the Tauri side (no CORS) and read `twitter:player`. Web users keep the plain link card. 2. **Tiny server helper:** an endpoint on LXC 151 that returns `twitter:player` for a Bandcamp URL. It would need a CSP `connect-src` entry. 3. **Upstream:** a Synapse patch to also read `property="twitter:*"`, which would help every client. I'd leave Bandcamp parked unless one of those is worth it to you. The rest of this issue (SoundCloud short links, the load-timeout fallback) is unaffected. SoundCloud's oEmbed does send `access-control-allow-origin: *`, so the on-click resolve for `on.soundcloud.com` links is feasible client-side.
jared commented 2026-09-23 22:09:47 -04:00
Author
Owner
Copy Link
Copy Source

SoundCloud short links: already work, no change needed

Tested with a real short link, https://on.soundcloud.com/2iQZMwo9IQ8wLY3vf9. Synapse follows the 302 while building the preview (og:url = https://soundcloud.com/ilaytsa/crashout), and UrlPreviewCard's existing og:url fallback re-parses that into the SoundCloud embed. In the app the card shows "Play: crashout", and clicking it loads the w.soundcloud.com player.

Short links to a profile (e.g. on.soundcloud.com/QAYE7paeKiXpQoaSA → /noit-avonni) stay a plain card, which is the same as canonical profile links, by design. I've corrected the stale code comment in videoEmbed.ts (76929a87).

What's left here: Bandcamp (see above: parked, needs a server helper or a Synapse patch), Vimeo events (needs vimeo.com in CSP frame-src), and the iframe load-timeout fallback.

## SoundCloud short links: already work, no change needed Tested with a real short link, `https://on.soundcloud.com/2iQZMwo9IQ8wLY3vf9`. Synapse follows the 302 while building the preview (`og:url = https://soundcloud.com/ilaytsa/crashout`), and `UrlPreviewCard`'s existing `og:url` fallback re-parses that into the SoundCloud embed. In the app the card shows "Play: crashout", and clicking it loads the `w.soundcloud.com` player. Short links to a **profile** (e.g. `on.soundcloud.com/QAYE7paeKiXpQoaSA` → `/noit-avonni`) stay a plain card, which is the same as canonical profile links, by design. I've corrected the stale code comment in `videoEmbed.ts` (`76929a87`). What's left here: Bandcamp (see above: parked, needs a server helper or a Synapse patch), Vimeo events (needs `vimeo.com` in CSP `frame-src`), and the iframe load-timeout fallback.
jared commented 2026-09-26 11:56:01 -04:00
Author
Owner
Copy Link
Copy Source

Iframe load-failure fallback: done in ba5b1ffe

Before choosing signals, I loaded good and broken embeds side by side and recorded what each provider actually does.

case what the user saw signal to the app
X, missing post (id=1) empty box postMessage twttr.private.no_results
X, real post (id=20) the post initialized → resize → rendered
Instagram, bad id its own "post may have been removed" page load
Bluesky, bad id its own "Post not found" page load
Reddit "blocked by network security" (Reddit blocks this server's IP, so not testable from here) load
request refused browser error page load still fires
network hang empty box no load, ever

What shipped

  • Deleted, private or suspended X posts: the embed swaps to "This post isn't available. It may have been deleted, or the account is private or suspended.", with an Open on X button.
  • Hung embeds (media players, Instagram/Reddit/Bluesky posts, TikTok, the Steam widget, X): after 20 s with no load, the player shows "This embed is taking too long to load." with Retry (remounts the iframe) and Open on <site>. If load arrives late, the notice clears itself.
  • Deliberately not done: treating a missing height message as a failure. Healthy embeds don't always send one, so it would raise false alarms. Instagram and Bluesky already show their own error page. A refused request can't be told apart from success, because load still fires.

Verified in the app (local Synapse, previews served by the test):

  • The missing X post showed the unavailable card within 6 s.
  • The real X post never showed a notice, even after 45 s.
  • A YouTube embed forced to hang showed the notice at 20 s. Retry loaded the player and the notice cleared.
  • Unit tests cover the no_results parser. The full unit suite (1247) and e2e (28) pass.

stalled
after retry

Still open on this issue:

  • Bandcamp: parked. It needs a server helper or a Synapse patch; see above.
  • Vimeo live events: need vimeo.com added to the CSP frame-src.
  • Items from the embed bug hunt, which need a decision first: dropping allow-popups-to-escape-sandbox/clipboard-write; the LotusDenoise message listener has no origin check; the host-normalisation gaps.
## Iframe load-failure fallback: done in `ba5b1ffe` Before choosing signals, I loaded good and broken embeds side by side and recorded what each provider actually does. | case | what the user saw | signal to the app | |---|---|---| | X, missing post (`id=1`) | **empty box** | postMessage `twttr.private.no_results` | | X, real post (`id=20`) | the post | `initialized` → `resize` → `rendered` | | Instagram, bad id | its own "post may have been removed" page | `load` | | Bluesky, bad id | its own "Post not found" page | `load` | | Reddit | "blocked by network security" (Reddit blocks this server's IP, so not testable from here) | `load` | | request refused | browser error page | **`load` still fires** | | network hang | empty box | **no `load`, ever** | **What shipped** - **Deleted, private or suspended X posts:** the embed swaps to "This post isn't available. It may have been deleted, or the account is private or suspended.", with an **Open on X** button. - **Hung embeds** (media players, Instagram/Reddit/Bluesky posts, TikTok, the Steam widget, X): after 20 s with no `load`, the player shows "This embed is taking too long to load." with **Retry** (remounts the iframe) and **Open on &lt;site&gt;**. If `load` arrives late, the notice clears itself. - **Deliberately not done:** treating a missing height message as a failure. Healthy embeds don't always send one, so it would raise false alarms. Instagram and Bluesky already show their own error page. A refused request can't be told apart from success, because `load` still fires. **Verified in the app** (local Synapse, previews served by the test): - The missing X post showed the unavailable card within 6 s. - The real X post never showed a notice, even after 45 s. - A YouTube embed forced to hang showed the notice at 20 s. Retry loaded the player and the notice cleared. - Unit tests cover the `no_results` parser. The full unit suite (1247) and e2e (28) pass. ![stalled](https://code.lotusguild.org/attachments/fa5a4752-955a-4684-8f5c-d591b9dfaa71) ![after retry](https://code.lotusguild.org/attachments/cda07828-9079-4a1e-a8f8-4371adeb389f) **Still open on this issue:** - Bandcamp: parked. It needs a server helper or a Synapse patch; see above. - Vimeo live events: need `vimeo.com` added to the CSP `frame-src`. - Items from the embed bug hunt, which need a decision first: dropping `allow-popups-to-escape-sandbox`/`clipboard-write`; the `LotusDenoise` message listener has no origin check; the host-normalisation gaps.
jared commented 2026-10-20 23:32:03 -04:00
Author
Owner
Copy Link
Copy Source

Owner decision (2026-10-21): Bandcamp stays parked. Its links keep the normal preview card, and no server helper or Synapse change will be added.

  • Vimeo live events: done in #444.
  • SoundCloud short links: already worked.
  • Iframe load fallback: done in ba5b1ffe.

Closing once #444 merges.

Owner decision (2026-10-21): **Bandcamp stays parked.** Its links keep the normal preview card, and no server helper or Synapse change will be added. - **Vimeo live events:** done in #444. - **SoundCloud short links:** already worked. - **Iframe load fallback:** done in ba5b1ffe. Closing once #444 merges.
jared commented 2026-10-22 20:31:50 -04:00
Author
Owner
Copy Link
Copy Source

Vimeo live events shipped in #444. Bandcamp is parked by the owner (see above). Closing.

Vimeo live events shipped in #444. Bandcamp is parked by the owner (see above). Closing.
jared closed this issue 2026-10-22 20:31:50 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
a11y
Accessibility: keyboard, screen reader, contrast, motion
 area: appearance
Client area: appearance
 area: auth-session
Client area: auth-session
 area: build-ci
Client area: build-ci
 area: calls
Client area: calls
 area: desktop
Client area: desktop
 area: media
Client area: media
 area: messaging
Client area: messaging
 area: mobile
Client area: mobile
 area: moderation
Client area: moderation
 area: navigation
Client area: navigation
 area: notifications
Client area: notifications
 area: settings
Client area: settings
 area: threads
Client area: threads
 bug
Something is not working
 dependencies
Third-party package versions and advisories
 docs
README / LOTUS_* docs wrong or missing
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 needs-human-review
performance
Re-render storms, leaks, heavy work on hot paths
 planning
priority: critical
Data loss, security hole, or crash on a main path
 priority: high
Broken feature or serious usability problem
 priority: low
Minor issue or polish
 priority: medium
Wrong behaviour in an edge case or notable degradation
 qa
Manual QA: shipped, needs a human in a real environment
 question
More information is needed
 research
security
XSS, unsafe URLs, data leaks, auth/session
 tech-debt
Code health, dead code, fragile patterns
 ux
Usability or visual inconsistency
 wontfix
This won't be fixed
No labels area: media enhancement priority: low
Milestone
No items
No Milestone Features 2026-Q4
Projects
Clear projects
No projects
Assignees
Clear assignees
cole (Cole Keller) jared (Jared Vititoe) nathan (Nathan Vititoe)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: LotusGuild/cinny#200