fix(gallery): paginate media, activity log and export on detached timeline sets — never the live timeline (#163)
RoomTimeline renders a numeric index window into the live timeline's event arrays; SDK back-pagination prepends, so any side panel calling paginateEventTimeline(room.getLiveTimeline()) shifted the visible messages into the past on the next render and broke at-bottom tracking. New utils/detachedTimeline.ts builds a timeline set that mirrors the already-loaded history and paginates independently: a room-registered filtered set (server-side contains_url / types filter) when the filter is usable, else a private EventTimelineSet seeded from the live timeline. useRoomMediaTimeline wraps it for the gallery (live events + redactions handled); RoomActivityLog uses a type filter (safe in encrypted rooms); ExportRoomHistory pages a private set so a full export no longer parks thousands of events in the live timeline. Verified with Playwright against a local Synapse in a 400-message plain room and a 200-message encrypted room: timeline stays at the bottom through gallery pages, activity load-more and a full export; live messages keep auto-scrolling; all media found in both rooms. Also adds scripts/dev-homeserver.sh + scripts/dev-seed.py (local throwaway Synapse for driving the real UI) and documents them. Closes #163 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
This commit is contained in:
@@ -10,3 +10,7 @@ public/decorations/
|
||||
# Playwright (npm run test:e2e)
|
||||
playwright-report/
|
||||
test-results/
|
||||
|
||||
# local dev homeserver (scripts/dev-homeserver.sh)
|
||||
.dev-homeserver/
|
||||
__pycache__/
|
||||
|
||||
+1
-1
@@ -1193,7 +1193,7 @@ When Lotus Chat is installed as a PWA (Android Chrome, desktop Chrome/Edge), the
|
||||
- **Files** — name/size/sender rows with download
|
||||
- **Jump to message** — a "Go to message" action on file rows, audio rows, and in the lightbox navigates the timeline to the source event (`useRoomNavigate`) and closes the drawer
|
||||
- Encrypted media is decrypted client-side on demand (no lock placeholder); download works for all types
|
||||
- **Auto-pagination** — an `IntersectionObserver` sentinel calls `mx.paginateEventTimeline()` to pull older media as you scroll (manual retry on error)
|
||||
- **Auto-pagination** — an `IntersectionObserver` sentinel pulls older media as you scroll (manual retry on error). Since Gitea #163 this pages through the gallery's **own timeline set** (`useRoomMediaTimeline` → `utils/detachedTimeline.ts`), never the room's live timeline: unencrypted rooms use a server-side `contains_url` filter (a page is 100 media events, not 100 events), encrypted rooms page raw history into a private set and filter after decrypting. The message list behind the drawer no longer jumps into the past; the Activity log and history Export use the same helper.
|
||||
|
||||
### Knock-to-Join
|
||||
|
||||
|
||||
@@ -11,6 +11,16 @@ This file keeps what a contributor needs to run and extend the **automated** cov
|
||||
|
||||
---
|
||||
|
||||
## Local dev environment — drive the real UI against a throwaway homeserver
|
||||
|
||||
```
|
||||
scripts/dev-homeserver.sh start # Synapse in .dev-homeserver/ (venv, SQLite), open registration, no rate limits, :8008
|
||||
python3 scripts/dev-seed.py 400 # alice + bob, "Busy Room": 400 messages, an image every 10th
|
||||
npm start # Vite on :5173
|
||||
```
|
||||
|
||||
Log in at `http://127.0.0.1:5173/login/http%3A%2F%2Flocalhost%3A8008/` as `alice` / `password123` (bob is the second participant; both can also be driven over the client API with their tokens). Playwright is installed (`npm run test:e2e:install`), so a scripted reproduction is `node` + `chromium.launch()` against `:5173` — this is how Gitea #163 was reproduced and its fix verified in both plain and encrypted rooms. `scripts/dev-homeserver.sh reset` wipes the database; `stop` shuts it down.
|
||||
|
||||
## Automated coverage map — what the unit tests already pin (2026-07)
|
||||
|
||||
**Read this before working a `qa` issue.** Much of the _logic_ the manual checks were written to catch is now locked by deterministic unit tests (`npm test`, 920+ cases, green in CI). Unit tests do **not** prove visual rendering, real-call behavior, the desktop build, E2EE, or cross-device sync — those still need a human. But where a decision is pure logic, you can **trust the test and spend your manual time on the human-only part**. For each row below, the middle column is "don't bother re-deriving this by hand"; the right column is "this is what your manual pass is actually for."
|
||||
|
||||
Executable
+75
@@ -0,0 +1,75 @@
|
||||
#!/usr/bin/env bash
|
||||
# Local throwaway Synapse for driving the real UI (Playwright / a browser)
|
||||
# against a homeserver you control — open registration, no rate limits,
|
||||
# SQLite, media served. Everything lives in .dev-homeserver/ (gitignored).
|
||||
#
|
||||
# scripts/dev-homeserver.sh start # install (first run) + start on :8008
|
||||
# scripts/dev-homeserver.sh stop
|
||||
# scripts/dev-homeserver.sh reset # wipe the database and media
|
||||
# python3 scripts/dev-seed.py 400 # alice/bob + "Busy Room" with images
|
||||
#
|
||||
# Then `npm start` and log in at http://127.0.0.1:5173/login/http%3A%2F%2Flocalhost%3A8008/
|
||||
# as alice / password123.
|
||||
set -euo pipefail
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
DIR="$ROOT/.dev-homeserver"
|
||||
VENV="$DIR/venv"
|
||||
CFG="$DIR/homeserver.yaml"
|
||||
PIDFILE="$DIR/synapse.pid"
|
||||
|
||||
install() {
|
||||
mkdir -p "$DIR"
|
||||
if [ ! -x "$VENV/bin/python" ]; then
|
||||
python3 -m venv --without-pip "$VENV"
|
||||
curl -sS https://bootstrap.pypa.io/get-pip.py -o "$DIR/get-pip.py"
|
||||
"$VENV/bin/python" "$DIR/get-pip.py" -q
|
||||
"$VENV/bin/pip" install -q matrix-synapse
|
||||
fi
|
||||
if [ ! -f "$CFG" ]; then
|
||||
(cd "$DIR" && "$VENV/bin/python" -m synapse.app.homeserver \
|
||||
--server-name localhost --config-path homeserver.yaml --generate-config --report-stats=no >/dev/null)
|
||||
# the generated listener only serves `client`; add media + open the door
|
||||
python3 - "$CFG" <<'PY'
|
||||
import sys, re
|
||||
p = sys.argv[1]; s = open(p).read()
|
||||
s = s.replace(" - client\n", " - client\n - media\n", 1)
|
||||
s += """
|
||||
enable_registration: true
|
||||
enable_registration_without_verification: true
|
||||
rc_message: { per_second: 1000, burst_count: 10000 }
|
||||
rc_registration: { per_second: 1000, burst_count: 10000 }
|
||||
rc_login: { address: { per_second: 1000, burst_count: 10000 }, account: { per_second: 1000, burst_count: 10000 }, failed_attempts: { per_second: 1000, burst_count: 10000 } }
|
||||
rc_joins: { local: { per_second: 1000, burst_count: 10000 }, remote: { per_second: 1000, burst_count: 10000 } }
|
||||
rc_presence: { per_user: { per_second: 1000, burst_count: 10000 } }
|
||||
max_upload_size: 50M
|
||||
suppress_key_server_warning: true
|
||||
"""
|
||||
open(p, "w").write(s)
|
||||
PY
|
||||
fi
|
||||
}
|
||||
|
||||
start() {
|
||||
install
|
||||
if [ -f "$PIDFILE" ] && kill -0 "$(cat "$PIDFILE")" 2>/dev/null; then
|
||||
echo "already running (pid $(cat "$PIDFILE"))"; return
|
||||
fi
|
||||
(cd "$DIR" && setsid nohup "$VENV/bin/python" -m synapse.app.homeserver --config-path homeserver.yaml \
|
||||
> synapse.log 2>&1 < /dev/null & echo $! > "$PIDFILE")
|
||||
for _ in $(seq 1 40); do
|
||||
sleep 1
|
||||
curl -sf http://127.0.0.1:8008/_matrix/client/versions >/dev/null && { echo "synapse up on http://localhost:8008"; return; }
|
||||
done
|
||||
echo "synapse did not come up — see $DIR/synapse.log" >&2; exit 1
|
||||
}
|
||||
|
||||
stop() {
|
||||
if [ -f "$PIDFILE" ]; then kill "$(cat "$PIDFILE")" 2>/dev/null || true; rm -f "$PIDFILE"; echo stopped; fi
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
start) start ;;
|
||||
stop) stop ;;
|
||||
reset) stop; rm -f "$DIR"/homeserver.db* ; rm -rf "$DIR/media_store"; echo "database wiped"; ;;
|
||||
*) echo "usage: $0 start|stop|reset" >&2; exit 2 ;;
|
||||
esac
|
||||
Executable
+63
@@ -0,0 +1,63 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Seed the local dev homeserver (scripts/dev-homeserver.sh) with two users
|
||||
and a busy unencrypted room: alice + bob, "Busy Room", N messages, one image
|
||||
every 10th message. Idempotent for the users; every run creates a new room.
|
||||
|
||||
python3 scripts/dev-seed.py [N=400]
|
||||
"""
|
||||
import json, struct, sys, time, urllib.error, urllib.parse, urllib.request, zlib
|
||||
|
||||
HS = "http://127.0.0.1:8008"
|
||||
PASSWORD = "password123"
|
||||
|
||||
|
||||
def req(method, path, data=None, token=None, raw=None, ctype="application/json"):
|
||||
headers = {"Content-Type": ctype}
|
||||
if token:
|
||||
headers["Authorization"] = f"Bearer {token}"
|
||||
body = raw if raw is not None else (json.dumps(data).encode() if data is not None else None)
|
||||
r = urllib.request.Request(HS + path, data=body, headers=headers, method=method)
|
||||
return json.load(urllib.request.urlopen(r))
|
||||
|
||||
|
||||
def register_or_login(user):
|
||||
try:
|
||||
return req("POST", "/_matrix/client/v3/register",
|
||||
{"username": user, "password": PASSWORD, "auth": {"type": "m.login.dummy"}})
|
||||
except urllib.error.HTTPError:
|
||||
return req("POST", "/_matrix/client/v3/login",
|
||||
{"type": "m.login.password", "identifier": {"type": "m.id.user", "user": user}, "password": PASSWORD})
|
||||
|
||||
|
||||
def png(w, h, rgb):
|
||||
raw = b"".join(b"\x00" + bytes(rgb) * w for _ in range(h))
|
||||
def chunk(t, d):
|
||||
return struct.pack(">I", len(d)) + t + d + struct.pack(">I", zlib.crc32(t + d) & 0xFFFFFFFF)
|
||||
return (b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0))
|
||||
+ chunk(b"IDAT", zlib.compress(raw)) + chunk(b"IEND", b""))
|
||||
|
||||
|
||||
def main():
|
||||
n = int(sys.argv[1]) if len(sys.argv) > 1 else 400
|
||||
alice, bob = register_or_login("alice"), register_or_login("bob")
|
||||
ta, tb = alice["access_token"], bob["access_token"]
|
||||
room = req("POST", "/_matrix/client/v3/createRoom",
|
||||
{"name": "Busy Room", "preset": "public_chat", "visibility": "public"}, ta)["room_id"]
|
||||
req("POST", f"/_matrix/client/v3/join/{urllib.parse.quote(room)}", {}, tb)
|
||||
txn = int(time.time() * 1000)
|
||||
for i in range(n):
|
||||
tok = ta if i % 2 == 0 else tb
|
||||
if i % 10 == 0:
|
||||
data = png(64, 48, ((i * 37) % 256, (i * 91) % 256, (i * 17) % 256))
|
||||
up = req("POST", f"/_matrix/media/v3/upload?filename=img{i}.png", token=tok, raw=data, ctype="image/png")
|
||||
content = {"msgtype": "m.image", "body": f"img{i}.png", "url": up["content_uri"],
|
||||
"info": {"mimetype": "image/png", "w": 64, "h": 48, "size": len(data)}}
|
||||
else:
|
||||
content = {"msgtype": "m.text", "body": f"message #{i}"}
|
||||
txn += 1
|
||||
req("PUT", f"/_matrix/client/v3/rooms/{urllib.parse.quote(room)}/send/m.room.message/{txn}", content, tok)
|
||||
print(json.dumps({"room": room, "alice": alice["user_id"], "bob": bob["user_id"], "count": n}))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -4,6 +4,7 @@ import { EventType } from 'matrix-js-sdk';
|
||||
import { Page, PageContent, PageHeader } from '../../components/page';
|
||||
import { useMatrixClient } from '../../hooks/useMatrixClient';
|
||||
import { useRoom } from '../../hooks/useRoom';
|
||||
import { createDetachedTimelineSet } from '../../utils/detachedTimeline';
|
||||
import { useRoomName } from '../../hooks/useRoomMeta';
|
||||
import { SequenceCard } from '../../components/sequence-card';
|
||||
import { SequenceCardStyle } from '../common-settings/styles.css';
|
||||
@@ -73,7 +74,10 @@ export function ExportRoomHistory({ requestClose }: ExportRoomHistoryProps) {
|
||||
// so we must deduplicate by eventId to avoid re-adding the same events
|
||||
// on each pagination step.
|
||||
const seen = new Set<string>();
|
||||
const timeline = room.getLiveTimeline();
|
||||
// [Gitea #163] Paginate a private timeline set, not the live one: the
|
||||
// export can pull thousands of events and the room behind this modal
|
||||
// renders the live timeline by index.
|
||||
const timeline = createDetachedTimelineSet(mx, room).set.getLiveTimeline();
|
||||
let canLoadMore = true;
|
||||
// Track the oldest collected timestamp incrementally so the fromTs check
|
||||
// doesn't rescan the whole `collected` array on every pagination step.
|
||||
|
||||
@@ -1,9 +1,14 @@
|
||||
import React, { useCallback, useEffect, useState } from 'react';
|
||||
import React, { useCallback, useEffect, useMemo, useState } from 'react';
|
||||
import { Box, Button, Icon, IconButton, Icons, Scroll, Spinner, Text, color, config } from 'folds';
|
||||
import { MatrixEvent } from 'matrix-js-sdk';
|
||||
import { Page, PageContent, PageHeader } from '../../components/page';
|
||||
import { useMatrixClient } from '../../hooks/useMatrixClient';
|
||||
import { useRoom } from '../../hooks/useRoom';
|
||||
import {
|
||||
collectTimelineEvents,
|
||||
createDetachedTimelineSet,
|
||||
createTypesFilter,
|
||||
} from '../../utils/detachedTimeline';
|
||||
|
||||
// ── Types ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -343,15 +348,23 @@ export function RoomActivityLog({ requestClose }: RoomActivityLogProps) {
|
||||
const [hasLoadedOnce, setHasLoadedOnce] = useState(false);
|
||||
const [canLoadMore, setCanLoadMore] = useState(true);
|
||||
|
||||
// [Gitea #163] Page through a detached, type-filtered timeline set: the
|
||||
// live timeline (rendered by index behind this modal) must not be mutated.
|
||||
const detached = useMemo(
|
||||
() =>
|
||||
createDetachedTimelineSet(mx, room, {
|
||||
filter: createTypesFilter(mx.getSafeUserId(), STATE_EVENT_TYPES, 'io.lotus.activity'),
|
||||
filterSafeWhenEncrypted: true,
|
||||
}),
|
||||
[mx, room],
|
||||
);
|
||||
|
||||
const getStateEvents = useCallback((): MatrixEvent[] => {
|
||||
const typeSet = new Set<string>(STATE_EVENT_TYPES);
|
||||
return room
|
||||
.getLiveTimeline()
|
||||
.getEvents()
|
||||
return collectTimelineEvents(detached.set.getLiveTimeline())
|
||||
.filter((ev) => typeSet.has(ev.getType()) && !ev.isRedacted())
|
||||
.slice()
|
||||
.reverse();
|
||||
}, [room]);
|
||||
}, [detached]);
|
||||
|
||||
const [events, setEvents] = useState<MatrixEvent[]>(() => getStateEvents());
|
||||
|
||||
@@ -363,7 +376,7 @@ export function RoomActivityLog({ requestClose }: RoomActivityLogProps) {
|
||||
if (loading || !canLoadMore) return;
|
||||
setLoading(true);
|
||||
try {
|
||||
const hasMore = await mx.paginateEventTimeline(room.getLiveTimeline(), {
|
||||
const hasMore = await mx.paginateEventTimeline(detached.set.getLiveTimeline(), {
|
||||
backwards: true,
|
||||
limit: 50,
|
||||
});
|
||||
@@ -375,7 +388,7 @@ export function RoomActivityLog({ requestClose }: RoomActivityLogProps) {
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}, [loading, canLoadMore, mx, room, getStateEvents]);
|
||||
}, [loading, canLoadMore, mx, detached, getStateEvents]);
|
||||
|
||||
// Auto-paginate on mount — state events are rarely in the initial sync
|
||||
// window, so we immediately fetch backwards to populate the log.
|
||||
|
||||
@@ -17,7 +17,7 @@ import {
|
||||
color,
|
||||
config,
|
||||
} from 'folds';
|
||||
import { EventType, MatrixClient, MatrixEvent, MsgType, Room } from 'matrix-js-sdk';
|
||||
import { MatrixClient, MatrixEvent, MsgType, Room } from 'matrix-js-sdk';
|
||||
import FocusTrap from 'focus-trap-react';
|
||||
import classNames from 'classnames';
|
||||
import { useNearViewport } from '../../hooks/useNearViewport';
|
||||
@@ -31,6 +31,7 @@ import { AudioContent, FileDownloadButton } from '../../components/message';
|
||||
import { MediaControl } from '../../components/media';
|
||||
import { getBlobSafeMimeType, mimeTypeToExt } from '../../utils/mimeTypes';
|
||||
import { useRoomNavigate } from '../../hooks/useRoomNavigate';
|
||||
import { useRoomMediaTimeline } from '../../hooks/useRoomMediaTimeline';
|
||||
import { ContainerColor } from '../../styles/ContainerColor.css';
|
||||
import { stopPropagation } from '../../utils/keyboard';
|
||||
import * as css from './MediaGallery.css';
|
||||
@@ -702,10 +703,17 @@ export function MediaGallery({ room, onClose }: MediaGalleryProps) {
|
||||
);
|
||||
|
||||
const [tab, setTab] = useState<GalleryTab>('image');
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [hasLoadedOnce, setHasLoadedOnce] = useState(false);
|
||||
const [canLoadMore, setCanLoadMore] = useState(true);
|
||||
const [loadError, setLoadError] = useState(false);
|
||||
// [Gitea #163] Media is paginated on its own timeline set — never on
|
||||
// `room.getLiveTimeline()`, which the message list behind this drawer is
|
||||
// rendering by index.
|
||||
const {
|
||||
events: mediaEvents,
|
||||
loadMore: handleLoadMore,
|
||||
loading,
|
||||
loadError,
|
||||
canLoadMore,
|
||||
hasLoadedOnce,
|
||||
} = useRoomMediaTimeline(mx, room);
|
||||
const [lightboxIndex, setLightboxIndex] = useState<number | null>(null);
|
||||
const sentinelRef = useRef<HTMLDivElement>(null);
|
||||
|
||||
@@ -730,51 +738,11 @@ export function MediaGallery({ room, onClose }: MediaGalleryProps) {
|
||||
|
||||
const msgtype = TAB_MSGTYPES[tab];
|
||||
|
||||
const getFilteredEvents = useCallback(
|
||||
(): MatrixEvent[] =>
|
||||
room
|
||||
.getLiveTimeline()
|
||||
.getEvents()
|
||||
.filter((ev) => {
|
||||
if (ev.isRedacted()) return false;
|
||||
const c = ev.getContent();
|
||||
return ev.getType() === EventType.RoomMessage && c.msgtype === msgtype;
|
||||
})
|
||||
.slice()
|
||||
.reverse(),
|
||||
[room, msgtype],
|
||||
const events = useMemo(
|
||||
() => mediaEvents.filter((ev) => ev.getContent().msgtype === msgtype),
|
||||
[mediaEvents, msgtype],
|
||||
);
|
||||
|
||||
const [events, setEvents] = useState<MatrixEvent[]>(() => getFilteredEvents());
|
||||
|
||||
useEffect(() => {
|
||||
setEvents(getFilteredEvents());
|
||||
setCanLoadMore(true);
|
||||
setHasLoadedOnce(false);
|
||||
setLoadError(false);
|
||||
}, [getFilteredEvents]);
|
||||
|
||||
const handleLoadMore = useCallback(async () => {
|
||||
if (loading || !canLoadMore) return;
|
||||
setLoading(true);
|
||||
setLoadError(false);
|
||||
try {
|
||||
const hasMore = await mx.paginateEventTimeline(room.getLiveTimeline(), {
|
||||
backwards: true,
|
||||
limit: 100,
|
||||
});
|
||||
setEvents(getFilteredEvents());
|
||||
setCanLoadMore(hasMore);
|
||||
setHasLoadedOnce(true);
|
||||
} catch {
|
||||
// Stop auto-retry: the sentinel would keep firing on every render otherwise.
|
||||
// The user can retry manually via the button shown in the error state.
|
||||
setLoadError(true);
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}, [loading, canLoadMore, mx, room, getFilteredEvents]);
|
||||
|
||||
// Auto-load when sentinel scrolls into view
|
||||
useEffect(() => {
|
||||
const sentinel = sentinelRef.current;
|
||||
@@ -814,14 +782,10 @@ export function MediaGallery({ room, onClose }: MediaGalleryProps) {
|
||||
};
|
||||
});
|
||||
|
||||
// Per-tab counts for the tab labels (single pass over loaded timeline)
|
||||
// Per-tab counts for the tab labels (single pass over the loaded media)
|
||||
const tabCounts = useMemo(() => {
|
||||
const counts: Record<GalleryTab, number> = { image: 0, video: 0, audio: 0, file: 0 };
|
||||
room
|
||||
.getLiveTimeline()
|
||||
.getEvents()
|
||||
.forEach((ev) => {
|
||||
if (ev.isRedacted() || ev.getType() !== EventType.RoomMessage) return;
|
||||
mediaEvents.forEach((ev) => {
|
||||
const mt = ev.getContent().msgtype;
|
||||
if (mt === MsgType.Image) counts.image += 1;
|
||||
else if (mt === MsgType.Video) counts.video += 1;
|
||||
@@ -829,10 +793,7 @@ export function MediaGallery({ room, onClose }: MediaGalleryProps) {
|
||||
else if (mt === MsgType.File) counts.file += 1;
|
||||
});
|
||||
return counts;
|
||||
// `events` is intentional: it changes when more history is paginated in, so
|
||||
// the counts stay in sync with the loaded window (it isn't read directly).
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [room, events]);
|
||||
}, [mediaEvents]);
|
||||
|
||||
// Group image/video events by month for the grid
|
||||
type MonthGroup = { label: string; events: MatrixEvent[] };
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from 'react';
|
||||
import {
|
||||
EventTimelineSetHandlerMap,
|
||||
EventType,
|
||||
MatrixClient,
|
||||
MatrixEvent,
|
||||
MatrixEventEvent,
|
||||
MsgType,
|
||||
Room,
|
||||
RoomEvent,
|
||||
RoomEventHandlerMap,
|
||||
} from 'matrix-js-sdk';
|
||||
|
||||
import {
|
||||
collectTimelineEvents,
|
||||
createDetachedTimelineSet,
|
||||
createMediaFilter,
|
||||
DetachedTimelineSet,
|
||||
} from '../utils/detachedTimeline';
|
||||
import { decryptAllTimelineEvent } from '../utils/room';
|
||||
|
||||
export const MEDIA_MSGTYPES: ReadonlySet<string> = new Set<string>([
|
||||
MsgType.Image,
|
||||
MsgType.Video,
|
||||
MsgType.Audio,
|
||||
MsgType.File,
|
||||
]);
|
||||
|
||||
export const isMediaMessage = (event: MatrixEvent): boolean =>
|
||||
event.getType() === EventType.RoomMessage &&
|
||||
!event.isRedacted() &&
|
||||
MEDIA_MSGTYPES.has(event.getContent().msgtype as string);
|
||||
|
||||
const PAGE_SIZE = 100;
|
||||
|
||||
export type RoomMediaTimeline = {
|
||||
/** Every loaded media message in the room, newest first. */
|
||||
events: MatrixEvent[];
|
||||
loadMore: () => Promise<void>;
|
||||
loading: boolean;
|
||||
loadError: boolean;
|
||||
canLoadMore: boolean;
|
||||
hasLoadedOnce: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* [Gitea #163] Media events for the gallery, paginated on a timeline set of
|
||||
* their own so the room's live timeline (and the message list rendering it)
|
||||
* is never mutated. Unencrypted rooms page through a server-side
|
||||
* `contains_url` filter (100 media per page); encrypted rooms page raw
|
||||
* history into a private set and filter after decrypting, exactly what the
|
||||
* gallery used to do on the live timeline.
|
||||
*/
|
||||
export const useRoomMediaTimeline = (mx: MatrixClient, room: Room): RoomMediaTimeline => {
|
||||
const detached = useMemo<DetachedTimelineSet>(
|
||||
() => createDetachedTimelineSet(mx, room, { filter: createMediaFilter(mx.getSafeUserId()) }),
|
||||
[mx, room],
|
||||
);
|
||||
|
||||
const readEvents = useCallback(
|
||||
(): MatrixEvent[] =>
|
||||
collectTimelineEvents(detached.set.getLiveTimeline()).filter(isMediaMessage).reverse(),
|
||||
[detached],
|
||||
);
|
||||
|
||||
const [events, setEvents] = useState<MatrixEvent[]>(readEvents);
|
||||
const [loading, setLoading] = useState(false);
|
||||
const [loadError, setLoadError] = useState(false);
|
||||
const [canLoadMore, setCanLoadMore] = useState(true);
|
||||
const [hasLoadedOnce, setHasLoadedOnce] = useState(false);
|
||||
const loadingRef = useRef(false);
|
||||
|
||||
useEffect(() => {
|
||||
setEvents(readEvents());
|
||||
setCanLoadMore(true);
|
||||
setHasLoadedOnce(false);
|
||||
setLoadError(false);
|
||||
}, [readEvents]);
|
||||
|
||||
// Live updates. The server-filtered set is registered with the room and
|
||||
// receives new events itself; the private set (encrypted rooms) is fed here
|
||||
// once the event has decrypted. Redactions are applied to both.
|
||||
useEffect(() => {
|
||||
const { set, serverFiltered } = detached;
|
||||
const refresh = () => setEvents(readEvents());
|
||||
|
||||
const addIfMedia = (event: MatrixEvent) => {
|
||||
if (!isMediaMessage(event)) return;
|
||||
if (set.findEventById(event.getId() ?? '')) return;
|
||||
set.addLiveEvent(event, { addToState: false });
|
||||
refresh();
|
||||
};
|
||||
|
||||
const onTimeline: EventTimelineSetHandlerMap[RoomEvent.Timeline] = (
|
||||
event,
|
||||
eventRoom,
|
||||
_toStart,
|
||||
_removed,
|
||||
data,
|
||||
) => {
|
||||
if (eventRoom?.roomId !== room.roomId || !data.liveEvent) return;
|
||||
if (serverFiltered) {
|
||||
// The room already routed it into our set (client-side filtered).
|
||||
if (data.timeline.getTimelineSet() === set) refresh();
|
||||
return;
|
||||
}
|
||||
if (event.isBeingDecrypted() || event.shouldAttemptDecryption()) {
|
||||
event.once(MatrixEventEvent.Decrypted, () => addIfMedia(event));
|
||||
return;
|
||||
}
|
||||
addIfMedia(event);
|
||||
};
|
||||
|
||||
const onRedaction: RoomEventHandlerMap[RoomEvent.Redaction] = (event, eventRoom) => {
|
||||
if (eventRoom?.roomId !== room.roomId) return;
|
||||
const redactedId = event.event.redacts;
|
||||
if (!redactedId) return;
|
||||
if (!serverFiltered) set.removeEvent(redactedId);
|
||||
refresh();
|
||||
};
|
||||
|
||||
room.on(RoomEvent.Timeline, onTimeline);
|
||||
room.on(RoomEvent.Redaction, onRedaction);
|
||||
return () => {
|
||||
room.removeListener(RoomEvent.Timeline, onTimeline);
|
||||
room.removeListener(RoomEvent.Redaction, onRedaction);
|
||||
};
|
||||
}, [detached, room, readEvents]);
|
||||
|
||||
const loadMore = useCallback(async () => {
|
||||
if (loadingRef.current || !canLoadMore) return;
|
||||
loadingRef.current = true;
|
||||
setLoading(true);
|
||||
setLoadError(false);
|
||||
try {
|
||||
const timeline = detached.set.getLiveTimeline();
|
||||
const hasMore = await mx.paginateEventTimeline(timeline, {
|
||||
backwards: true,
|
||||
limit: PAGE_SIZE,
|
||||
});
|
||||
if (room.hasEncryptionStateEvent()) {
|
||||
await decryptAllTimelineEvent(mx, timeline);
|
||||
}
|
||||
setEvents(readEvents());
|
||||
setCanLoadMore(hasMore);
|
||||
setHasLoadedOnce(true);
|
||||
} catch {
|
||||
// Stop auto-retry: the sentinel would keep firing on every render otherwise.
|
||||
// The user can retry manually via the button shown in the error state.
|
||||
setLoadError(true);
|
||||
} finally {
|
||||
loadingRef.current = false;
|
||||
setLoading(false);
|
||||
}
|
||||
}, [mx, room, detached, canLoadMore, readEvents]);
|
||||
|
||||
return { events, loadMore, loading, loadError, canLoadMore, hasLoadedOnce };
|
||||
};
|
||||
@@ -0,0 +1,141 @@
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { Direction, EventTimeline, MatrixClient, MatrixEvent, Room } from 'matrix-js-sdk';
|
||||
import {
|
||||
collectTimelineEvents,
|
||||
createDetachedTimelineSet,
|
||||
createMediaFilter,
|
||||
createTypesFilter,
|
||||
getEarliestLinkedTimeline,
|
||||
} from './detachedTimeline';
|
||||
|
||||
const ROOM_ID = '!room:localhost';
|
||||
|
||||
const makeClient = (): MatrixClient =>
|
||||
({
|
||||
getUserId: () => '@alice:localhost',
|
||||
getSafeUserId: () => '@alice:localhost',
|
||||
supportsThreads: () => false,
|
||||
canSupport: new Map(),
|
||||
isRoomEncrypted: () => false,
|
||||
getRooms: () => [],
|
||||
decryptEventIfNeeded: () => Promise.resolve(),
|
||||
reEmitter: { reEmit: () => undefined },
|
||||
getCrypto: () => undefined,
|
||||
}) as unknown as MatrixClient;
|
||||
|
||||
const makeRoom = (mx: MatrixClient, encrypted = false): Room => {
|
||||
const room = new Room(ROOM_ID, mx, '@alice:localhost', { timelineSupport: true });
|
||||
if (encrypted) {
|
||||
room.currentState.setStateEvents([
|
||||
new MatrixEvent({
|
||||
type: 'm.room.encryption',
|
||||
state_key: '',
|
||||
room_id: ROOM_ID,
|
||||
sender: '@alice:localhost',
|
||||
content: { algorithm: 'm.megolm.v1.aes-sha2' },
|
||||
event_id: '$enc',
|
||||
origin_server_ts: 1,
|
||||
}),
|
||||
]);
|
||||
}
|
||||
return room;
|
||||
};
|
||||
|
||||
let counter = 0;
|
||||
const msg = (body: string, extra: Record<string, unknown> = {}): MatrixEvent => {
|
||||
counter += 1;
|
||||
return new MatrixEvent({
|
||||
type: 'm.room.message',
|
||||
room_id: ROOM_ID,
|
||||
sender: '@bob:localhost',
|
||||
content: { msgtype: 'm.text', body, ...extra },
|
||||
event_id: `$e${counter}`,
|
||||
origin_server_ts: counter,
|
||||
});
|
||||
};
|
||||
|
||||
test('collectTimelineEvents walks backward neighbours oldest-first', () => {
|
||||
const room = makeRoom(makeClient());
|
||||
const set = room.getUnfilteredTimelineSet();
|
||||
const live = set.getLiveTimeline();
|
||||
const older = new EventTimeline(set);
|
||||
older.setNeighbouringTimeline(live, Direction.Forward);
|
||||
live.setNeighbouringTimeline(older, Direction.Backward);
|
||||
const a = msg('a');
|
||||
const b = msg('b');
|
||||
const c = msg('c');
|
||||
older.addEvent(a, { toStartOfTimeline: false, addToState: false });
|
||||
live.addEvent(b, { toStartOfTimeline: false, addToState: false });
|
||||
live.addEvent(c, { toStartOfTimeline: false, addToState: false });
|
||||
assert.deepEqual(
|
||||
collectTimelineEvents(live).map((e) => e.getContent().body),
|
||||
['a', 'b', 'c'],
|
||||
);
|
||||
assert.equal(getEarliestLinkedTimeline(live), older);
|
||||
assert.equal(getEarliestLinkedTimeline(older), older);
|
||||
});
|
||||
|
||||
test('createMediaFilter targets m.room.message with a url; types filter is type-only', () => {
|
||||
const media = createMediaFilter('@alice:localhost').getDefinition();
|
||||
assert.deepEqual(media.room?.timeline, { types: ['m.room.message'], contains_url: true });
|
||||
const types = createTypesFilter('@alice:localhost', ['m.room.member'], 'x').getDefinition();
|
||||
assert.deepEqual(types.room?.timeline, { types: ['m.room.member'] });
|
||||
});
|
||||
|
||||
test('private detached set: seeded from live history, back token copied, live timeline untouched', () => {
|
||||
const mx = makeClient();
|
||||
const room = makeRoom(mx);
|
||||
const live = room.getLiveTimeline();
|
||||
live.setPaginationToken('tok-back', Direction.Backward);
|
||||
const events = [msg('1'), msg('2', { msgtype: 'm.image', url: 'mxc://x/y' }), msg('3')];
|
||||
events.forEach((e) => live.addEvent(e, { toStartOfTimeline: false, addToState: false }));
|
||||
|
||||
const { set, serverFiltered } = createDetachedTimelineSet(mx, room); // no filter → private
|
||||
assert.equal(serverFiltered, false);
|
||||
assert.notEqual(set, room.getUnfilteredTimelineSet());
|
||||
assert.equal(room.getTimelineSets().includes(set), false, 'must not be registered on the room');
|
||||
assert.deepEqual(
|
||||
set
|
||||
.getLiveTimeline()
|
||||
.getEvents()
|
||||
.map((e) => e.getContent().body),
|
||||
['1', '2', '3'],
|
||||
);
|
||||
assert.equal(set.getLiveTimeline().getPaginationToken(Direction.Backward), 'tok-back');
|
||||
|
||||
// Prepending into the detached set (what /messages pagination does) leaves
|
||||
// the live timeline's arrays — and therefore RoomTimeline's indices — alone.
|
||||
set.getLiveTimeline().addEvent(msg('0'), { toStartOfTimeline: true, addToState: false });
|
||||
assert.equal(live.getEvents().length, 3);
|
||||
assert.equal(set.getLiveTimeline().getEvents().length, 4);
|
||||
});
|
||||
|
||||
test('filtered detached set is used in plain rooms and skipped in encrypted rooms', () => {
|
||||
const mx = makeClient();
|
||||
const filter = createMediaFilter('@alice:localhost');
|
||||
|
||||
const plain = makeRoom(mx);
|
||||
const plainLive = plain.getLiveTimeline();
|
||||
[msg('t'), msg('i', { msgtype: 'm.image', url: 'mxc://x/y' })].forEach((e) =>
|
||||
plainLive.addEvent(e, { toStartOfTimeline: false, addToState: false }),
|
||||
);
|
||||
const a = createDetachedTimelineSet(mx, plain, { filter });
|
||||
assert.equal(a.serverFiltered, true);
|
||||
assert.equal(plain.getTimelineSets().includes(a.set), true, 'registered so it gets live events');
|
||||
assert.deepEqual(
|
||||
a.set
|
||||
.getLiveTimeline()
|
||||
.getEvents()
|
||||
.map((e) => e.getContent().body),
|
||||
['i'],
|
||||
'prepopulated with the client-side filtered subset',
|
||||
);
|
||||
assert.equal(createDetachedTimelineSet(mx, plain, { filter }).set, a.set, 'cached per filter');
|
||||
|
||||
const enc = makeRoom(mx, true);
|
||||
const b = createDetachedTimelineSet(mx, enc, { filter });
|
||||
assert.equal(b.serverFiltered, false, 'contains_url cannot see ciphertext');
|
||||
const c = createDetachedTimelineSet(mx, enc, { filter, filterSafeWhenEncrypted: true });
|
||||
assert.equal(c.serverFiltered, true, 'type-only filters are fine in encrypted rooms');
|
||||
});
|
||||
@@ -0,0 +1,143 @@
|
||||
import {
|
||||
Direction,
|
||||
EventTimeline,
|
||||
EventTimelineSet,
|
||||
Filter,
|
||||
MatrixClient,
|
||||
MatrixEvent,
|
||||
Room,
|
||||
} from 'matrix-js-sdk';
|
||||
|
||||
/**
|
||||
* [Gitea #163] Timelines that paginate history WITHOUT touching the room's
|
||||
* live timeline.
|
||||
*
|
||||
* `RoomTimeline` renders a numeric window (`range`) of absolute indices into
|
||||
* the live timeline's event arrays. Backwards pagination in the SDK PREPENDS
|
||||
* (`events.splice(0, 0, …)`), so anything else that calls
|
||||
* `paginateEventTimeline(room.getLiveTimeline())` — the media gallery, the
|
||||
* activity log, history export — silently shifts what those indices point at
|
||||
* and the visible timeline "jumps into the past" on its next render. The fix
|
||||
* is for side panels to page through their own `EventTimelineSet`; this file
|
||||
* is the shared plumbing for that.
|
||||
*/
|
||||
|
||||
/** All events of a timeline and its backward neighbours, oldest first. */
|
||||
export const collectTimelineEvents = (timeline: EventTimeline): MatrixEvent[] => {
|
||||
const timelines: EventTimeline[] = [];
|
||||
let current: EventTimeline | null = timeline;
|
||||
while (current) {
|
||||
timelines.unshift(current);
|
||||
current = current.getNeighbouringTimeline(Direction.Backward);
|
||||
}
|
||||
return timelines.flatMap((t) => t.getEvents());
|
||||
};
|
||||
|
||||
/** Earliest timeline linked backwards from `timeline` (where the back token lives). */
|
||||
export const getEarliestLinkedTimeline = (timeline: EventTimeline): EventTimeline => {
|
||||
let current = timeline;
|
||||
let prev = current.getNeighbouringTimeline(Direction.Backward);
|
||||
while (prev) {
|
||||
current = prev;
|
||||
prev = current.getNeighbouringTimeline(Direction.Backward);
|
||||
}
|
||||
return current;
|
||||
};
|
||||
|
||||
/**
|
||||
* Server-side filter for the media gallery in unencrypted rooms: only
|
||||
* `m.room.message` events whose content carries a `url`, i.e. image / video /
|
||||
* audio / file. `/messages` then returns 100 media events per page instead of
|
||||
* 100 events of which a handful are media. Useless in encrypted rooms (the
|
||||
* server only sees ciphertext), so `createDetachedTimelineSet` skips it there.
|
||||
*/
|
||||
export const createMediaFilter = (userId: string): Filter => {
|
||||
const filter = new Filter(userId);
|
||||
filter.setDefinition({
|
||||
room: {
|
||||
timeline: {
|
||||
types: ['m.room.message'],
|
||||
contains_url: true,
|
||||
},
|
||||
},
|
||||
});
|
||||
// Key for `room.filteredTimelineSets` (the SDK caches per filterId) — only
|
||||
// ever sent to the server inline on `/messages`, never registered via
|
||||
// `/filter`, so any stable string will do.
|
||||
filter.filterId = 'io.lotus.media';
|
||||
return filter;
|
||||
};
|
||||
|
||||
/**
|
||||
* Server-side filter on event `type` only — safe in encrypted rooms because
|
||||
* state events are never encrypted.
|
||||
*/
|
||||
export const createTypesFilter = (userId: string, types: readonly string[], id: string): Filter => {
|
||||
const filter = new Filter(userId);
|
||||
filter.setDefinition({ room: { timeline: { types: [...types] } } });
|
||||
filter.filterId = id;
|
||||
return filter;
|
||||
};
|
||||
|
||||
export type DetachedTimelineSet = {
|
||||
set: EventTimelineSet;
|
||||
/** True when the server filters pages for us (the filter was usable). */
|
||||
serverFiltered: boolean;
|
||||
};
|
||||
|
||||
export type DetachedTimelineOptions = {
|
||||
/** Server-side filter for `/messages` (and client-side for live events). */
|
||||
filter?: Filter;
|
||||
/**
|
||||
* Set when the filter only looks at unencrypted fields (event `type`,
|
||||
* `sender`, `state_key`), so it is still correct in an encrypted room. A
|
||||
* content filter such as `contains_url` must leave this false.
|
||||
*/
|
||||
filterSafeWhenEncrypted?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* Build a timeline set for `room` that mirrors the live timeline's already
|
||||
* loaded history (no refetch) and can be paginated backwards without ever
|
||||
* mutating `room.getLiveTimeline()`.
|
||||
*
|
||||
* - usable `filter` → `room.getOrCreateFilteredTimelineSet`: registered with
|
||||
* the room, so it also receives live events (client-side filtered) and
|
||||
* redactions for free, and it is cached on the room across open/close.
|
||||
* - otherwise → a private `EventTimelineSet` seeded from the live timeline;
|
||||
* the caller feeds it live events via `addLiveEvent` (see
|
||||
* `useRoomMediaTimeline`). Not registered with the room, so a
|
||||
* `TimelineReset` after a sync gap does not clear it — acceptable for a
|
||||
* read-only side panel that is rebuilt on every open.
|
||||
*/
|
||||
export const createDetachedTimelineSet = (
|
||||
mx: MatrixClient,
|
||||
room: Room,
|
||||
{ filter, filterSafeWhenEncrypted = false }: DetachedTimelineOptions = {},
|
||||
): DetachedTimelineSet => {
|
||||
const filterUsable = !!filter && (filterSafeWhenEncrypted || !room.hasEncryptionStateEvent());
|
||||
|
||||
if (filter && filterUsable) {
|
||||
const set = room.getOrCreateFilteredTimelineSet(filter, {
|
||||
prepopulateTimeline: true,
|
||||
useSyncEvents: true,
|
||||
pendingEvents: false,
|
||||
});
|
||||
return { set, serverFiltered: true };
|
||||
}
|
||||
|
||||
const set = new EventTimelineSet(room, { timelineSupport: true, pendingEvents: false }, mx);
|
||||
const live = room.getLiveTimeline();
|
||||
// Seed with everything already loaded (already decrypted where applicable)
|
||||
// so opening the panel never refetches what the timeline has.
|
||||
collectTimelineEvents(live).forEach((event) => {
|
||||
set.addLiveEvent(event, { addToState: false });
|
||||
});
|
||||
set
|
||||
.getLiveTimeline()
|
||||
.setPaginationToken(
|
||||
getEarliestLinkedTimeline(live).getPaginationToken(Direction.Backward),
|
||||
Direction.Backward,
|
||||
);
|
||||
return { set, serverFiltered: false };
|
||||
};
|
||||
Reference in New Issue
Block a user