278 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Lotus CIandClaude Opus 5.5	b2877323c3	feat(synapse): nightly state compression on CT 109 ... The Synapse database grew from 2.2 GB (after the July cleanup) to 6.2 GB, almost all of it state_groups_state from large federated rooms with one or two local members. synapse_auto_compressor (matrix-org/rust-synapse-compress-state, built from main) now runs nightly at 04:30, after the 03:00 vzdump backup. It verifies compressed state against the original, writes each state group in its own transaction, skips chunks that would not shrink, and resumes from stored progress. The initial backlog took ~8 minutes with no Synapse errors: state_groups_state 11.0 M -> 4.25 M rows (-61%). Also documents the new "Synapse Database Growing Fast" Grafana alert (> 500 MB in 7 days). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-11-03 19:08:01 -05:00
jared	ded4a30530	Merge pull request 'docs: all 13 Hookshot webhooks rotated' (#58) from docs/rotated-remaining-webhooks into main	2026-11-03 17:03:09 -05:00
Lotus CIandClaude Opus 5.5	f3eb3ea406	docs: all 13 Hookshot webhooks rotated; old IDs in history are dead ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-11-03 16:59:31 -05:00
jared	7b01a9fcdc	Merge pull request 'docs: call stack upgraded (lk-jwt-service 0.9.0, LiveKit 1.13.11)' (#57) from docs/call-stack-upgraded into main	2026-11-03 16:42:04 -05:00
Lotus CIandClaude Opus 5.5	bc504a881d	docs: call stack upgraded to lk-jwt-service 0.9.0 and LiveKit 1.13.11 ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-11-03 16:36:28 -05:00
jared	89291f8707	Merge pull request 'docs: note the 2026-11-03 webhook rotation' (#56) from docs/rotated-webhook-prefixes into main	2026-11-03 00:57:42 -05:00
Lotus CIandClaude Opus 5.5	9eca253229	docs: note the 2026-11-03 webhook rotation (Grafana, Proxmox, Uptime Kuma, Ci-Alert) ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-11-03 00:51:35 -05:00
jared	a06ebfe6f6	Merge pull request 'docs: stop publishing Hookshot webhook IDs; record Hookshot + call-stack upgrades' (#55) from docs/hookshot-ids-and-upgrades into main	2026-11-03 00:36:18 -05:00
Lotus CIandClaude Opus 5.5	6e41477a0b	docs: stop publishing Hookshot webhook IDs; record the Hookshot and call-stack upgrades ... - The README's webhook table listed every Hookshot webhook's full UUID. The UUID is the webhook's only secret and this repo is public, so anyone could post into Spam and Stuff. Shorten them to 8 characters and add CI (readme-no-webhook-ids) rejecting full UUIDs in README.md. They remain in git history until rotated. - Hookshot is now 7.5.0: how it's built (pnpm 7.4.1+, Node 22 limit of 7.6.0) and how to swap a new build in. - Planned lk-jwt-service 0.9.0 / LiveKit 1.13.11 upgrade: the LiveKit webhook delegated leave needs, the prometheus config key, and restart caveats, from local testing. - Graceful restart: describe the fixed check (#54). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-11-03 00:34:03 -05:00
jared	8a0e16cabd	Merge pull request 'fix(livekit): graceful restart could restart LiveKit in the middle of a call' (#54) from fix/livekit-graceful-restart-detection into main	2026-11-03 00:28:12 -05:00
Lotus CIandClaude Opus 5.5	d9f023762e	fix(livekit): graceful restart could restart LiveKit in the middle of a call ... livekit-graceful-restart.sh decided "no active calls" by counting established TCP connections on :7881 with `ss ... state established | grep -c ESTAB`. Two bugs: - With a `state` filter ss omits the State column, so `grep -c ESTAB` is always 0: the check never saw anything. - :7881 is only LiveKit's ICE-TCP media fallback. Signalling is on :7880 and media is normally UDP. Reproduced in a two-person call on the local stack: the old check counted 0 and would have restarted. Now it waits while LiveKit's own livekit_participant_total is above 0 or any signalling WebSocket on :7880 is open, and also waits if the metrics can't be read. The deploy hook now installs the script when it changes (it never did, so edits to it never reached the server). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-11-03 00:25:58 -05:00
jared	d32c74052c	Merge pull request 'voice-bridge: leave Discord within ~8-18 s of the last Matrix person leaving' (#53) from voice-bridge-timing into main	2026-10-29 21:12:00 -04:00
Lotus CIandClaude Opus 5.5	0716e2acd8	voice-bridge: leave Discord within ~8-18 s of the last Matrix person leaving ... Upstream defaults (12 s leave grace + 90 s anchor linger) kept the Discord bot in the channel for about a minute after the call ended on the Matrix side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-29 21:09:01 -04:00
jared	8eb3201b38	Merge pull request 'docs: voice bridge live — bot profile, room setup, upgrade steps' (#52) from docs/voice-bridge-live into main	2026-10-29 13:24:35 -04:00
Lotus CIandClaude Opus 5.5	811ae9d2cc	docs: voice bridge live — bot profile, room setup, upgrade steps ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-29 13:22:36 -04:00
jared	b2cab883d6	voice-bridge: Discord voice ↔ Lotus voice room (patched nether-voicebridge)	2026-10-29 12:56:04 -04:00
Lotus CIandClaude Opus 5.5	46d0e98424	voice-bridge: Discord voice <-> Lotus voice room (patched nether-voicebridge) ... Production setup for the Discord voice bridge on LXC 151: config and appservice-registration templates (secrets rendered on the server), an idempotent install script (one Synapse restart on first install, creates the bot account so it can be invited), and a hardened systemd unit. voice-limit-guard: VOICE_LIMIT_EXEMPT_USERS lets the bridge's own bot listen in a call without taking a slot; relayed Discord speakers still count. Unit tests added (61 pass); verified live on the dev stack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-29 12:51:56 -04:00
jared	db2f523028	Merge pull request 'docs: bot profiles (names, avatars, where each is set)' (#50) from docs/bot-profiles into main	2026-10-29 01:32:54 -04:00
Lotus CIandClaude Opus 5.5	b06f422837	docs: bot profiles (names, avatars, where each is set) ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-29 01:30:24 -04:00
jared	f2d8a12aa7	Merge pull request 'docs: the Discord bot login stays on @jared' (#49) from docs/discord-login-owner into main	2026-10-28 22:19:19 -04:00
Lotus CIandClaude Opus 5.5	7cf68b6850	docs: the Discord bot login stays on @jared ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 22:14:37 -04:00
jared	894dd5277e	Merge pull request 'discord-bridge: bot profile, down alert, login-owner account' (#48) from discord-bridge-profile into main	2026-10-28 22:00:30 -04:00
Lotus CIandClaude Opus 5.5	0386474118	discord-bridge: bot profile, down alert, login-owner account ... Add the Discord-logo avatar and a script that uploads it as the bridge bot and sets the display name to "Discord Bridge". Document the new "Discord Bridge Down" Grafana alert, and why the bot login belongs on a dedicated non-chatting account: whoever holds it sends to Discord as the bot instead of through the relay webhook. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 21:58:34 -04:00
jared	b8a088d2a6	Merge pull request 'discord-bridge: Matrix avatars on Discord, catch up missed messages' (#47) from discord-bridge-avatars into main	2026-10-28 21:53:34 -04:00
Lotus CIandClaude Opus 5.5	8f35ab23a3	discord-bridge: Matrix avatars on Discord, catch up missed messages ... Set bridge.public_address so relayed messages carry the sender's avatar (served through a new NPM location for /mautrix-discord/avatar/), listen on 0.0.0.0 so NPM can reach it, and backfill all Discord messages missed while the bridge was down. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 21:51:02 -04:00
jared	1e4922d6a3	Merge pull request 'docs: Discord bridge is live; linking notes' (#46) from docs/discord-bridge-live into main	2026-10-28 21:48:00 -04:00
Lotus CIandClaude Opus 5.5	0587f18063	docs: Discord bridge is live; linking notes ... Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 21:45:21 -04:00
jared	7c10c9591e	Merge pull request 'Add mautrix-discord bridge config and installer' (#44) from discord-bridge into main	2026-10-28 21:25:52 -04:00
jared	3ad2fef28e	Merge branch 'main' into discord-bridge	2026-10-28 21:21:59 -04:00
jared	e16ce42785	Merge pull request 'ci(pip-audit): audit on Python 3.13 instead of ignoring filelock CVEs' (#45) from ci/pip-audit-py313 into main	2026-10-28 21:21:37 -04:00
Lotus CIandClaude Opus 5.5	246774e3eb	ci(pip-audit): audit on Python 3.13 instead of ignoring filelock CVEs ... The audit env ran Python 3.10, which can't install filelock 4.2+, so every new filelock CVE in pip-audit's own CacheControl dependency broke the job and had to be ignored by ID (11 more landed today). Audit on 3.13, which matches the bot's runtime on LXC 151 (Debian trixie), so the fixed filelock installs and the per-CVE ignores can go. Also scan discord-bridge/ for secrets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 21:18:13 -04:00
Lotus CIandClaude Opus 5.5	3e6f210f46	Add mautrix-discord bridge config and installer ... Bot-login Discord bridge on LXC 151: config template (DB password is filled in on the server), systemd unit, and a one-time install script run from the Proxmox node that sets up the PostgreSQL role/database, installs the checksum-verified v0.7.7 binary, generates the appservice registration, adds it to Synapse and restarts Synapse once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 21:08:45 -04:00
jared	3213f8c41b	Merge pull request 'docs: Synapse 1.164.1' (#43) from docs/synapse-1.164.1 into main	2026-10-28 15:05:11 -04:00
Lotus CIandClaude Opus 5.5	6cdac33ca4	docs: Synapse 1.164.1 ... Upgraded on LXC 151 (now on large1) on 2026-10-28 from 1.163.2. 1.164.1 brings msgpack 1.2.4 (GHSA-j586-36cw-2gc2: memory exhaustion via malformed public room list tokens, reachable unauthenticated) on top of 1.164.0. Checked first: no upgrade notes to act on; MSC4210 (legacy mention push rules off) handled by matrix#42 and Lotus already sending m.mentions; null profile values not used (Lotus clears with an empty string). Verified: client and federation versions report 1.164.1, federation tester OK, no ERROR/CRITICAL since restart, clients resynced (200). Rollback package kept (1.163.2+trixie1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-28 15:02:28 -04:00
jared	6b39bd931e	Merge pull request 'fix: explicit m.mentions for ticket notifications and roasts (ahead of Synapse 1.164)' (#42) from fix/explicit-mentions into main	2026-10-27 20:06:14 -04:00
Lotus CIandClaude Opus 5.5	e1b2c8575c	fix: explicit m.mentions for ticket notifications and roasts (ahead of Synapse 1.164) ... Synapse 1.164 turns on MSC4210 by default: the legacy push rules that pinged when a user ID or display name appeared in the body are removed, so a message only notifies the users listed in m.mentions. - hookshot/tinker-tickets.js deliberately pings assignees/watchers (notify_users) with matrix.to links; those pings would stop. Return mentions: { user_ids: notifyUsers } (hookshot 7.3.2 v2 transforms support it). - matrixbot: send_html takes optional mention_user_ids; !roast mentions the target when given as a full Matrix ID. Winner announcements stay without a ping (the player is already in the room). No @room is sent anywhere. Lotus Chat already sends m.mentions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-27 20:04:27 -04:00
jared	54bdada04d	Merge pull request 'docs: Synapse 1.163.2 (Pillow 13 security update)' (#41) from docs/synapse-1.163.2 into main	2026-10-26 19:30:06 -04:00
Lotus CIandClaude Opus 5.5	8a2867e64b	docs: Synapse 1.163.2 (Pillow 13 security update) ... Upgraded on LXC 151 on 2026-10-26 from 1.162.0 (the README still said 1.159.0). 1.163.2 brings Pillow 13.0.0 (security fixes in the image thumbnailer). Upgrade notes checked: the new start-up check for limit_profile_requests_to_users_who_share_rooms does not apply (unset). Verified: /versions and federation version report 1.163.2, federation tester OK, no errors since restart, Synapse thumbnailer works with Pillow 13. Rollback package kept in /var/cache/apt/archives (1.162.0+trixie1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-26 19:23:53 -04:00
jared	2e6e8e72b8	Merge pull request 'fix(cinny-nginx): own rate budget for content-hashed app files' (#40) from fix/asset-rate-limit into main	2026-10-24 17:39:18 -04:00
Lotus CIandClaude Opus 5.5	d36c782e56	ci(pip-audit): ignore filelock CVEs in pip-audit's own dependency ... pip-audit --local scans the whole audit env, which includes pip-audit's own CacheControl[filecache] -> filelock 4.1.0. Six new CVEs (fixed in 4.2.1) started failing every run; 4.2+ requires Python 3.11 and the audit env is 3.10. The bot does not use filelock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-24 17:35:45 -04:00
Lotus CIandClaude Opus 5.5	ad7d735739	fix(cinny-nginx): give content-hashed app files their own rate budget ... A cold page load fires ~75 requests within a second and the service worker precaches all ~126 build files at once (every deploy renames them), both past chat_limit's burst of 60. The excess got 503 "limiting requests": a fresh visit to /register/matrix.org rendered blank (4 chunks refused), and precache installs could fail after deploys. Like decorations (matrix #22), hashed js/css/fonts/images/wasm now use assets_limit (100 r/s, burst 400); pages and the webhook keep chat_limit. The deploy validates with nginx -t and restores the old config on failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-24 17:05:11 -04:00
jared	4bb2dd4449	Merge pull request 'feat(cinny-deploy): fail and alert when a deployed page references missing files' (#39) from fix/deploy-asset-check into main	2026-10-20 02:38:06 -04:00
Lotus CIandClaude Opus 5.5	8aea38fc67	feat(cinny-deploy): fail and alert when a deployed page references missing files ... After copying, check every script/stylesheet the app and call pages reference (src/href in both index.html files) exists in the webroot. If any is missing, log it, post an alert with the upstream checker's Matrix credentials (/etc/cinny-monitor.env, best effort) and exit 1. This would have caught the 2026-10-20 call-page outage at deploy time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-20 02:28:52 -04:00
jared	963bc87c54	Merge pull request 'fix(cinny-deploy): the call page's scripts are deployed again (regression from #30)' (#38) from fix/deploy-call-assets into main	2026-10-20 01:55:37 -04:00
Lotus CIandClaude Opus 5.5	85c58d5ed0	fix(cinny-deploy): the call page's scripts are deployed again ... #30 added `rsync --exclude assets/` to keep old app assets, but an unanchored exclude matches every directory named assets, including public/element-call/assets/. Each deploy then updated the call page's index.html but not its hashed scripts, so after the lotus.28 deploy the call page requested files that did not exist (66 x 404 from 00:05 on 2026-10-20 until the assets were copied in by hand). The excludes are now anchored (/assets/, /public/element-call/assets/), and both asset directories get the same treatment: copied without --delete, pruned after 14 days, so an open call page keeps its scripts across a deploy. config.json stays unanchored on purpose (the production call config lives at public/element-call/config.json). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-20 00:19:47 -04:00
jared	8e5cdf50b5	Merge pull request 'feat(voice-limit-guard): enforce the limit at connect time too (#25)' (#37) from feat/voice-limit-connect-time into main	2026-10-19 23:34:04 -04:00
Lotus CIandClaude Opus 5.5	d3f389cab1	feat(voice-limit-guard): enforce the limit at connect time too (#25) ... The limit was only checked at token issue, and lk-jwt tokens last an hour, so a token fetched while there was space could be used to connect once the room was full. The reconcile loop now compares each limited room's distinct Matrix users with max_users and RemoveParticipants the people who arrived since the previous sweep, newest first, until it is back within the limit. People already in the call are never removed (lowering a limit mid-call kicks nobody); extra devices of one person never count twice. Fixes #25 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-19 23:30:15 -04:00
jared	f7ce35c636	Merge pull request 'fix(cinny-web): per-client rate limits; own budget for decorations' (#36) from fix/nginx-real-ip-decorations into main	2026-10-19 12:44:26 -04:00
Lotus CIandClaude Opus 5.5	a52d6a89bc	fix(cinny-web): per-client rate limits; own budget for decorations ... Every request reaches LXC 106 from Nginx Proxy Manager (10.10.10.27), and nginx had no realip config, so chat_limit (20 r/s, burst 60, keyed by $binary_remote_addr) was ONE bucket for the whole community: a scanner burst 503ed real users. Trust X-Real-IP from the proxy only, in both server blocks. Decorations (#22) now come from this host, and the profile picker can fetch all 629 thumbnails: ~500 of them 503ed in testing. /decorations/ gets its own zone (100 r/s, burst 700) instead of chat_limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5	2026-10-19 12:41:46 -04:00
jared	adeb872abb	Merge pull request 'feat(cinny-web): serve avatar decorations from chat.lotusguild.org/decorations (#22)' (#35) from feat/decorations-package into main	2026-10-19 12:35:22 -04:00