Compare commits

..
Author SHA1 Message Date
jared 8bf2e6d828 Merge pull request 'fix(matrixbot): bring live ATM10 edits into the repo' (#70) from fix/lotusbot-atm10-live-edits into main
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 11s
Lint / JS (eslint) (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 13s
Lint / Shell (shellcheck) (push) Successful in 11s
Lint / No Hookshot webhook IDs in README (push) Successful in 18s
Lint / Python deps (pip-audit) (push) Successful in 41s
2026-11-19 20:57:48 -05:00
jaredandClaude Opus 5.5 71054d2778 fix(matrixbot): bring live ATM10 edits into the repo
Lint / Shell (shellcheck) (pull_request) Successful in 8s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 10s
Lint / Secret scan (gitleaks) (push) Successful in 12s
Lint / JS (eslint) (pull_request) Successful in 8s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 7s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Secret scan (gitleaks) (pull_request) Successful in 13s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 10s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m2s
Lint / Shell (shellcheck) (push) Successful in 11s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 38s
These changes were made directly on CT151 (2026-10-30 to 11-01) and never
committed, so the auto-deploy triggered by b18aae8 on 2026-11-18 overwrote
them. Restored live on 2026-11-19; this commits them so the next deploy
keeps them.

- !minecraft: 60 s per-user cooldown, audit log line, clear message when the
  Minecraft account doesn't exist
- !minecraft reply: Lotus Minecraft Server / All the Mods 10 8.6 (was the
  old Paper "Lotus SMP, fresh world, hard mode" text)
- !help: compact list with !help <command> for details, since the Discord
  bridge drops messages over 2000 characters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KQ9u9cUsWuUQN791N3JzFG
2026-11-19 20:49:15 -05:00
jared 7c497d2a0a Merge pull request 'docs: large1 → compute-storage-02; no fixed node for HA guests' (#69) from docs/large1-leftovers into main
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Python (ruff) (push) Successful in 5s
Lint / JS (eslint) (push) Successful in 7s
Lint / No Hookshot webhook IDs in README (push) Successful in 19s
Lint / Python deps (pip-audit) (push) Successful in 45s
Lint / Shell (shellcheck) (push) Successful in 8s
2026-11-18 16:22:13 -05:00
jaredandClaude Opus 5.5 b18aae89d9 docs: remove large1 leftovers; no node names for HA guests
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 11s
Lint / Shell (shellcheck) (pull_request) Successful in 12s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 6s
Lint / JS (eslint) (pull_request) Successful in 7s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 9s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Secret scan (gitleaks) (pull_request) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 1m6s
Lint / Python (ruff) (pull_request) Successful in 7s
Lint / Python deps (pip-audit) (pull_request) Successful in 37s
Lint / Shell (shellcheck) (push) Successful in 9s
large1 was renamed to compute-storage-02 on 2026-11-18. Updates text that still
named large1. HA guests no longer carry a fixed node (they move between nodes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 16:15:33 -05:00
jared 3a0a4e15a8 now-playing: Navidrome "Listening to" service for Lotus Chat (cinny #491) (#68)
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 5s
Lint / JS (eslint) (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 11s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 36s
2026-11-16 23:51:07 -05:00
jared b3fec4af2a discord-bridge: check the gateway echo before retrying ambiguous webhook sends (#67)
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 6s
Lint / Python (ruff) (push) Successful in 4s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 4s
Lint / Python deps (pip-audit) (push) Successful in 36s
2026-11-15 21:46:31 -05:00
jared a57c91860b Merge pull request 'Docs: drop stale node names from the infrastructure tables' (#66) from rename/micro1-to-monitor-01 into main
Lint / No secrets in webhook configs (push) Successful in 9s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 7s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 1m28s
2026-11-14 17:57:54 -05:00
jaredandClaude Opus 5.5 c8b7b2b049 Docs: drop stale node names from the infrastructure tables
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 12s
Lint / Shell (shellcheck) (pull_request) Successful in 13s
Lint / Secret scan (gitleaks) (push) Successful in 4s
Lint / JS (eslint) (push) Successful in 11s
Lint / Python (ruff) (push) Successful in 23s
Lint / JS (eslint) (pull_request) Successful in 10s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 5s
Lint / No secrets in webhook configs (pull_request) Successful in 7s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Secret scan (gitleaks) (pull_request) Successful in 11s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 19s
Lint / Python deps (pip-audit) (push) Successful in 1m34s
Lint / Python (ruff) (pull_request) Successful in 6s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m19s
These guests are HA and move between nodes; micro1 no longer exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:46:56 -05:00
jared c2ffb8907d discord-bridge: retry Matrix→Discord sends that failed before reaching Discord (#65)
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 34s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 4s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 4s
Lint / Python deps (pip-audit) (push) Successful in 49s
2026-11-14 14:31:23 -05:00
jared 16eca51e4c hookshot: remove the Lidarr webhook (#64)
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 5s
Lint / Python deps (pip-audit) (push) Successful in 39s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / No Hookshot webhook IDs in README (push) Successful in 5s
Co-authored-by: Jared Vititoe <jjvititoe1@gmail.com>
2026-11-10 13:04:34 -05:00
jared 3c28ec16dd Merge pull request 'docs: background web push is not planned (cinny #202)' (#63) from docs/no-web-push into main
Lint / No secrets in webhook configs (push) Successful in 18s
Lint / Secret scan (gitleaks) (push) Successful in 19s
Lint / Landing page is rendered (matrix (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 11s
Lint / JS (eslint) (push) Successful in 16s
Lint / No Hookshot webhook IDs in README (push) Successful in 32s
Lint / Python deps (pip-audit) (push) Successful in 1m50s
Lint / Shell (shellcheck) (push) Successful in 18s
2026-11-09 19:20:54 -05:00
jaredandClaude Opus 5.5 3e1659224b docs: background web push is not planned (cinny #202)
Lint / No secrets in webhook configs (push) Successful in 11s
Lint / Shell (shellcheck) (pull_request) Successful in 28s
Lint / Landing page is rendered (matrix (push) Successful in 9s
Lint / Landing page is rendered (matrix (pull_request) Successful in 16s
Lint / Secret scan (gitleaks) (push) Successful in 18s
Lint / JS (eslint) (push) Successful in 18s
Lint / Python (ruff) (push) Successful in 25s
Lint / JS (eslint) (pull_request) Successful in 25s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 11s
Lint / No secrets in webhook configs (pull_request) Successful in 12s
Lint / Secret scan (gitleaks) (pull_request) Successful in 12s
Lint / No Hookshot webhook IDs in README (push) Successful in 17s
Lint / Python deps (pip-audit) (push) Successful in 59s
Lint / Python (ruff) (pull_request) Successful in 13s
Lint / Shell (shellcheck) (push) Successful in 20s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m25s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-09 19:12:48 -05:00
jared a73afea6b9 Merge pull request 'minecraft: show players online as the bridge bot's status' (#62) from feat/mc-player-status into main
Lint / No secrets in webhook configs (push) Successful in 9s
Lint / Landing page is rendered (matrix (push) Successful in 13s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / JS (eslint) (push) Successful in 14s
Lint / Python (ruff) (push) Successful in 12s
Lint / Shell (shellcheck) (push) Successful in 15s
Lint / No Hookshot webhook IDs in README (push) Successful in 10s
Lint / Python deps (pip-audit) (push) Successful in 1m11s
2026-11-04 22:42:03 -05:00
jaredandClaude Opus 5.5 2d2ebf64a5 minecraft: show players online as the bridge bot's status
Lint / Shell (shellcheck) (pull_request) Successful in 26s
Lint / Landing page is rendered (matrix (pull_request) Successful in 13s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / JS (eslint) (pull_request) Successful in 23s
Lint / No secrets in webhook configs (pull_request) Successful in 9s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 11s
Lint / Secret scan (gitleaks) (pull_request) Successful in 24s
Lint / JS (eslint) (push) Successful in 23s
Lint / Python (ruff) (push) Successful in 7s
Lint / Python deps (pip-audit) (push) Successful in 56s
Lint / Python (ruff) (pull_request) Successful in 12s
Lint / Shell (shellcheck) (push) Successful in 27s
Lint / No Hookshot webhook IDs in README (push) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m43s
A small service on CT 154 pings the Lotus Minecraft server and sets the result
as the Matrix status of @lotus-mc-bridge, re-sent every 20 s because the mod
syncs with set_presence=offline and Synapse would otherwise drop it to offline.
Uses the mod's own token, so no new account or secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-04 22:37:55 -05:00
jared c793d4fb29 Merge pull request 'cinny: point the status banner at Kuma's public page' (#61) from fix/status-banner-public-page into main
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Landing page is rendered (matrix (push) Successful in 15s
Lint / Secret scan (gitleaks) (push) Successful in 17s
Lint / JS (eslint) (push) Successful in 13s
Lint / Python (ruff) (push) Successful in 9s
Lint / Shell (shellcheck) (push) Successful in 30s
Lint / No Hookshot webhook IDs in README (push) Successful in 9s
Lint / Python deps (pip-audit) (push) Successful in 1m44s
2026-11-04 22:21:53 -05:00
jaredandClaude Opus 5.5 5268156e48 cinny: point the status banner at Kuma's public page
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Shell (shellcheck) (pull_request) Successful in 22s
Lint / No secrets in webhook configs (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / JS (eslint) (pull_request) Successful in 30s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 15s
Lint / No secrets in webhook configs (pull_request) Successful in 15s
Lint / Secret scan (gitleaks) (pull_request) Successful in 9s
Lint / JS (eslint) (push) Successful in 11s
Lint / Python (ruff) (push) Successful in 6s
Lint / Python (ruff) (pull_request) Successful in 8s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m9s
Lint / Shell (shellcheck) (push) Successful in 13s
Lint / No Hookshot webhook IDs in README (push) Successful in 16s
Lint / Python deps (pip-audit) (push) Successful in 1m34s
The "matrix" status page was merged into "public" on 2026-10-11, so the banner
has read a 404 and stayed silent since. Kuma's Matrix group on "public" is now
split into Homeserver / Voice calls / Discord, and notices are scoped with
noticePrefix "Matrix:" (cinny #486, already live), so other services'
maintenance and incidents don't reach Lotus users.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-04 21:53:23 -05:00
jared dde36cdcd2 Merge pull request 'discord-bridge: Discord presence into Matrix ghosts' (#60) from feat/discord-presence into main
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Landing page is rendered (matrix (push) Successful in 17s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / JS (eslint) (push) Successful in 27s
Lint / Python (ruff) (push) Successful in 21s
Lint / Shell (shellcheck) (push) Successful in 29s
Lint / No Hookshot webhook IDs in README (push) Successful in 17s
Lint / Python deps (pip-audit) (push) Successful in 1m15s
2026-11-04 19:41:07 -05:00
jaredandClaude Opus 5.5 f8f0ca4549 discord-bridge: Discord presence into Matrix ghosts
Lint / Shell (shellcheck) (pull_request) Successful in 11s
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 17s
Lint / Secret scan (gitleaks) (push) Successful in 31s
Lint / JS (eslint) (pull_request) Successful in 10s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 6s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Landing page is rendered (matrix (pull_request) Successful in 6s
Lint / Secret scan (gitleaks) (pull_request) Successful in 13s
Lint / JS (eslint) (push) Successful in 18s
Lint / Python (ruff) (push) Successful in 16s
Lint / Python (ruff) (pull_request) Successful in 7s
Lint / Shell (shellcheck) (push) Successful in 26s
Lint / No Hookshot webhook IDs in README (push) Successful in 21s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m54s
Lint / Python deps (pip-audit) (push) Successful in 1m37s
mautrix-discord doesn't bridge presence. lotus_presence.py watches the Discord
server (a second gateway session on the bridge's own bot, Guilds + Presence
intents only) and sets presence and a status message on each @discord_* ghost:
online / idle / DND / offline -> online / unavailable / msc3026 busy / offline;
custom status and the first activity become the status message.

Its own appservice registration is non-exclusive on the ghost namespace, so it
can update ghosts but not create them. Synapse drops an unrefreshed user to
offline after about a minute, so live states are re-sent every 20 s; if the
service dies, ghosts go offline rather than stale. The installer generates the
credentials on the server and doesn't restart Synapse. Statuses and activities
are never logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-04 19:36:38 -05:00
jared ed1300f146 Merge pull request 'feat(synapse): nightly state compression on CT 109' (#59) from feat/synapse-state-compressor into main
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 5s
Lint / Shell (shellcheck) (push) Successful in 9s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 39s
2026-11-03 19:10:33 -05:00
Lotus CIandClaude Opus 5.5 b2877323c3 feat(synapse): nightly state compression on CT 109
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Shell (shellcheck) (pull_request) Successful in 8s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Python (ruff) (push) Successful in 5s
Lint / JS (eslint) (pull_request) Successful in 7s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 5s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Secret scan (gitleaks) (pull_request) Successful in 7s
Lint / JS (eslint) (push) Successful in 6s
Lint / No Hookshot webhook IDs in README (push) Successful in 12s
Lint / Shell (shellcheck) (push) Successful in 9s
Lint / Python deps (pip-audit) (push) Successful in 55s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 42s
The Synapse database grew from 2.2 GB (after the July cleanup) to
6.2 GB, almost all of it state_groups_state from large federated rooms
with one or two local members.

synapse_auto_compressor (matrix-org/rust-synapse-compress-state, built
from main) now runs nightly at 04:30, after the 03:00 vzdump backup.
It verifies compressed state against the original, writes each state
group in its own transaction, skips chunks that would not shrink, and
resumes from stored progress. The initial backlog took ~8 minutes with
no Synapse errors: state_groups_state 11.0 M -> 4.25 M rows (-61%).

Also documents the new "Synapse Database Growing Fast" Grafana alert
(> 500 MB in 7 days).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 19:08:01 -05:00
jared ded4a30530 Merge pull request 'docs: all 13 Hookshot webhooks rotated' (#58) from docs/rotated-remaining-webhooks into main
Lint / No secrets in webhook configs (push) Successful in 8s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 25s
Lint / JS (eslint) (push) Successful in 13s
Lint / Python (ruff) (push) Successful in 10s
Lint / Shell (shellcheck) (push) Successful in 14s
Lint / No Hookshot webhook IDs in README (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m10s
2026-11-03 17:03:09 -05:00
Lotus CIandClaude Opus 5.5 f3eb3ea406 docs: all 13 Hookshot webhooks rotated; old IDs in history are dead
Lint / No secrets in webhook configs (push) Successful in 14s
Lint / Landing page is rendered (matrix (push) Successful in 24s
Lint / Secret scan (gitleaks) (push) Successful in 12s
Lint / Shell (shellcheck) (pull_request) Successful in 15s
Lint / JS (eslint) (push) Successful in 17s
Lint / Python (ruff) (push) Successful in 22s
Lint / JS (eslint) (pull_request) Successful in 17s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 16s
Lint / Landing page is rendered (matrix (pull_request) Successful in 7s
Lint / No secrets in webhook configs (pull_request) Successful in 8s
Lint / Secret scan (gitleaks) (pull_request) Successful in 13s
Lint / Shell (shellcheck) (push) Successful in 17s
Lint / No Hookshot webhook IDs in README (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m7s
Lint / Python (ruff) (pull_request) Successful in 15s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 16:59:31 -05:00
jared 7b01a9fcdc Merge pull request 'docs: call stack upgraded (lk-jwt-service 0.9.0, LiveKit 1.13.11)' (#57) from docs/call-stack-upgraded into main
Lint / No secrets in webhook configs (push) Successful in 17s
Lint / Landing page is rendered (matrix (push) Successful in 17s
Lint / Secret scan (gitleaks) (push) Successful in 20s
Lint / JS (eslint) (push) Successful in 24s
Lint / Python (ruff) (push) Successful in 23s
Lint / Shell (shellcheck) (push) Successful in 18s
Lint / No Hookshot webhook IDs in README (push) Successful in 17s
Lint / Python deps (pip-audit) (push) Successful in 1m11s
2026-11-03 16:42:04 -05:00
Lotus CIandClaude Opus 5.5 bc504a881d docs: call stack upgraded to lk-jwt-service 0.9.0 and LiveKit 1.13.11
Lint / No secrets in webhook configs (push) Successful in 22s
Lint / Landing page is rendered (matrix (push) Successful in 14s
Lint / Secret scan (gitleaks) (push) Successful in 28s
Lint / Shell (shellcheck) (pull_request) Successful in 22s
Lint / Landing page is rendered (matrix (pull_request) Successful in 22s
Lint / JS (eslint) (push) Successful in 1m31s
Lint / Python (ruff) (push) Successful in 14s
Lint / JS (eslint) (pull_request) Successful in 17s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 19s
Lint / No secrets in webhook configs (pull_request) Successful in 16s
Lint / Secret scan (gitleaks) (pull_request) Successful in 25s
Lint / Shell (shellcheck) (push) Successful in 1m31s
Lint / No Hookshot webhook IDs in README (push) Successful in 13s
Lint / Python deps (pip-audit) (push) Successful in 1m7s
Lint / Python (ruff) (pull_request) Successful in 15s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m25s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 16:36:28 -05:00
jared 89291f8707 Merge pull request 'docs: note the 2026-11-03 webhook rotation' (#56) from docs/rotated-webhook-prefixes into main
Lint / No secrets in webhook configs (push) Successful in 10s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 14s
Lint / JS (eslint) (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 18s
Lint / No Hookshot webhook IDs in README (push) Successful in 10s
Lint / Python deps (pip-audit) (push) Successful in 55s
2026-11-03 00:57:42 -05:00
Lotus CIandClaude Opus 5.5 9eca253229 docs: note the 2026-11-03 webhook rotation (Grafana, Proxmox, Uptime Kuma, Ci-Alert)
Lint / Landing page is rendered (matrix (push) Successful in 13s
Lint / Secret scan (gitleaks) (push) Successful in 17s
Lint / Shell (shellcheck) (pull_request) Successful in 15s
Lint / Landing page is rendered (matrix (pull_request) Successful in 9s
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / JS (eslint) (push) Successful in 38s
Lint / Python (ruff) (push) Successful in 25s
Lint / JS (eslint) (pull_request) Successful in 16s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 7s
Lint / No secrets in webhook configs (pull_request) Successful in 7s
Lint / Secret scan (gitleaks) (pull_request) Successful in 24s
Lint / No Hookshot webhook IDs in README (push) Successful in 36s
Lint / Shell (shellcheck) (push) Successful in 16s
Lint / Python deps (pip-audit) (push) Successful in 1m15s
Lint / Python (ruff) (pull_request) Successful in 9s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m8s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 00:51:35 -05:00
jared a06ebfe6f6 Merge pull request 'docs: stop publishing Hookshot webhook IDs; record Hookshot + call-stack upgrades' (#55) from docs/hookshot-ids-and-upgrades into main
Lint / No secrets in webhook configs (push) Successful in 10s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / JS (eslint) (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 5s
Lint / Shell (shellcheck) (push) Successful in 13s
Lint / No Hookshot webhook IDs in README (push) Successful in 12s
Lint / Python deps (pip-audit) (push) Successful in 39s
2026-11-03 00:36:18 -05:00
Lotus CIandClaude Opus 5.5 6e41477a0b docs: stop publishing Hookshot webhook IDs; record the Hookshot and call-stack upgrades
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / Shell (shellcheck) (pull_request) Successful in 12s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 10s
Lint / JS (eslint) (pull_request) Successful in 10s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 12s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Secret scan (gitleaks) (pull_request) Successful in 7s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / No Hookshot webhook IDs in README (push) Successful in 5s
Lint / Python deps (pip-audit) (push) Successful in 56s
Lint / Python (ruff) (pull_request) Successful in 6s
Lint / Python deps (pip-audit) (pull_request) Successful in 39s
- The README's webhook table listed every Hookshot webhook's full UUID.
  The UUID is the webhook's only secret and this repo is public, so
  anyone could post into Spam and Stuff. Shorten them to 8 characters
  and add CI (readme-no-webhook-ids) rejecting full UUIDs in README.md.
  They remain in git history until rotated.
- Hookshot is now 7.5.0: how it's built (pnpm 7.4.1+, Node 22 limit
  of 7.6.0) and how to swap a new build in.
- Planned lk-jwt-service 0.9.0 / LiveKit 1.13.11 upgrade: the LiveKit
  webhook delegated leave needs, the prometheus config key, and
  restart caveats, from local testing.
- Graceful restart: describe the fixed check (#54).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 00:34:03 -05:00
jared 8a0e16cabd Merge pull request 'fix(livekit): graceful restart could restart LiveKit in the middle of a call' (#54) from fix/livekit-graceful-restart-detection into main
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / Python deps (pip-audit) (push) Successful in 49s
2026-11-03 00:28:12 -05:00
Lotus CIandClaude Opus 5.5 d9f023762e fix(livekit): graceful restart could restart LiveKit in the middle of a call
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Shell (shellcheck) (pull_request) Successful in 10s
Lint / JS (eslint) (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 6s
Lint / JS (eslint) (pull_request) Successful in 8s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Secret scan (gitleaks) (pull_request) Successful in 11s
Lint / Shell (shellcheck) (push) Successful in 13s
Lint / Python deps (pip-audit) (push) Successful in 41s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 56s
livekit-graceful-restart.sh decided "no active calls" by counting
established TCP connections on :7881 with `ss ... state established |
grep -c ESTAB`. Two bugs:

- With a `state` filter ss omits the State column, so `grep -c ESTAB`
  is always 0: the check never saw anything.
- :7881 is only LiveKit's ICE-TCP media fallback. Signalling is on
  :7880 and media is normally UDP.

Reproduced in a two-person call on the local stack: the old check
counted 0 and would have restarted. Now it waits while LiveKit's own
livekit_participant_total is above 0 or any signalling WebSocket on
:7880 is open, and also waits if the metrics can't be read.

The deploy hook now installs the script when it changes (it never
did, so edits to it never reached the server).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 00:25:58 -05:00
jared d32c74052c Merge pull request 'voice-bridge: leave Discord within ~8-18 s of the last Matrix person leaving' (#53) from voice-bridge-timing into main
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 4s
Lint / JS (eslint) (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 7s
Lint / Python deps (pip-audit) (push) Failing after 5m4s
2026-10-29 21:12:00 -04:00
Lotus CIandClaude Opus 5.5 0716e2acd8 voice-bridge: leave Discord within ~8-18 s of the last Matrix person leaving
Lint / Landing page is rendered (matrix (push) Successful in 3s
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 12s
Lint / Shell (shellcheck) (pull_request) Successful in 14s
Lint / Landing page is rendered (matrix (pull_request) Successful in 14s
Lint / JS (eslint) (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 6s
Lint / JS (eslint) (pull_request) Successful in 9s
Lint / No secrets in webhook configs (pull_request) Successful in 7s
Lint / Python deps (pip-audit) (push) Successful in 1m16s
Lint / Secret scan (gitleaks) (pull_request) Successful in 28s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / Python (ruff) (pull_request) Successful in 12s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m18s
Upstream defaults (12 s leave grace + 90 s anchor linger) kept the Discord bot in the channel for about a minute after the call ended on the Matrix side.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-29 21:09:01 -04:00
jared 8eb3201b38 Merge pull request 'docs: voice bridge live — bot profile, room setup, upgrade steps' (#52) from docs/voice-bridge-live into main
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 5s
Lint / Shell (shellcheck) (push) Successful in 9s
Lint / Python deps (pip-audit) (push) Successful in 34s
2026-10-29 13:24:35 -04:00
Lotus CIandClaude Opus 5.5 811ae9d2cc docs: voice bridge live — bot profile, room setup, upgrade steps
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Shell (shellcheck) (pull_request) Successful in 8s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / JS (eslint) (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 5s
Lint / JS (eslint) (pull_request) Successful in 10s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Secret scan (gitleaks) (pull_request) Successful in 9s
Lint / Shell (shellcheck) (push) Successful in 7s
Lint / Python deps (pip-audit) (push) Successful in 39s
Lint / Python (ruff) (pull_request) Successful in 17s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m3s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-29 13:22:36 -04:00
jared b2cab883d6 voice-bridge: Discord voice ↔ Lotus voice room (patched nether-voicebridge)
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 10s
Lint / Python (ruff) (push) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / Python deps (pip-audit) (push) Successful in 32s
2026-10-29 12:56:04 -04:00
Lotus CIandClaude Opus 5.5 46d0e98424 voice-bridge: Discord voice <-> Lotus voice room (patched nether-voicebridge)
Lint / No secrets in webhook configs (push) Successful in 18s
Lint / Landing page is rendered (matrix (push) Successful in 18s
Lint / Secret scan (gitleaks) (push) Successful in 14s
Lint / Shell (shellcheck) (pull_request) Successful in 9s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / JS (eslint) (push) Successful in 2m1s
Lint / Python (ruff) (push) Successful in 25s
Lint / JS (eslint) (pull_request) Successful in 9s
Lint / No secrets in webhook configs (pull_request) Successful in 7s
Lint / Secret scan (gitleaks) (pull_request) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 2m0s
Lint / Python deps (pip-audit) (push) Successful in 1m7s
Lint / Python (ruff) (pull_request) Successful in 6s
Lint / Python deps (pip-audit) (pull_request) Successful in 30s
Production setup for the Discord voice bridge on LXC 151: config and
appservice-registration templates (secrets rendered on the server), an
idempotent install script (one Synapse restart on first install, creates
the bot account so it can be invited), and a hardened systemd unit.

voice-limit-guard: VOICE_LIMIT_EXEMPT_USERS lets the bridge's own bot
listen in a call without taking a slot; relayed Discord speakers still
count. Unit tests added (61 pass); verified live on the dev stack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-29 12:51:56 -04:00
jared db2f523028 Merge pull request 'docs: bot profiles (names, avatars, where each is set)' (#50) from docs/bot-profiles into main
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / JS (eslint) (push) Successful in 6s
Lint / Python (ruff) (push) Successful in 11s
Lint / Shell (shellcheck) (push) Successful in 7s
Lint / Python deps (pip-audit) (push) Successful in 42s
2026-10-29 01:32:54 -04:00
Lotus CIandClaude Opus 5.5 b06f422837 docs: bot profiles (names, avatars, where each is set)
Lint / No secrets in webhook configs (push) Successful in 10s
Lint / Landing page is rendered (matrix (push) Successful in 9s
Lint / Secret scan (gitleaks) (push) Successful in 10s
Lint / Shell (shellcheck) (pull_request) Successful in 21s
Lint / Landing page is rendered (matrix (pull_request) Successful in 11s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 16s
Lint / JS (eslint) (pull_request) Successful in 12s
Lint / No secrets in webhook configs (pull_request) Successful in 11s
Lint / Secret scan (gitleaks) (pull_request) Successful in 9s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m25s
Lint / Python (ruff) (pull_request) Successful in 9s
Lint / Python deps (pip-audit) (pull_request) Successful in 36s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-29 01:30:24 -04:00
jared f2d8a12aa7 Merge pull request 'docs: the Discord bot login stays on @jared' (#49) from docs/discord-login-owner into main
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 5s
Lint / Shell (shellcheck) (push) Successful in 15s
Lint / Python deps (pip-audit) (push) Successful in 1m53s
2026-10-28 22:19:19 -04:00
Lotus CIandClaude Opus 5.5 7cf68b6850 docs: the Discord bot login stays on @jared
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 8s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Shell (shellcheck) (pull_request) Successful in 19s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 7s
Lint / JS (eslint) (pull_request) Successful in 17s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Secret scan (gitleaks) (pull_request) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 19s
Lint / Python deps (pip-audit) (push) Successful in 53s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m45s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-28 22:14:37 -04:00
jared 894dd5277e Merge pull request 'discord-bridge: bot profile, down alert, login-owner account' (#48) from discord-bridge-profile into main
Lint / No secrets in webhook configs (push) Successful in 8s
Lint / Landing page is rendered (matrix (push) Successful in 8s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 15s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 55s
2026-10-28 22:00:30 -04:00
Lotus CIandClaude Opus 5.5 0386474118 discord-bridge: bot profile, down alert, login-owner account
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Shell (shellcheck) (pull_request) Successful in 11s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 6s
Lint / JS (eslint) (pull_request) Successful in 8s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (push) Successful in 54s
Lint / Secret scan (gitleaks) (pull_request) Successful in 8s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / Python (ruff) (pull_request) Successful in 10s
Lint / Python deps (pip-audit) (pull_request) Successful in 41s
Add the Discord-logo avatar and a script that uploads it as the bridge
bot and sets the display name to "Discord Bridge". Document the new
"Discord Bridge Down" Grafana alert, and why the bot login belongs on a
dedicated non-chatting account: whoever holds it sends to Discord as the
bot instead of through the relay webhook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-28 21:58:34 -04:00
jared b8a088d2a6 Merge pull request 'discord-bridge: Matrix avatars on Discord, catch up missed messages' (#47) from discord-bridge-avatars into main
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 11s
Lint / Python (ruff) (push) Successful in 7s
Lint / Shell (shellcheck) (push) Successful in 11s
Lint / Python deps (pip-audit) (push) Successful in 52s
2026-10-28 21:53:34 -04:00
Lotus CIandClaude Opus 5.5 8f35ab23a3 discord-bridge: Matrix avatars on Discord, catch up missed messages
Lint / Landing page is rendered (matrix (push) Successful in 9s
Lint / No secrets in webhook configs (push) Successful in 9s
Lint / Secret scan (gitleaks) (push) Successful in 15s
Lint / Shell (shellcheck) (pull_request) Successful in 14s
Lint / Landing page is rendered (matrix (pull_request) Successful in 7s
Lint / JS (eslint) (push) Successful in 15s
Lint / Python (ruff) (push) Successful in 11s
Lint / JS (eslint) (pull_request) Successful in 8s
Lint / No secrets in webhook configs (pull_request) Successful in 7s
Lint / Secret scan (gitleaks) (pull_request) Successful in 9s
Lint / Shell (shellcheck) (push) Successful in 15s
Lint / Python deps (pip-audit) (push) Successful in 1m6s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 54s
Set bridge.public_address so relayed messages carry the sender's avatar
(served through a new NPM location for /mautrix-discord/avatar/), listen
on 0.0.0.0 so NPM can reach it, and backfill all Discord messages missed
while the bridge was down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-28 21:51:02 -04:00
jared 1e4922d6a3 Merge pull request 'docs: Discord bridge is live; linking notes' (#46) from docs/discord-bridge-live into main
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 47s
Lint / Shell (shellcheck) (push) Successful in 9s
2026-10-28 21:48:00 -04:00
Lotus CIandClaude Opus 5.5 0587f18063 docs: Discord bridge is live; linking notes
Lint / No secrets in webhook configs (push) Successful in 8s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / Shell (shellcheck) (pull_request) Successful in 12s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / JS (eslint) (push) Successful in 9s
Lint / Python (ruff) (push) Successful in 9s
Lint / JS (eslint) (pull_request) Successful in 11s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (push) Successful in 1m33s
Lint / Secret scan (gitleaks) (pull_request) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / Python (ruff) (pull_request) Successful in 6s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m18s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-28 21:45:21 -04:00
jared 7c10c9591e Merge pull request 'Add mautrix-discord bridge config and installer' (#44) from discord-bridge into main
Lint / No secrets in webhook configs (push) Successful in 7s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 10s
Lint / JS (eslint) (push) Successful in 10s
Lint / Python (ruff) (push) Successful in 17s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / Python deps (pip-audit) (push) Successful in 1m47s
2026-10-28 21:25:52 -04:00
jared 3ad2fef28e Merge branch 'main' into discord-bridge
Lint / No secrets in webhook configs (push) Successful in 7s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / Shell (shellcheck) (pull_request) Successful in 10s
Lint / Landing page is rendered (matrix (pull_request) Successful in 7s
Lint / JS (eslint) (push) Successful in 31s
Lint / Python (ruff) (push) Successful in 5s
Lint / JS (eslint) (pull_request) Successful in 12s
Lint / No secrets in webhook configs (pull_request) Successful in 8s
Lint / Secret scan (gitleaks) (pull_request) Successful in 10s
Lint / Shell (shellcheck) (push) Successful in 14s
Lint / Python (ruff) (pull_request) Successful in 7s
Lint / Python deps (pip-audit) (push) Successful in 1m19s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m19s
2026-10-28 21:21:59 -04:00
jared e16ce42785 Merge pull request 'ci(pip-audit): audit on Python 3.13 instead of ignoring filelock CVEs' (#45) from ci/pip-audit-py313 into main
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 14s
Lint / JS (eslint) (push) Successful in 10s
Lint / Python (ruff) (push) Successful in 12s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / Python deps (pip-audit) (push) Successful in 1m31s
2026-10-28 21:21:37 -04:00
Lotus CIandClaude Opus 5.5 246774e3eb ci(pip-audit): audit on Python 3.13 instead of ignoring filelock CVEs
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 4s
Lint / Secret scan (gitleaks) (push) Successful in 11s
Lint / Shell (shellcheck) (pull_request) Successful in 15s
Lint / Landing page is rendered (matrix (pull_request) Successful in 8s
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (ruff) (push) Successful in 11s
Lint / JS (eslint) (pull_request) Successful in 11s
Lint / No secrets in webhook configs (pull_request) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m33s
Lint / Secret scan (gitleaks) (pull_request) Successful in 10s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / Python (ruff) (pull_request) Successful in 8s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m0s
The audit env ran Python 3.10, which can't install filelock 4.2+, so
every new filelock CVE in pip-audit's own CacheControl dependency broke
the job and had to be ignored by ID (11 more landed today). Audit on
3.13, which matches the bot's runtime on LXC 151 (Debian trixie), so
the fixed filelock installs and the per-CVE ignores can go. Also scan
discord-bridge/ for secrets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-28 21:18:13 -04:00
Lotus CIandClaude Opus 5.5 3e6f210f46 Add mautrix-discord bridge config and installer
Lint / No secrets in webhook configs (push) Successful in 7s
Lint / Landing page is rendered (matrix (push) Successful in 8s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Shell (shellcheck) (pull_request) Successful in 15s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / JS (eslint) (push) Successful in 11s
Lint / Python (ruff) (push) Successful in 12s
Lint / JS (eslint) (pull_request) Successful in 14s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Secret scan (gitleaks) (pull_request) Successful in 9s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / Python deps (pip-audit) (push) Failing after 1m1s
Lint / Python (ruff) (pull_request) Successful in 7s
Lint / Python deps (pip-audit) (pull_request) Failing after 1m54s
Bot-login Discord bridge on LXC 151: config template (DB password is
filled in on the server), systemd unit, and a one-time install script
run from the Proxmox node that sets up the PostgreSQL role/database,
installs the checksum-verified v0.7.7 binary, generates the appservice
registration, adds it to Synapse and restarts Synapse once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-10-28 21:08:45 -04:00
jared 3213f8c41b Merge pull request 'docs: Synapse 1.164.1' (#43) from docs/synapse-1.164.1 into main
Lint / Landing page is rendered (matrix (push) Successful in 11s
Lint / No secrets in webhook configs (push) Successful in 13s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / JS (eslint) (push) Successful in 15s
Lint / Python (ruff) (push) Successful in 12s
Lint / Python deps (pip-audit) (push) Successful in 49s
Lint / Shell (shellcheck) (push) Successful in 15s
2026-10-28 15:05:11 -04:00
40 changed files with 3521 additions and 122 deletions

No files matched your search

+23 -16
View File
@@ -31,6 +31,20 @@ jobs:
- name: Run ESLint
run: npx eslint --ext .js hookshot/
readme-no-webhook-ids:
name: No Hookshot webhook IDs in README
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
# A Hookshot webhook URL's UUID is its only secret, and this repo is public.
- name: Reject full UUIDs
run: |
if grep -nE '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}' README.md; then
echo "::error::README.md contains a full UUID. Webhook IDs are secrets: list only the first 8 characters."
exit 1
fi
hooks-no-secrets:
name: No secrets in webhook configs
runs-on: ubuntu-latest
@@ -71,10 +85,10 @@ jobs:
mv ruff /usr/local/bin/ruff
- name: Check syntax errors
run: ruff check matrixbot/ --select E9,F63,F7,F82 --output-format=github
run: ruff check matrixbot/ now-playing/ --select E9,F63,F7,F82 --output-format=github
- name: Run full lint
run: ruff check matrixbot/ --output-format=github
run: ruff check matrixbot/ now-playing/ --output-format=github
python-audit:
name: Python deps (pip-audit)
@@ -82,28 +96,21 @@ jobs:
steps:
- uses: actions/checkout@v3
- name: Install Python 3.10 and pip-audit
- name: Install Python 3.13 and pip-audit
run: |
# Debian Bullseye only ships Python 3.9; use a prebuilt standalone binary
curl -sSL "https://github.com/indygreg/python-build-standalone/releases/download/20241002/cpython-3.10.15+20241002-x86_64-unknown-linux-gnu-install_only.tar.gz" \
# Debian Bullseye only ships Python 3.9; use a prebuilt standalone binary.
# 3.13 matches the bot's runtime on LXC 151 (Debian trixie).
curl -sSL "https://github.com/astral-sh/python-build-standalone/releases/download/20250317/cpython-3.13.2+20250317-x86_64-unknown-linux-gnu-install_only.tar.gz" \
| tar -xz -C /opt
# Install bot deps + pip-audit into the same env; --local below avoids
# pip-audit creating an internal venv (ensurepip fails on standalone builds)
/opt/python/bin/pip install --upgrade pip setuptools
/opt/python/bin/pip install pip-audit -r matrixbot/requirements.txt
/opt/python/bin/pip install --upgrade pip-audit -r matrixbot/requirements.txt
- name: Audit matrixbot dependencies
# --local scans the env without spawning a venv (required for standalone Python)
# CVE-2026-3219 is in pip itself, not our code — ignore it explicitly.
# The filelock CVEs (fixed in 4.2.1) are in pip-audit's own dependency
# (CacheControl[filecache]); the bot doesn't use filelock, and 4.2+
# needs Python 3.11, which this 3.10 audit env can't install.
run: >-
/opt/python/bin/pip-audit --local
--ignore-vuln CVE-2026-3219
--ignore-vuln CVE-2026-112906 --ignore-vuln CVE-2026-112907
--ignore-vuln CVE-2026-112913 --ignore-vuln CVE-2026-112914
--ignore-vuln CVE-2026-112937 --ignore-vuln CVE-2026-112940
run: /opt/python/bin/pip-audit --local --ignore-vuln CVE-2026-3219
secret-scan:
name: Secret scan (gitleaks)
@@ -124,7 +131,7 @@ jobs:
run: |
# Scan application code directories — deploy/ is excluded because
# it contains intentional Gitea webhook HMAC secrets in hooks-lxc*.json
for dir in matrixbot/ hookshot/ .gitea/ systemd/ cinny/ landing/; do
for dir in matrixbot/ hookshot/ .gitea/ systemd/ cinny/ landing/ discord-bridge/; do
[ -d "$dir" ] || continue
gitleaks detect --source "$dir" --no-git --config .gitleaks.toml \
--redact --exit-code 1
+306 -35
View File
@@ -26,6 +26,13 @@ matrix/
│ └── index.html # matrix.lotusguild.org landing page
├── draupnir/
│ └── production.yaml # Draupnir config (access token is redacted — see rotation docs below)
├── discord-bridge/
│ ├── config.yaml # mautrix-discord config (DB password filled in on the server, never committed)
│ └── install.sh # One-time installer, run on the Proxmox node hosting LXC 109 + 151
├── now-playing/ # "Listening to" service for Lotus Chat (cinny #491), LXC 151 :8072
│ ├── lotus_now_playing.py
│ ├── test_lotus_now_playing.py
│ └── install.sh
├── deploy/ # Auto-deployment infrastructure
│ ├── lxc151-hookshot.sh # Deploy script for LXC 151 (matrix/hookshot/livekit)
│ ├── lxc106-cinny.sh # Deploy script for LXC 106 (cinny)
@@ -41,6 +48,8 @@ matrix/
├── livekit-graceful-restart.service # oneshot — checks pending restart flag
├── livekit-graceful-restart.timer # Runs every 5 min
├── draupnir.service
├── mautrix-discord.service # Discord bridge (LXC 151)
├── lotus-now-playing.service # "Listening to" service (LXC 151)
└── cinny-upstream-check.cron # Installed to /etc/cron.d/ on LXC 106 — runs daily at noon
```
@@ -59,7 +68,7 @@ matrix/
| NPM | 10.10.10.27 | 139 | — | — | Nginx Proxy Manager + matrix landing page |
| Authelia | 10.10.10.36 | 167 | — | — | SSO/OIDC provider |
| LLDAP | 10.10.10.39 | 147 | — | — | LDAP user directory |
| Uptime Kuma | 10.10.10.25 | 101 | — | — | Uptime monitoring (micro1 node) |
| Uptime Kuma | 10.10.10.25 | 101 | — | — | Uptime monitoring |
**Key paths on Synapse LXC (151):**
- Synapse config: `/etc/matrix-synapse/homeserver.yaml`
@@ -187,9 +196,14 @@ Killing livekit-server while a call is active drops everyone. Instead:
1. Deploy to LXC 151 copies the new `livekit-server.service` and sets a `/run/livekit-restart-pending` flag
2. `livekit-graceful-restart.timer` runs every 5 minutes
3. The timer script counts established TCP connections on port 7881 (`ss -tn state established`)
4. If zero connections → restarts livekit-server and clears the flag
5. If connections exist → logs and exits, retries in 5 minutes
3. The timer script reads LiveKit's own `livekit_participant_total` (`:6789/metrics`) and counts open
signalling WebSockets on :7880. If the metrics can't be read it assumes a call is active.
4. Zero participants and zero signalling connections → restarts livekit-server and clears the flag
5. Otherwise → logs and exits, retries in 5 minutes
Until 2026-11-03 it counted `ESTAB` lines on :7881, which was always 0 (an `ss` state filter drops the
State column, and :7881 is only the ICE-TCP fallback), so it would have restarted mid-call (matrix #54).
To stage a restart by hand, for example after replacing the binary: `touch /run/livekit-restart-pending`.
---
@@ -286,6 +300,7 @@ The token in `draupnir/production.yaml` in this repo is **intentionally redacted
| 9002 | Hookshot bridge (appservice) | 127.0.0.1 |
| 9003 | Hookshot webhooks | 0.0.0.0 |
| 9004 | Hookshot metrics | 0.0.0.0 |
| 29334 | mautrix-discord (appservice + avatar proxy for NPM) | 0.0.0.0 |
| 9100 | node_exporter | 0.0.0.0 |
| 9101 | matrix-admin exporter | 0.0.0.0 |
| 9500 | webhook (auto-deploy) | 0.0.0.0 |
@@ -294,6 +309,7 @@ The token in `draupnir/production.yaml` in this repo is **intentionally redacted
| 7881 | LiveKit RTC TCP | 0.0.0.0 |
| 8070 | voice-limit-guard (fronts lk-jwt-service) | 0.0.0.0 |
| 8071 | lk-jwt-service (behind guard) | 127.0.0.1 (matrix #24; was 0.0.0.0) |
| 8072 | lotus-now-playing ("Listening to", cinny #491) | 0.0.0.0 |
| 8080 | synapse-admin (nginx) | 0.0.0.0 |
| 3478 | coturn STUN/TURN | 0.0.0.0 |
| 5349 | coturn TURNS/TLS | 0.0.0.0 |
@@ -325,6 +341,7 @@ The token in `draupnir/production.yaml` in this repo is **intentionally redacted
| Commands | `!ou56mVZQ8ZB7AhDYPmBV5_BR28WMZ4x5zwZkPCqjq1s` | restricted (Space members) |
| Memes | `!GK6v5cLEEnowIooQJv5jECfISUjADjt8aKhWv9VbG5U` | restricted (Space members) |
| Music | `!ktQu0gavhjpCMkgxk8SYdb6mnJRY-u7mY7_KfksV0SU` | restricted (Space members) |
| Discord bridge (Discord channel `1565159372122882070`) | `!hm4ghdqvSYMokV30Ow-0TKK5hMZRSXfDiHOIzrK-c_A` | invite |
| Voice Room | `!ARbRFSPNp2U0MslWTBGoTT3gbmJJ25dPRL6enQntvPo` | restricted (Space members) |
| Management | `!mEvR5fe3jMmzwd-FwNygD72OY_yu8H3UP_N-57oK7MI` | invite |
| Cool Kids | `!R7DT3QZHG9P8QQvX6zsZYxjkKgmUucxDz_n31qNrC94` | invite |
@@ -337,34 +354,40 @@ The token in `draupnir/production.yaml` in this repo is **intentionally redacted
---
## Webhook Integrations (matrix-hookshot 7.3.2)
## Webhook Integrations (matrix-hookshot 7.5.0)
Generic webhooks bridged into **Spam and Stuff**.
Each service gets its own virtual user (`@hookshot_<service>`) with a unique avatar.
Webhook URL format: `https://matrix.lotusguild.org/webhook/<uuid>`
> The UUID **is** the secret: anyone who has it can post into the room. This repo is public, so
> the table lists only the first 8 characters (enough to match Hookshot's logs); the full URLs
> live in each sender's settings. Full UUIDs were in this file from 2026-02-20 to 2026-11-03,
> so the IDs below are exposed in git history until rotated (see *Webhook secrets — rotating*).
> All 13 were rotated on 2026-11-03, so the IDs in git history are dead (they return 404).
> CI (`readme-no-webhook-ids`) rejects full UUIDs in this README.
| Service | Webhook UUID | Notes |
|---------|-------------|-------|
| Grafana | `df4a1302-2d62-4a01-b858-fb56f4d3781a` | Unified alerting contact point |
| Proxmox | `9b3eafe5-7689-4011-addd-c466e524661d` | Notification system (8.1+), Discord embed format |
| Sonarr | `aeffc311-0686-42cb-9eeb-6757140c072e` | All event types |
| Radarr | `34913454-c1ac-4cda-82ea-924d4a9e60eb` | All event types |
| Readarr | `e57ab4f3-56e6-4dc4-8b30-2f4fd4bbeb0b` | All event types |
| Lidarr | `66ac6fdd-69f6-4f47-bb00-b7f6d84d7c1c` | All event types |
| Uptime Kuma | `1a02e890-bb25-42f1-99fe-bba6a19f1811` | Status change notifications |
| Seerr | `555185af-90a1-42ff-aed5-c344e11955cf` | Request/approval events |
| Owncast (Livestream) | `9993e911-c68b-4271-a178-c2d65ca88499` | STREAM_STARTED / STREAM_STOPPED |
| Bazarr | `470fb267-3436-4dd3-a70c-e6e8db1721be` | Subtitle events (Apprise JSON notifier) |
| Tinker-Tickets | `6e306faf-8eea-4ba5-83ef-bf8f421f929e` | Custom transformation code |
| Ci-Alert | `5e7051e7-fd0c-4930-a711-9c67daf6f0b9` | Gitea Actions `notify-failure` jobs (org secret `MATRIX_WEBHOOK_URL`); transform `hookshot/ci-alert.js` |
| Devdesk | `4a3ccfc9-e1fe-4124-b014-7842fe51b9dc` | DevDesk "Claude needs you" (CT 126 `devdesk-notify-matrix`, URL in `/etc/devdesk/matrix-webhook`); transform `hookshot/devdesk.js`; avatar set via the appservice token |
| Grafana | `e713b383…` | Unified alerting contact point |
| Proxmox | `ca27432c…` | Notification system (8.1+), Discord embed format |
| Sonarr | `c3dde7ba…` | All event types |
| Radarr | `a7584167…` | All event types |
| Readarr | `3b30c0df…` | All event types |
| Uptime Kuma | `4b3fb94b…` | Status change notifications |
| Seerr | `9c56d489…` | Request/approval events |
| Owncast (Livestream) | `ff37824a…` | STREAM_STARTED / STREAM_STOPPED |
| Bazarr | `263a8301…` | Subtitle events (Apprise JSON notifier) |
| Tinker-Tickets | `135b040b…` | Custom transformation code |
| Ci-Alert | `66c1aa17…` | Gitea Actions `notify-failure` jobs (org secret `MATRIX_WEBHOOK_URL`); transform `hookshot/ci-alert.js` |
| Devdesk | `db7df1a1…` | DevDesk "Claude needs you" (CT 126 `devdesk-notify-matrix`, URL in `/etc/devdesk/matrix-webhook`); transform `hookshot/devdesk.js`; avatar set via the appservice token |
**Hookshot notes:**
- Spam and Stuff is intentionally **unencrypted** — hookshot bridges cannot join E2EE rooms
- JS transformation functions use hookshot v2 API: `result = { version: "v2", plain, html, msgtype }`
- The `result` variable must be assigned without `var`/`let`/`const` (QuickJS IIFE sandbox)
- NPM proxies `https://matrix.lotusguild.org/webhook/*` → `http://10.10.10.29:9003`
- NPM proxies `/sfu/get` and `/get_token` → `http://10.10.10.29:8070` (lk-jwt-service). Both paths are in `/data/nginx/proxy_host/49.conf` on LXC 139 — **NPM will overwrite these if proxy host 49 is re-saved via the UI; re-add both location blocks after any NPM save**
- NPM proxies `/sfu/get`, `/get_token` and `/delegate_delayed_leave` → `http://10.10.10.29:8070` (voice-limit-guard → lk-jwt-service), and `/now-playing/` → `:8072`. Since 2026-11-12 all of proxy host 49's custom locations live in its **Advanced** config (NPM sqlite `proxy_host.advanced_config`), so a UI save keeps them. Change them there (or in both the DB and `/data/nginx/proxy_host/49.conf`), never only in the conf file.
- Proxmox sends Discord embed format: `data.embeds[0].{title,description,fields}` — NOT flat fields
- Transform functions are stored as Matrix room state (`uk.half-shot.matrix-hookshot.generic.hook`) and deployed via `hookshot/deploy.sh`
- Hook UUIDs are **not** in that state. Hookshot creates them and keeps a `{uuid: name}` map in the `@hookshot` bot's room account data (same event type). Deploying a new transform creates the hook. To look a UUID up on LXC 151, use the appservice token from `/etc/matrix-synapse/hookshot-registration.yaml`: `GET /_matrix/client/v3/user/@hookshot:matrix.lotusguild.org/rooms/<room>/account_data/uk.half-shot.matrix-hookshot.generic.hook?user_id=@hookshot:matrix.lotusguild.org`
@@ -381,6 +404,149 @@ bash /opt/matrix-config/hookshot/deploy.sh proxmox.js # deploy one
---
## Discord Bridge (mautrix-discord v0.7.7)
Bridges Discord channels to Matrix rooms on LXC 151. The bridge logs in as a **Discord bot**
(not anyone's personal account, which breaks Discord's ToS): Discord users appear in Matrix as
`@discord_<id>:matrix.lotusguild.org` ghosts, and Matrix users are relayed into Discord through a
channel webhook with their own display name.
- Bridge bot: `@discordbot:matrix.lotusguild.org`
- Config: `/opt/mautrix-discord/config.yaml` (template in `discord-bridge/config.yaml`; the bridge
rewrites it on start, so it is **not** auto-deployed)
- Registration: `/etc/matrix-synapse/discord-registration.yaml` (listed in `app_service_config_files`)
- Database: `mautrix_discord` on LXC 109, role `mautrix_discord`, allowed from 10.10.10.29 in `pg_hba.conf`
- Service: `systemctl status mautrix-discord`, logs in `journalctl -u mautrix-discord` and `/var/log/mautrix-discord/`
- End-to-bridge encryption is allowed, so it can join encrypted rooms
- **Patched build.** The deployed binary is upstream v0.7.7 plus `discord-bridge/patches/`, built with
`discord-bridge/build-patched.sh` (deploy steps are in the script's header). `install.sh` still
downloads the stock release, so rebuild and redeploy the patched binary after any (re)install or upgrade.
- `0001`: when a Matrix → Discord send fails before the request reaches Discord (DNS, connect, or
`TLS handshake timeout`), the bridge retries after 2, 5, 10, 20, 30, 60, 60 and 60 s instead of
dropping the message with "⚠ Your message may not have been bridged". The room's later messages wait
in order behind the retry. Edits, reactions and deletes are not retried.
- `0002`: when a relay (webhook) send fails after the request may already have reached Discord
(connection reset, EOF, timeout waiting for the reply, Discord 5xx), the bridge waits up to 6 s for
its own post to come back over the Discord gateway. If it does, the send counts as delivered (no
resend, no copy back into Matrix); if not, it retries with the 0001 backoff, checking again before
each resend. A webhook post has no dedupe, so this echo check is what keeps those retries from
double-posting.
**Install (one time):** after this repo's `main` has reached `/opt/matrix-config` on LXC 151, run this
on the Proxmox node hosting LXC 109 and 151. It restarts Synapse once and is safe to re-run.
```bash
pct exec 151 -- cat /opt/matrix-config/discord-bridge/install.sh | bash
```
**Discord side:** Developer Portal → application → Bot. Turn on the **Message Content** and
**Server Members** intents. Invite the bot with the `bot` scope and View Channels, Send Messages,
Read Message History, Manage Webhooks, Attach Files, Add Reactions, Manage Messages and Embed Links.
**Linking a room:**
1. DM `@discordbot:matrix.lotusguild.org`: `login-token bot <bot token>` (the token lives only in the bridge DB)
2. `guilds bridge <discord server id>` — the server ID comes from Discord Developer Mode → right-click
3. In the Matrix room: invite `@discordbot`, then `!discord bridge <channel id>` and `!discord set-relay --create <webhook name>`
Skip `guilds bridge`: it sets the guild to create a Matrix room for every channel on its first message.
`bridge` alone bumps the guild to `if-portal-exists`. To move a link to another room, run
`!discord bridge --replace <channel id>` there (never `--replace=delete`, which kicks everyone from the
old room). The relay webhook stays with the channel. Keep `@discordbot` at the default power level so it
can't overwrite the room's name and icon with the Discord channel's.
**Presence (`discord-bridge/lotus_presence.py`, `lotus-discord-presence.service`):** the bridge doesn't carry Discord status, so this small service does. It reuses the bridge's bot (token read from the bridge database at startup; a second gateway session with only the Guilds and Presence intents; **Presence Intent** must stay on in the developer portal) and has its own appservice registration, `/etc/matrix-synapse/discord-presence-registration.yaml`, non-exclusive on `@discord_*`, so it can update ghosts but not create them. Online / idle / DND / offline map to online / unavailable / `org.matrix.msc3026.busy` / offline; custom status and the first activity become the status message. Only people who already have a ghost show up. Synapse drops an unrefreshed user to offline after about a minute, so live states are re-sent every 20 s; if the service dies, ghosts go offline rather than stale. Install once with `discord-bridge/install-presence.sh`, then restart Synapse. Statuses and activities are never logged.
**Bot profile:** `@discordbot` shows as "Discord Bridge" with the Discord logo
(`discord-bridge/bot-avatar.png`). To reapply it: `pct exec 151 -- bash /opt/matrix-config/discord-bridge/set-bot-profile.sh`.
**Who the Discord bot login belongs to:** `@jared` holds it. The account holding the login sends its
own messages to Discord *as the bot* ("matrix-bridge APP"), because the bridge prefers a real Discord
connection over the webhook. Everyone else is relayed through the channel webhook under their own name
and avatar. To change that, log the bot in from a dedicated non-chatting account; that automatically
logs out whichever account held it before.
**Rotating the bot token:** reset it in the Developer Portal, then DM the bridge bot `logout` and
`login-token bot <new token>`.
**Matrix avatars on Discord:** relayed messages carry the sender's avatar as a signed URL under
`bridge.public_address` (`https://matrix.lotusguild.org`), which Discord fetches. NPM proxies
`/mautrix-discord/avatar/` → `http://10.10.10.29:29334` (hand-added location in
`/data/nginx/proxy_host/49.conf` on LXC 139, like `/sfu/get` — **re-add it after any NPM UI save of
proxy host 49**). The bridge listens on `0.0.0.0:29334` for this; the appservice API on that port
still requires the `hs_token`, and only the avatar path is proxied publicly.
**Catch-up after downtime:** `backfill.forward_limits.missed.channel: -1`, so Discord messages sent
while the bridge or Synapse was down are bridged when it starts again.
---
## Now Playing ("Listening to", cinny #491)
`now-playing/lotus_now_playing.py` (`lotus-now-playing.service`, LXC 151, `:8072`, NPM `/now-playing/`) gives Lotus Chat what people are playing, for a Discord/Spotify-style "Listening to" card.
- **Source:** Navidrome `getNowPlaying`, polled every 10 s as the non-admin Navidrome user `lotus-nowplaying` (direct to `http://10.10.10.28:4533`). Navidrome is the scrobbling master and Dropped Needle passes the real listener through, so every play shows up under the listener's own Navidrome username, which is their SSO username. Dropped Needle's own `droppedneedle` service user is ignored.
- **Who's asking:** Lotus Chat sends a Matrix OpenID token; the service checks it with Synapse (`/_matrix/federation/v1/openid/userinfo`). Only users of this homeserver can read anything.
- **Matrix user → SSO username:** `/etc/lotus-now-playing/users.json` first (server-only, not in this public repo; for accounts whose SSO link has no email), then the Synapse admin API: an email `<name>@lotusguild.org`, or the legacy plain-username external ID. Uses `MATRIX_TOKEN` from `/etc/matrix-deploy.env`.
- **Opt-in:** off by default; each person turns on "Share what I'm listening to" in Lotus Chat. Stored in `/var/lib/lotus-now-playing/state.json`.
- **API:** `GET`/`PUT /now-playing/v1/me` (`{share, linked, playing}`), `GET /now-playing/v1/users?ids=@a,@b` (up to 50; only opted-in, matched, playing users), and `GET /now-playing/v1/cover/<id>?size=`. The cover endpoint has no auth (for `<img>`) but only serves covers an opted-in user is playing right now.
- Titles, artists and tokens are never logged.
- **Install / update:** create `/etc/lotus-now-playing/env` (`NAVIDROME_USER`, `NAVIDROME_PASSWORD`) once, then `bash /opt/matrix-config/now-playing/install.sh`. Tests: `python3 -m unittest now-playing/test_lotus_now_playing.py`.
## Voice Bridge (nether-voicebridge, Lotus-patched)
Bridges Discord voice into a Lotus voice room. People on Discord appear **inside the Matrix call**
under their Discord names, with a speaking indicator. People in Lotus appear in the Discord channel
as `[Matrix] <name>`, with mute mirrored to Discord.
| | Text | Voice |
|---|---|---|
| Discord (Lotus Guild server) | #matrix-bridge | matrix-bridge |
| Matrix (Lotus) | Discord Bridge (mautrix-discord) | Discord Voice Bridge (`!yS_ZIHL3sOKQJBt4NiIQyiHKt3cv6HrIDeq7OxDpSM0`) |
- **Build:** upstream [nether-voicebridge](https://nether.codes/nether/nether-voicebridge) `cf52945`
plus 12 patches to report upstream and 1 Lotus-only patch: Discord users without an avatar get
Discord's default one. Patches, bug log and issue drafts live in
`/root/code/voice-bridge-trial/`. **Don't run the unpatched release**. Without the patches the
bridge can't see anyone using current Element Call or Lotus Call (`livekit_alias`), and encrypted
voice breaks after a restart on Synapse.
- **Accounts:** the bot is `@lotus-voice_bot`, shown as "Discord Voice Bridge" with the Discord logo.
It has **no special power** in the room; it only needs calls at power level 0. The room has the
Discord logo, a topic, and an `m.bridge` record, so Lotus shows the Discord chip linking to the channel.
Discord speakers are `@lotus-voice_<discord id>`. The appservice registration is
`/etc/matrix-synapse/lotus-voice-registration.yaml`.
- **Discord bots:** Lotus Voice 1–3, one per Matrix person speaking at once. Each has only Connect,
Speak, Use Voice Activity and Change Nickname, and none can see #matrix-bridge text.
- **Files on LXC 151:**
- binary: `/opt/lotus-voice/lotus-voice-bridge`
- config: `/etc/lotus-voice/config.toml` (0600, holds the bot tokens)
- state: `/var/lib/lotus-voice`
- service: `lotus-voice-bridge`, logs via `journalctl -u lotus-voice-bridge`
- **Voice limits:** `voice-limit-guard` ignores `@lotus-voice_bot` (`VOICE_LIMIT_EXEMPT_USERS`), so
the bot never takes a slot. Discord speakers count like anyone else. When a room is full the
bridge stops trying cleanly, and the Discord user just isn't heard.
- **Install / upgrade:** on the Proxmox node hosting LXC 151, run
`bash voice-bridge/install.sh <binary> <secrets file with BOT1..BOT3>`. The first install
registers the appservice, restarts Synapse once and creates the bot account. Then invite
`@lotus-voice_bot:matrix.lotusguild.org` to the voice room. **To upgrade:** push the new binary to
`/opt/lotus-voice/lotus-voice-bridge` (keep the old one as `.prev`), then run
`systemctl restart lotus-voice-bridge`. Synapse isn't touched.
## Bot profiles
Every bot account has a proper name and an avatar in one style (white glyph on a coloured square). Image sources are in `avatars/` and `discord-bridge/bot-avatar.png`.
| Account | Name | How it's set |
|---------|------|--------------|
| `@discordbot` | Discord Bridge | `discord-bridge/set-bot-profile.sh` (bridge config `appservice.bot`, applied on bridge start) |
| `@discord-relay` | Discord Relay | Set by hand while signed in as it (`avatars/discord-relay.png`); holds the Discord bot login |
| `@hookshot` | Lotus Webhooks | `bot:` section in `/opt/hookshot/config.yml` on LXC 151 (`avatars/hookshot.png`), applied on hookshot start |
| `@draupnir` | Draupnir | Profile API with Draupnir's token (Draupnir doesn't manage its own profile) |
| `@lotusbot` | Lotus Bot | Already set |
| `@notices` | Server Notices | `server_notices.system_mxid_avatar_url` in `homeserver.yaml` (Lotus logo); needs a Synapse restart |
| `@lotus-mc-bridge` | Lotus Minecraft | Profile API with its own token, from the Minecraft mod's config (voxel block with a lotus, not the white-glyph style; image not in `avatars/`). Its status is the player count: `minecraft-bridge/lotus_mc_status.py`, which runs on CT 154 as `lotus-mc-status.service` (not on LXC 151; installed by hand) |
---
## Moderation (Draupnir v2.9.0)
Draupnir runs on LXC 110, manages moderation across all protected rooms (including the Lotus Guild space) via `#management:matrix.lotusguild.org`.
@@ -521,6 +687,27 @@ Infra notes for THIS repo:
**Full handoff & step-by-step plan:** `LotusGuild/cinny` →
[`HANDOFF_ELEMENT_CALL_FORK.md`](https://code.lotusguild.org/LotusGuild/cinny/src/branch/lotus/HANDOFF_ELEMENT_CALL_FORK.md).
### Not planned: background web push
**Decision (2026-11-09):** the web app does **not** get background push
notifications (notifications while no Lotus tab is open), and no push gateway
(Sygnal) is run on our infrastructure.
- **Why:** people who want notifications keep the **desktop app** open (it
notifies natively and keeps running in the tray), and on phones people use
**Element X** or **FluffyChat**, which already push through their own
gateways. A gateway would only help someone using the website with every tab
closed, which doesn't justify another service to run, monitor and secure on
LXC 151.
- **What users get:** OS notifications while the web or desktop app is open,
in-app toasts, quiet hours and pause, email notifications (optional, Synapse).
- **If this changes:** the plan (Sygnal on LXC 151, VAPID key, NPM route locked
to Synapse, config.json `webPush` entry) is in
[cinny #202](https://code.lotusguild.org/LotusGuild/cinny/issues/202#issuecomment-6221),
and the finished-but-unmerged client side (service-worker `push` handler,
pusher registration, setting, tests) is in closed
[cinny PR #489](https://code.lotusguild.org/LotusGuild/cinny/pulls/489).
### Custom Features
All custom code lives in `src/app/` on the `lotus` branch of `code.lotusguild.org/LotusGuild/cinny`. Changes survive upstream merges as long as they don't conflict with the same files upstream touched.
@@ -617,7 +804,7 @@ Periodic `TLS/TCP socket error: Connection reset by peer` in coturn logs. Normal
`net.ipv4.tcp_congestion_control = bbr` must be set on the Proxmox **host**, not inside an unprivileged LXC. All other sysctl tuning (TCP/UDP buffers, fin_timeout) is applied inside LXC 151.
Applied 2026-07-26 via `/etc/sysctl.d/99-bbr.conf` on `compute-storage-01` and `large1`, and
Applied 2026-07-26 via `/etc/sysctl.d/99-bbr.conf` on `compute-storage-01` and `large1` (now `compute-storage-02`), and
**2026-08-16 on `compute-storage-gpu-01`** (which was found on `cubic`/`fq_codel` with `tcp_bbr` not
even loaded). All three nodes that may host LXC 151 now report `bbr`/`fq`. Because the file lives on
the host, **a node that has never hosted 151 will not have it** — check before allowing 151 to migrate
@@ -627,9 +814,9 @@ somewhere new:
sysctl net.ipv4.tcp_congestion_control net.core.default_qdisc # want: bbr / fq
```
### ⚠️ `large1` has a failing DIMM — keep Synapse and PostgreSQL off it
### ⚠️ `compute-storage-02` has a failing DIMM — keep Synapse and PostgreSQL off it
`large1` logs **corrected ECC errors** concentrated on one module (**P0 CHANNEL B, DIMM 1** —
`compute-storage-02` (called `large1` until the 2026-11-18 rename) logs **corrected ECC errors** concentrated on one module (**P0 CHANNEL B, DIMM 1** —
EDAC `mc#0csrow#2channel#1`, 421 corrected errors observed, with the MCE `Over`/overflow flag set
meaning more occurred than were logged). It has hard-crashed **on its own** twice — 2026-07-24 14:19
and 2026-07-26 00:20 — with no clean-shutdown markers and no OOM events, which is the classic
@@ -640,17 +827,18 @@ signature of an uncorrectable ECC error.
Consequences for Matrix:
- On the 2026-07-26 crash, HA relocated **LXC 109 (PostgreSQL)** and **LXC 151 (Synapse)** to
`compute-storage-01`, which has **zero ECC errors across all boots**. They should stay there.
- The HA rule `ha-rule-43cb668b-becf` in `/etc/pve/ha/rules.cfg` currently weights
`compute-storage-01:1,large1:1` — **equal priority, non-strict** — so a future failover can put
Synapse straight back onto the bad-RAM node. Re-weighting to `compute-storage-01:2,large1:1` would
prefer the healthy node while keeping large1 as a fallback. Not changed automatically because that
rule governs 10 resources and `compute-storage-01` has only 46GB RAM vs large1's 128GB.
- LXC 109 (PostgreSQL) and LXC 151 (Synapse) are HA guests and move between nodes; keep them
off compute-storage-02 through the HA rule weights, not by pinning.
- The HA rule `ha-rule-43cb668b-becf` in `/etc/pve/ha/rules.cfg` (checked 2026-11-18) weights
`compute-storage-01:1,compute-storage-02:2,storage-01:2`, so the bad-RAM node is one of the two
**preferred** nodes, and a failover can put Synapse or PostgreSQL onto it. Lowering
`compute-storage-02` below `storage-01` would keep it as a fallback only. Not changed automatically,
because that rule governs 9 CTs and a VM. Look up where 151 and 109 run now with
`ha-manager status | grep -E 'ct:(151|109) '`; they move.
- **Never run a large `VACUUM FULL` or state-compression job on a host with failing RAM** — those
rewrite the whole table through memory, and a bit-flip becomes silent, backup-propagated corruption.
- Ceph's **`noout` flag is intentionally left set.** With 10 OSDs on an intermittently-crashing node,
`noout` prevents a full 10-OSD rebalance every time it bounces. Do not clear it until large1 is stable.
`noout` prevents a full 10-OSD rebalance every time it bounces. Do not clear it until compute-storage-02 is stable.
A replacement DIMM is not currently in budget. Free mitigations worth evaluating: physically removing
the bad module (96GB in flex mode, or 64GB symmetric dual-channel), kernel page soft-offlining via
@@ -734,6 +922,43 @@ SELECT room_id, count(*) FROM state_groups_state GROUP BY room_id ORDER BY 2 DES
Spaces (`m.space`) are the worst offenders per byte — they carry huge membership with zero messages.
Leaving a space does **not** leave its child rooms; they are independent rooms with their own IDs.
**It grows back.** From 2183 MB after the July cleanup, the database reached 6.2 GB by 2026-11-03
(`state_groups_state` 2.75 GB, 11 M rows). Almost all of that came from big federated rooms, each with
one or two local members: *Jellyfin General Chat* (16,754 members, 5.3 M rows), an unnamed 1,213-member
room (2.4 M), *Homelabbing*, *Element X Android*, the *Cinny community* space.
#### Nightly state compression (CT 109)
`synapse_auto_compressor` (from [rust-synapse-compress-state](https://github.com/matrix-org/rust-synapse-compress-state),
built from `main`) runs every night at 04:30, after the 03:00 vzdump backup. It does not delete anything:
it rewrites how a room's state history is stored as smaller deltas.
- **Safety.** It checks that the compressed state matches the original before writing, writes each
state group in its own transaction, and skips any chunk that would not shrink.
- **Progress.** It stores its progress in `state_compressor_*` tables, so each night continues where
the last one stopped.
- **Cost.** No downtime. Synapse's DB p99 stayed at ~10 ms while the initial backlog ran.
- **First run (2026-11-03).** The initial backlog took ~8 minutes, with 0 Synapse errors.
`state_groups_state` went from **11.0 M to 4.25 M rows (−61%)**.
- **Disk.** The table file stays ~2.77 GB on disk: the freed ~1.75 GB is reused space inside it, so
the database stops growing until that is used up. To give it back to the filesystem you need
`VACUUM FULL state_groups_state`. That takes an exclusive lock and stalls Synapse for a minute or
two, so it is only worth doing if CT 109's disk gets tight (it is at 14%).
Units: `systemd/synapse-state-compressor.{service,timer}` (CT 109 has no auto-deploy hook, so install them by
hand). Binary: `/usr/local/bin/synapse_auto_compressor`.
```bash
# on CT 109
install -m644 systemd/synapse-state-compressor.{service,timer} /etc/systemd/system/
systemctl daemon-reload && systemctl enable --now synapse-state-compressor.timer
journalctl -u synapse-state-compressor -n 5 # "Finished running compressor. Saved N rows."
```
The Grafana alert **Synapse Database Growing Fast** (Matrix → Database) warns if the database grows by
more than 500 MB in 7 days. That means either the timer stopped running, or a room is growing faster
than nightly compression can keep up with.
---
## Server Checklist
@@ -793,7 +1018,7 @@ Leaving a space does **not** leave its child rooms; they are independent rooms w
- [x] TCP retransmit timeout lowered (`tcp_retries2=5`, `tcp_syn_retries=4`, `tcp_keepalive_probes=3`) — stalled outbound federation connections now fail in ~15-30s instead of ~15 min
- [x] Unreachable routes added for servers with asymmetric connectivity (can reach us but we can't reach their federation port) — prevents 90s TCP hangs from being added to lag; defined in `/etc/network/interfaces` post-up hooks and survive reboots (bark.lgbt ×2, parodia.dev, chat.ohaa.xyz, matrix.k8ekat.dev)
- [x] Stuck `device_lists_remote_resync` entries cleared for dead-server users (@dalite:bark.lgbt, @arndot:matrix.goch.social) — device list resync was firing every 30s
- [x] BBR congestion control — applied 2026-07-26 on `compute-storage-01` and `large1`, and 2026-08-16 on `compute-storage-gpu-01`, via `/etc/sysctl.d/99-bbr.conf` (`net.ipv4.tcp_congestion_control=bbr`, `net.core.default_qdisc=fq`). Persists across reboots. All three nodes that may host LXC 151 are now covered. If 151 is ever allowed to run on another node, apply it there too.
- [x] BBR congestion control — applied 2026-07-26 on `compute-storage-01` and `large1` (now `compute-storage-02`), and 2026-08-16 on `compute-storage-gpu-01`, via `/etc/sysctl.d/99-bbr.conf` (`net.ipv4.tcp_congestion_control=bbr`, `net.core.default_qdisc=fq`). Persists across reboots. All three nodes that may host LXC 151 are now covered. If 151 is ever allowed to run on another node, apply it there too.
- [x] LXC 109 rootfs grown 30GB → 80GB (2026-07-26). It was at **89% full with only 3.4GB free**, and `pg_wal` shares that volume — PostgreSQL stops outright at 100%. Also a hard prerequisite for any `VACUUM FULL`, which must rewrite the table alongside the original.
- [x] **Purged the `#community:matrix.org` space** (`!iMZEhwCvbfeAYUxAjZ:t2l.io`) 2026-07-26 — 81,724,747 rows / 17GB for a directory wrapper with **zero messages** and one local member. Admin API `DELETE /_synapse/admin/v2/rooms/` with `purge:true`, then `VACUUM FULL`. ~2.5 min downtime.
- [x] **Compressed room state** with `rust-synapse-compress-state` (built at `/opt/rust-synapse-compress-state/` on LXC 109). Every run verified `New state map matches old one` before commit:
@@ -809,7 +1034,7 @@ Leaving a space does **not** leave its child rooms; they are independent rooms w
> **Do NOT compress `Spam and Stuff`.** Its 107,763 state groups hold only 107,799 rows (~1 row each) — already minimal. The compressor's leveled structure would *add* rows (dry run: **110.16%**) and it refuses to run. High state-group count is not the same as compressible bloat.
- [x] PostgreSQL: `jit=off`, `wal_compression=lz4`, `max_worker_processes` 3→8, `maintenance_work_mem` 256MB→512MB, `shared_buffers` 1500MB→2GB in `/etc/postgresql/17/main/conf.d/synapse_tuning.conf`. Previous version kept as `synapse_tuning.conf.bak-20260726`. **That file is not in this repo — the tuning is unversioned.**
- [ ] `synchronous_commit = off` — **deliberately NOT applied.** Would be a real write-latency win, but this cluster hard-resets every few hours (HA self-fences on corosync quorum loss + the large1 DIMM), so it would drop up to ~600ms of committed messages *per reset* rather than once in a blue moon. Revisit once the cluster is stable.
- [ ] `synchronous_commit = off` — **deliberately NOT applied.** Would be a real write-latency win, but this cluster hard-resets every few hours (HA self-fences on corosync quorum loss + the compute-storage-02 DIMM), so it would drop up to ~600ms of committed messages *per reset* rather than once in a blue moon. Revisit once the cluster is stable.
### Auth & SSO
- [x] Open registration disabled (`enable_registration: false`, 2026-10-16): accounts come from LLDAP and are created on first sign-in through Lotus SSO (Authelia OIDC, which keeps its own registration on). The old registration token was deleted.
@@ -823,6 +1048,7 @@ Leaving a space does **not** leave its child rooms; they are independent rooms w
- [x] Per-service JS transformation functions (stored in git, auto-deployed)
- [x] Per-service virtual user avatars
- [x] NPM reverse proxy for `/webhook` path
- [x] mautrix-discord bridge (bot login + webhook relay) — config and installer in `discord-bridge/`, live since 2026-10-28
### Room Structure
- [x] The Lotus Guild space with all core rooms
@@ -862,6 +1088,49 @@ Leaving a space does **not** leave its child rooms; they are independent rooms w
---
## Hookshot Upgrade Notes
Hookshot is a **git checkout built from source** at `/opt/hookshot` (`ExecStart=node lib/App/BridgeApp.js`).
Only `config.yml` and `passkey.pem` in it are not generated; storage is in-memory (connections are room state).
- **7.4.1+ builds with pnpm**, and `scripts/build-app.sh` calls `pnpm` directly. Without a global pnpm,
put a wrapper first in `PATH`: `printf '#!/bin/sh\nexec corepack pnpm@<version> "$@"\n'` (version from
`devEngines.packageManager` in `package.json`). The Rust layer needs `~/.cargo/bin` in `PATH`.
- **7.6.0 drops Node 22** (LXC 151 runs Node 22), so 7.5.0 is the newest that runs as-is.
- Build the new tag in a separate directory (`git clone --depth 1 --branch <tag> … /opt/hookshot-<tag>`,
`pnpm install --frozen-lockfile && pnpm run build`), copy in `config.yml` + `passkey.pem`, then stop the
service, swap the directories (keep the old one for rollback) and start it. Check the log says
`Bridge is now ready. Found N connections`.
- 2026-11-03: 7.3.2 → 7.5.0 (GHSA-5c49-6fvm-2xvc: `/metrics` exposed webhook URLs, plus two DoS fixes).
Old build kept at `/opt/hookshot-7.3.2`.
## Call Stack Upgrade (lk-jwt-service 0.4.1 → 0.9.0, LiveKit 1.9.11 → 1.13.11) — done 2026-11-03
Tested on the local stack, then applied on LXC 151 on 2026-11-03 with release binaries (sha256-checked).
Rollback: `/usr/local/bin/lk-jwt-service.0.4.1`, `/usr/local/bin/livekit-server.1.9.11`,
`/etc/livekit/config.yaml.bak-20261103`, and drop the `LIVEKIT_SANITY_CHECK_INTERVAL_SECONDS` line from
`/etc/systemd/system/lk-jwt-service.service.d/override.conf`. The changes:
1. **lk-jwt-service 0.9.0** (a Rust rewrite; fixes GHSA-rq24-fh66-rq7x). Existing env vars carry over.
Add `LIVEKIT_SANITY_CHECK_INTERVAL_SECONDS=30`.
2. **LiveKit webhook → lk-jwt.** 0.9.0 serves `/delegate_delayed_leave`, so Element Call hands it the
job of removing a crashed client's call membership and sets its own fallback timer to **1 hour**.
lk-jwt only learns that someone left from LiveKit webhooks; without them crashed clients linger for an
hour (also on Discord, via the voice bridge). In `/etc/livekit/config.yaml`:
```yaml
webhook:
api_key: <the key under keys:, same as LIVEKIT_KEY>
urls:
- http://127.0.0.1:8071/sfu_webhook
```
3. **`prometheus_port: 6789` → `prometheus: {port: 6789}`.** LiveKit 1.13.8 removed the old key (1.9.11
already accepts the new one). Otherwise metrics and the *LiveKit Down* alert stop working.
4. Restart lk-jwt-service only when no calls are active: its leave jobs are in memory (or set
`LIVEKIT_REDIS_URL`). LiveKit: replace the binary and `touch /run/livekit-restart-pending`.
Not affected: TURN (external coturn with `ttl` set, as 1.13.1 requires), `room.auto_create: false`,
the voice bridge (uses the legacy `/sfu/get`, still served).
## Synapse Upgrade Runbook
Synapse is installed from the **matrix.org apt repo** (`matrix-synapse-py3`) on **LXC 151** — it is
@@ -947,6 +1216,7 @@ Checklist in this file.
| PostgreSQL Down | `pg_up` < 1 for 2m | critical |
| LiveKit Down | `up{job="livekit"}` < 1 for 2m | critical |
| Hookshot Down | `up{job="hookshot"}` < 1 for 2m | critical |
| Discord Bridge Down | `node_systemd_unit_state{name="mautrix-discord.service",state="active"}` < 1 (or no data) for 3m | critical |
| Draupnir Down | `up{job="draupnir-node"}` < 0.5 for 2m | critical |
| PG Connection Saturation | connections > 80% of max for 5m | warning |
| Federation Queue Backing Up | pending PDUs > 100 for 10m | warning |
@@ -1052,11 +1322,12 @@ All commands use the `!` prefix. Run `!help` in any room for the full list.
| Homeserver | Synapse | 1.159.0 |
| Database | PostgreSQL | 17.9 |
| TURN | coturn | latest |
| Video/voice calls | LiveKit SFU | 1.9.11 |
| LiveKit JWT | lk-jwt-service | latest |
| Video/voice calls | LiveKit SFU | 1.13.11 |
| LiveKit JWT | lk-jwt-service | 0.9.0 |
| Moderation | Draupnir | 2.9.0 |
| SSO | Authelia (OIDC) + LLDAP | — |
| Webhook bridge | matrix-hookshot | 7.3.2 |
| Webhook bridge | matrix-hookshot | 7.5.0 |
| Discord bridge | mautrix-discord | 0.7.7 |
| Reverse proxy | Nginx Proxy Manager | — |
| Web client | Lotus Cinny (fork of `cinnyapp/cinny` main) | custom |
| Element Call embed | `@lotusguild/element-call-embedded` (self-built fork of `element-hq/element-call`) | 0.20.1-lotus.1 |
Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

+7 -1
View File
@@ -28,12 +28,18 @@
"statusPages": {
"matrix.lotusguild.org": {
"url": "https://isitup.lotusguild.org",
"slug": "matrix",
"slug": "public",
"noticePrefix": "Matrix:",
"groups": {
"homeserver": "Homeserver",
"calls": "Voice calls",
"login": "Login"
}
}
},
"nowPlaying": {
"matrix.lotusguild.org": {
"url": "https://matrix.lotusguild.org/now-playing"
}
}
}
+19 -6
View File
@@ -1,18 +1,31 @@
#!/bin/bash
# Checks every 5 minutes if a livekit restart is pending.
# Only restarts when there are no active WebSocket connections on port 7881
# (established connections = active call participants).
# Only restarts when LiveKit reports no participants AND nobody holds a
# signalling connection, so a restart never drops a call.
# Run by: livekit-graceful-restart.timer (systemd)
if [ ! -f /run/livekit-restart-pending ]; then
exit 0
fi
# Count established WebSocket signaling connections on livekit port 7881
ACTIVE=$(ss -tn state established '( dport = :7881 or sport = :7881 )' | grep -c ESTAB || true)
# Participants LiveKit itself counts, across all rooms. Counting TCP connections
# on :7881 (the old check) only saw clients on the ICE-TCP fallback: signalling
# is on :7880 and media is normally UDP, so most calls looked idle.
METRICS_URL="${LIVEKIT_METRICS_URL:-http://127.0.0.1:6789/metrics}"
if ! METRICS=$(curl -sf --max-time 5 "$METRICS_URL") \
|| ! printf '%s\n' "$METRICS" | grep -q '^livekit_participant_total'; then
echo "$(date): Livekit restart pending but its metrics are unreadable — assuming a call is active, waiting."
exit 0
fi
PARTICIPANTS=$(printf '%s\n' "$METRICS" | awk '/^livekit_participant_total/ {s += $NF} END {printf "%d", s}')
if [ "$ACTIVE" -gt 0 ]; then
echo "$(date): Livekit restart pending but $ACTIVE active connection(s) — waiting."
# Belt and braces: open signalling WebSockets (a client mid-join may not be a
# participant yet). Count lines (-H drops the header): with a `state` filter ss
# omits the State column, so the old `grep -c ESTAB` was always 0.
SIGNALLING=$(ss -Htn state established '( sport = :7880 )' | wc -l)
if [ "$PARTICIPANTS" -gt 0 ] || [ "$SIGNALLING" -gt 0 ]; then
echo "$(date): Livekit restart pending but $PARTICIPANTS participant(s), $SIGNALLING signalling connection(s) — waiting."
exit 0
fi
+20
View File
@@ -47,6 +47,26 @@ else
echo "Restart pending — will apply when no active calls."
fi
# The graceful-restart check itself (run by livekit-graceful-restart.timer)
if echo "$CHANGED" | grep -q '^deploy/livekit-graceful-restart\.sh'; then
echo "Deploying livekit-graceful-restart.sh..."
bash -n "$REPO_DIR/deploy/livekit-graceful-restart.sh" \
&& install -m755 "$REPO_DIR/deploy/livekit-graceful-restart.sh" /usr/local/bin/livekit-graceful-restart.sh
fi
# Discord presence service: only once installed (discord-bridge/install-presence.sh).
if [ -f /etc/lotus-presence/env ] && echo "$CHANGED" | grep -qE '^discord-bridge/lotus_presence\.py|^systemd/lotus-discord-presence\.service'; then
echo "Deploying lotus-discord-presence..."
if python3 -m py_compile "$REPO_DIR/discord-bridge/lotus_presence.py"; then
install -D -m644 "$REPO_DIR/discord-bridge/lotus_presence.py" /opt/lotus-presence/lotus_presence.py
install -m644 "$REPO_DIR/systemd/lotus-discord-presence.service" /etc/systemd/system/lotus-discord-presence.service
systemctl daemon-reload
systemctl restart lotus-discord-presence || true
else
echo "ERROR: lotus_presence.py has a syntax error, keeping the running version."
fi
fi
# Voice-limit / quality guard (fronts lk-jwt-service on :8070)
if echo "$CHANGED" | grep -qE '^livekit/voice-limit-guard\.py|^systemd/voice-limit-guard\.service'; then
echo "Deploying voice-limit-guard..."
Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Build mautrix-discord with the Lotus patches in discord-bridge/patches/.
#
# Needs: git, Go (version from upstream go.mod or newer), a C compiler and libolm-dev.
# Output: ./mautrix-discord-lotus (dynamically linked against libolm.so.3, which
# LXC 151 has from the libolm3 package).
#
# Deploy (on the Proxmox node hosting LXC 151):
# pct push 151 mautrix-discord-lotus /opt/mautrix-discord/mautrix-discord.new
# pct exec 151 -- bash -c 'cd /opt/mautrix-discord && cp -a mautrix-discord mautrix-discord.bak-$(date +%Y%m%d) \
# && chown mautrix-discord: mautrix-discord.new && chmod 755 mautrix-discord.new \
# && mv mautrix-discord.new mautrix-discord && systemctl restart mautrix-discord'
set -euo pipefail
VERSION="${VERSION:-v0.7.7}"
HERE="$(cd "$(dirname "$0")" && pwd)"
OUT="${OUT:-$PWD/mautrix-discord-lotus}"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
git clone -q --depth 1 --branch "$VERSION" https://github.com/mautrix/discord.git "$WORK/src"
cd "$WORK/src"
for p in "$HERE"/patches/*.patch; do
echo "applying $(basename "$p")"
git apply "$p"
done
go test -race -count=1 -run 'Retry|PreSend|Ambiguous|Echo' .
COMMIT="$(git rev-parse HEAD)"
MAUTRIX="$(go list -m -f '{{.Version}}' maunium.net/go/mautrix)"
CGO_ENABLED=1 go build -o "$OUT" -ldflags "-s -w \
-X main.Tag=$VERSION \
-X main.Commit=$COMMIT+lotus \
-X 'main.BuildTime=$(date '+%b %_d %Y, %H:%M:%S')' \
-X 'maunium.net/go/mautrix.GoModVersion=$MAUTRIX'" .
"$OUT" --version
echo "built $OUT"
+386
View File
@@ -0,0 +1,386 @@
# Homeserver details.
homeserver:
# The address that this appservice can use to connect to the homeserver.
address: http://127.0.0.1:8008
# The domain of the homeserver (also known as server_name, used for MXIDs, etc).
domain: matrix.lotusguild.org
# What software is the homeserver running?
# Standard Matrix homeservers like Synapse, Dendrite and Conduit should just use "standard" here.
software: standard
# The URL to push real-time bridge status to.
# If set, the bridge will make POST requests to this URL whenever a user's discord connection state changes.
# The bridge will use the appservice as_token to authorize requests.
status_endpoint: null
# Endpoint for reporting per-message status.
message_send_checkpoint_endpoint: null
# Does the homeserver support https://github.com/matrix-org/matrix-spec-proposals/pull/2246?
async_media: false
# Should the bridge use a websocket for connecting to the homeserver?
# The server side is currently not documented anywhere and is only implemented by mautrix-wsproxy,
# mautrix-asmux (deprecated), and hungryserv (proprietary).
websocket: false
# How often should the websocket be pinged? Pinging will be disabled if this is zero.
ping_interval_seconds: 0
# Application service host/registration related details.
# Changing these values requires regeneration of the registration.
appservice:
# The address that the homeserver can use to connect to this appservice.
address: http://127.0.0.1:29334
# The hostname and port where this appservice should listen.
hostname: 0.0.0.0
port: 29334
# Database config.
database:
# The database type. "sqlite3-fk-wal" and "postgres" are supported.
type: postgres
# The database URI.
# SQLite: A raw file path is supported, but `file:<path>?_txlock=immediate` is recommended.
# https://github.com/mattn/go-sqlite3#connection-string
# Postgres: Connection string. For example, postgres://user:password@host/database?sslmode=disable
# To connect via Unix socket, use something like postgres:///dbname?host=/var/run/postgresql
# Password is filled in on the server by install.sh; never commit it.
uri: postgres://mautrix_discord:DB_PASSWORD@10.10.10.44/mautrix_discord?sslmode=disable
# Maximum number of connections. Mostly relevant for Postgres.
max_open_conns: 20
max_idle_conns: 2
# Maximum connection idle time and lifetime before they're closed. Disabled if null.
# Parsed with https://pkg.go.dev/time#ParseDuration
max_conn_idle_time: null
max_conn_lifetime: null
# The unique ID of this appservice.
id: discord
# Appservice bot details.
bot:
# Username of the appservice bot.
username: discordbot
# Display name and avatar for bot. Set to "remove" to remove display name/avatar, leave empty
# to leave display name/avatar as-is.
displayname: Discord Bridge
avatar: mxc://maunium.net/nIdEykemnwdisvHbpxflpDlC
# Whether or not to receive ephemeral events via appservice transactions.
# Requires MSC2409 support (i.e. Synapse 1.22+).
ephemeral_events: true
# Should incoming events be handled asynchronously?
# This may be necessary for large public instances with lots of messages going through.
# However, messages will not be guaranteed to be bridged in the same order they were sent in.
async_transactions: false
# Authentication tokens for AS <-> HS communication. Autogenerated; do not modify.
as_token: "This value is generated when generating the registration"
hs_token: "This value is generated when generating the registration"
# Bridge config
bridge:
# Localpart template of MXIDs for Discord users.
# {{.}} is replaced with the internal ID of the Discord user.
username_template: discord_{{.}}
# Displayname template for Discord users. This is also used as the room name in DMs if private_chat_portal_meta is enabled.
# Available variables:
# .ID - Internal user ID
# .Username - Legacy display/username on Discord
# .GlobalName - New displayname on Discord
# .Discriminator - The 4 numbers after the name on Discord
# .Bot - Whether the user is a bot
# .System - Whether the user is an official system user
# .Webhook - Whether the user is a webhook and is not an application
# .Application - Whether the user is an application
displayname_template: '{{if .Webhook}}Webhook{{else}}{{or .GlobalName .Username}}{{if .Bot}} (bot){{end}}{{end}}'
# Displayname template for Discord channels (bridged as rooms, or spaces when type=4).
# Available variables:
# .Name - Channel name, or user displayname (pre-formatted with displayname_template) in DMs.
# .ParentName - Parent channel name (used for categories).
# .GuildName - Guild name.
# .NSFW - Whether the channel is marked as NSFW.
# .Type - Channel type (see values at https://github.com/bwmarrin/discordgo/blob/v0.25.0/structs.go#L251-L267)
channel_name_template: '{{if or (eq .Type 3) (eq .Type 4)}}{{.Name}}{{else}}#{{.Name}}{{end}}'
# Displayname template for Discord guilds (bridged as spaces).
# Available variables:
# .Name - Guild name
guild_name_template: '{{.Name}}'
# Whether to explicitly set the avatar and room name for private chat portal rooms.
# If set to `default`, this will be enabled in encrypted rooms and disabled in unencrypted rooms.
# If set to `always`, all DM rooms will have explicit names and avatars set.
# If set to `never`, DM rooms will never have names and avatars set.
private_chat_portal_meta: default
# Publicly accessible base URL that Discord can use to reach the bridge, used for avatars in relay mode.
# If not set, avatars will not be bridged. Only the /mautrix-discord/avatar/{server}/{id}/{hash} endpoint is used on this address.
# This should not have a trailing slash, the endpoint above will be appended to the provided address.
public_address: https://matrix.lotusguild.org
# A random key used to sign the avatar URLs. The bridge will only accept requests with a valid signature.
avatar_proxy_key: generate
portal_message_buffer: 128
# Number of private channel portals to create on bridge startup.
# Other portals will be created when receiving messages.
startup_private_channel_create_limit: 5
# Should the bridge send a read receipt from the bridge bot when a message has been sent to Discord?
delivery_receipts: false
# Whether the bridge should send the message status as a custom com.beeper.message_send_status event.
message_status_events: false
# Whether the bridge should send error notices via m.notice events when a message fails to bridge.
message_error_notices: true
# Should the bridge use space-restricted join rules instead of invite-only for guild rooms?
# This can avoid unnecessary invite events in guild rooms when members are synced in.
restricted_rooms: false
# Should the bridge automatically join the user to threads on Discord when the thread is opened on Matrix?
# This only works with clients that support thread read receipts (MSC3771 added in Matrix v1.4).
autojoin_thread_on_open: true
# Should inline fields in Discord embeds be bridged as HTML tables to Matrix?
# Tables aren't supported in all clients, but are the only way to emulate the Discord inline field UI.
embed_fields_as_tables: true
# Should guild channels be muted when the portal is created? This only meant for single-user instances,
# it won't mute it for all users if there are multiple Matrix users in the same Discord guild.
mute_channels_on_create: false
# Should the bridge update the m.direct account data event when double puppeting is enabled.
# Note that updating the m.direct event is not atomic (except with mautrix-asmux)
# and is therefore prone to race conditions.
sync_direct_chat_list: false
# Set this to true to tell the bridge to re-send m.bridge events to all rooms on the next run.
# This field will automatically be changed back to false after it, except if the config file is not writable.
resend_bridge_info: false
# Should incoming custom emoji reactions be bridged as mxc:// URIs?
# If set to false, custom emoji reactions will be bridged as the shortcode instead, and the image won't be available.
custom_emoji_reactions: true
# Should the bridge attempt to completely delete portal rooms when a channel is deleted on Discord?
# If true, the bridge will try to kick Matrix users from the room. Otherwise, the bridge only makes ghosts leave.
delete_portal_on_channel_delete: false
# Should the bridge delete all portal rooms when you leave a guild on Discord?
# This only applies if the guild has no other Matrix users on this bridge instance.
delete_guild_on_leave: true
# Whether or not created rooms should have federation enabled.
# If false, created portal rooms will never be federated.
federate_rooms: true
# Prefix messages from webhooks with the profile info? This can be used along with a custom displayname_template
# to better handle webhooks that change their name all the time (like ones used by bridges).
#
# This will use the fallback mode in MSC4144, which means clients that support MSC4144 will not show the prefix
# (and will instead show the name and avatar as the message sender).
prefix_webhook_messages: true
# Bridge webhook avatars?
enable_webhook_avatars: false
# Should the bridge upload media to the Discord CDN directly before sending the message when using a user token,
# like the official client does? The other option is sending the media in the message send request as a form part
# (which is always used by bots and webhooks).
use_discord_cdn_upload: true
# Should the bridge forbid direct messages from users to other users who they aren't friends with? Discord generally
# considers this to be a "risky" action. Note that the bridge will conservatively reject all outgoing DMs from users
# until it has synced that user's relationships from Discord.
forbid_dming_strangers: true
# Proxy for Discord connections
proxy:
# Should mxc uris copied from Discord be cached?
# This can be `never` to never cache, `unencrypted` to only cache unencrypted mxc uris, or `always` to cache everything.
# If you have a media repo that generates non-unique mxc uris, you should set this to never.
cache_media: unencrypted
# Settings for converting Discord media to custom mxc:// URIs instead of reuploading.
# More details can be found at https://docs.mau.fi/bridges/go/discord/direct-media.html
direct_media:
# Should custom mxc:// URIs be used instead of reuploading media?
enabled: false
# The server name to use for the custom mxc:// URIs.
# This server name will effectively be a real Matrix server, it just won't implement anything other than media.
# You must either set up .well-known delegation from this domain to the bridge, or proxy the domain directly to the bridge.
server_name: discord-media.example.com
# Optionally a custom .well-known response. This defaults to `server_name:443`
well_known_response:
# The bridge supports MSC3860 media download redirects and will use them if the requester supports it.
# Optionally, you can force redirects and not allow proxying at all by setting this to false.
allow_proxy: true
# Matrix server signing key to make the federation tester pass, same format as synapse's .signing.key file.
# This key is also used to sign the mxc:// URIs to ensure only the bridge can generate them.
server_key: generate
# Settings for converting animated stickers.
animated_sticker:
# Format to which animated stickers should be converted.
# disable - No conversion, send as-is (lottie JSON)
# png - converts to non-animated png (fastest)
# gif - converts to animated gif
# webm - converts to webm video, requires ffmpeg executable with vp9 codec and webm container support
# webp - converts to animated webp, requires ffmpeg executable with webp codec/container support
target: webp
# Arguments for converter. All converters take width and height.
args:
width: 320
height: 320
fps: 25 # only for webm, webp and gif (2, 5, 10, 20 or 25 recommended)
# Servers to always allow double puppeting from
double_puppet_server_map:
example.com: https://example.com
# Allow using double puppeting from any server with a valid client .well-known file.
double_puppet_allow_discovery: false
# Shared secrets for https://github.com/devture/matrix-synapse-shared-secret-auth
#
# If set, double puppeting will be enabled automatically for local users
# instead of users having to find an access token and run `login-matrix`
# manually.
login_shared_secret_map:
example.com: foobar
# The prefix for commands. Only required in non-management rooms.
command_prefix: '!discord'
# Messages sent upon joining a management room.
# Markdown is supported. The defaults are listed below.
management_room_text:
# Sent when joining a room.
welcome: "Hello, I'm a Discord bridge bot."
# Sent when joining a management room and the user is already logged in.
welcome_connected: "Use `help` for help."
# Sent when joining a management room and the user is not logged in.
welcome_unconnected: "Use `help` for help or `login` to log in."
# Optional extra text sent when joining a management room.
additional_help: ""
# Settings for backfilling messages.
backfill:
# Limits for forward backfilling.
forward_limits:
# Initial backfill (when creating portal). 0 means backfill is disabled.
# A special unlimited value is not supported, you must set a limit. Initial backfill will
# fetch all messages first before backfilling anything, so high limits can take a lot of time.
initial:
dm: 0
channel: 0
thread: 0
# Missed message backfill (on startup).
# 0 means backfill is disabled, -1 means fetch all messages since last bridged message.
# When using unlimited backfill (-1), messages are backfilled as they are fetched.
# With limits, all messages up to the limit are fetched first and backfilled afterwards.
missed:
dm: 0
channel: -1
thread: 0
# Maximum members in a guild to enable backfilling. Set to -1 to disable limit.
# This can be used as a rough heuristic to disable backfilling in channels that are too active.
# Currently only applies to missed message backfill.
max_guild_members: -1
# End-to-bridge encryption support options.
#
# See https://docs.mau.fi/bridges/general/end-to-bridge-encryption.html for more info.
encryption:
# Allow encryption, work in group chat rooms with e2ee enabled
allow: true
# Default to encryption, force-enable encryption in all portals the bridge creates
# This will cause the bridge bot to be in private chats for the encryption to work properly.
default: false
# Whether to use MSC2409/MSC3202 instead of /sync long polling for receiving encryption-related data.
# Changing this option requires updating the appservice registration file.
appservice: false
# Whether to use MSC4190 instead of appservice login to create the bridge bot device.
# Requires the homeserver to support MSC4190 and the device masquerading parts of MSC3202.
# Only relevant when using end-to-bridge encryption, required when using encryption with next-gen auth (MSC3861).
# Changing this option requires updating the appservice registration file.
msc4190: false
# Require encryption, drop any unencrypted messages.
require: false
# Enable key sharing? If enabled, key requests for rooms where users are in will be fulfilled.
# You must use a client that supports requesting keys from other users to use this feature.
allow_key_sharing: false
# Should users mentions be in the event wire content to enable the server to send push notifications?
plaintext_mentions: false
# Options for deleting megolm sessions from the bridge.
delete_keys:
# Beeper-specific: delete outbound sessions when hungryserv confirms
# that the user has uploaded the key to key backup.
delete_outbound_on_ack: false
# Don't store outbound sessions in the inbound table.
dont_store_outbound: false
# Ratchet megolm sessions forward after decrypting messages.
ratchet_on_decrypt: false
# Delete fully used keys (index >= max_messages) after decrypting messages.
delete_fully_used_on_decrypt: false
# Delete previous megolm sessions from same device when receiving a new one.
delete_prev_on_new_session: false
# Delete megolm sessions received from a device when the device is deleted.
delete_on_device_delete: false
# Periodically delete megolm sessions when 2x max_age has passed since receiving the session.
periodically_delete_expired: false
# Delete inbound megolm sessions that don't have the received_at field used for
# automatic ratcheting and expired session deletion. This is meant as a migration
# to delete old keys prior to the bridge update.
delete_outdated_inbound: false
# What level of device verification should be required from users?
#
# Valid levels:
# unverified - Send keys to all device in the room.
# cross-signed-untrusted - Require valid cross-signing, but trust all cross-signing keys.
# cross-signed-tofu - Require valid cross-signing, trust cross-signing keys on first use (and reject changes).
# cross-signed-verified - Require valid cross-signing, plus a valid user signature from the bridge bot.
# Note that creating user signatures from the bridge bot is not currently possible.
# verified - Require manual per-device verification
# (currently only possible by modifying the `trust` column in the `crypto_device` database table).
verification_levels:
# Minimum level for which the bridge should send keys to when bridging messages from WhatsApp to Matrix.
receive: unverified
# Minimum level that the bridge should accept for incoming Matrix messages.
send: unverified
# Minimum level that the bridge should require for accepting key requests.
share: cross-signed-tofu
# Options for Megolm room key rotation. These options allow you to
# configure the m.room.encryption event content. See:
# https://spec.matrix.org/v1.3/client-server-api/#mroomencryption for
# more information about that event.
rotation:
# Enable custom Megolm room key rotation settings. Note that these
# settings will only apply to rooms created after this option is
# set.
enable_custom: false
# The maximum number of milliseconds a session should be used
# before changing it. The Matrix spec recommends 604800000 (a week)
# as the default.
milliseconds: 604800000
# The maximum number of messages that should be sent with a given a
# session before changing it. The Matrix spec recommends 100 as the
# default.
messages: 100
# Disable rotating keys when a user's devices change?
# You should not enable this option unless you understand all the implications.
disable_device_change_key_rotation: false
# Settings for provisioning API
provisioning:
# Prefix for the provisioning API paths.
prefix: /_matrix/provision
# Shared secret for authentication. If set to "generate", a random secret will be generated,
# or if set to "disable", the provisioning API will be disabled.
shared_secret: generate
# Enable debug API at /debug with provisioning authentication.
debug_endpoints: false
# Permissions for using the bridge.
# Permitted values:
# relay - Talk through the relaybot (if enabled), no access otherwise
# user - Access to use the bridge to chat with a Discord account.
# admin - User level and some additional administration tools
# Permitted keys:
# * - All Matrix users
# domain - All users on that homeserver
# mxid - Specific user
permissions:
"*": relay
"matrix.lotusguild.org": user
"@jared:matrix.lotusguild.org": admin
# Logging config. See https://github.com/tulir/zeroconfig for details.
logging:
min_level: info
writers:
- type: stdout
format: pretty-colored
- type: file
format: json
filename: /var/log/mautrix-discord/mautrix-discord.log
max_size: 100
max_backups: 10
compress: true
+56
View File
@@ -0,0 +1,56 @@
#!/bin/bash
# One-time install of the Discord presence service (lotus_presence.py).
# Run on LXC 151 with this repo's main branch pulled to /opt/matrix-config:
# bash /opt/matrix-config/discord-bridge/install-presence.sh
# Generates its own appservice credentials on this machine and never prints them.
# Does NOT restart Synapse: until you do, Discord works but Matrix answers 401.
# Safe to re-run: existing credentials are kept.
set -euo pipefail
REPO=/opt/matrix-config
REG=/etc/matrix-synapse/discord-presence-registration.yaml
ENV_FILE=/etc/lotus-presence/env
HS=/etc/matrix-synapse/homeserver.yaml
install -d -m750 -o root -g mautrix-discord /etc/lotus-presence
install -D -m644 "$REPO/discord-bridge/lotus_presence.py" /opt/lotus-presence/lotus_presence.py
if [ ! -f "$ENV_FILE" ]; then
AS_TOKEN="$(openssl rand -hex 32)"
HS_TOKEN="$(openssl rand -hex 32)"
(umask 027; printf 'AS_TOKEN=%s\n' "$AS_TOKEN" > "$ENV_FILE")
chown root:mautrix-discord "$ENV_FILE"
(umask 027; cat > "$REG" <<YAML
# Discord presence service. Non-exclusive on the ghost namespace: it updates
# presence on ghosts the text bridge owns, and cannot create them.
id: lotus-discord-presence
url: null
as_token: $AS_TOKEN
hs_token: $HS_TOKEN
sender_localpart: discordpresence
rate_limited: false
namespaces:
users:
- regex: '^@discord_[0-9]+:matrix\.lotusguild\.org$'
exclusive: false
rooms: []
aliases: []
YAML
)
chown root:matrix-synapse "$REG"
echo "generated appservice credentials"
else
echo "keeping existing credentials"
fi
if ! grep -qF "$REG" "$HS"; then
cp -p "$HS" "$HS.bak-presence-$(date +%Y%m%d%H%M%S)"
sed -i "/^app_service_config_files:/a\\ - $REG" "$HS"
echo "added the registration to homeserver.yaml (takes effect on the next Synapse restart)"
fi
install -m644 "$REPO/systemd/lotus-discord-presence.service" /etc/systemd/system/lotus-discord-presence.service
systemctl daemon-reload
systemctl enable --now lotus-discord-presence
systemctl restart lotus-discord-presence
echo "installed. Restart Synapse once to activate the registration: systemctl restart matrix-synapse"
+103
View File
@@ -0,0 +1,103 @@
#!/bin/bash
# One-time install of the mautrix-discord bridge.
# Run on the Proxmox node that hosts LXC 109 (PostgreSQL) and LXC 151 (Synapse):
# pct exec 151 -- cat /opt/matrix-config/discord-bridge/install.sh | bash
# Needs this repo's main branch already pulled to /opt/matrix-config on LXC 151
# (the auto-deploy does that). Restarts Synapse once (~30-60s of downtime).
# Safe to re-run: existing database, user, config and registration are kept.
set -euo pipefail
VERSION="v0.7.7"
SHA256="1f3b6953b4eae00d141ce706f6e97c1b5d0e50268a447bc605df2fa4f44c5d9d"
PG_CT=109
SYNAPSE_CT=151
SYNAPSE_IP="10.10.10.29"
# Only used if the database role does not exist yet; never printed.
DB_PASSWORD="$(openssl rand -hex 24 </dev/null)"
echo "== [1/3] PostgreSQL (LXC $PG_CT): role, database, pg_hba =="
pct exec "$PG_CT" -- bash -s "$DB_PASSWORD" "$SYNAPSE_IP" <<'PG'
set -euo pipefail
PW="$1"; SRC="$2"
HBA=/etc/postgresql/17/main/pg_hba.conf
if su postgres -c "psql -Atc \"select 1 from pg_roles where rolname='mautrix_discord'\"" | grep -q 1; then
echo "role mautrix_discord exists, keeping it"
echo "EXISTS" > /root/.mautrix-discord-role-existed
else
su postgres -c "psql -v ON_ERROR_STOP=1 -q" <<SQL
CREATE ROLE mautrix_discord LOGIN PASSWORD '$PW';
SQL
rm -f /root/.mautrix-discord-role-existed
fi
if ! su postgres -c "psql -Atc \"select 1 from pg_database where datname='mautrix_discord'\"" | grep -q 1; then
su postgres -c "psql -v ON_ERROR_STOP=1 -q -c \"CREATE DATABASE mautrix_discord OWNER mautrix_discord ENCODING 'UTF8' LC_COLLATE 'C' LC_CTYPE 'C' TEMPLATE template0\""
fi
if ! grep -q '^host *mautrix_discord ' "$HBA"; then
cp "$HBA" "$HBA.bak-$(date +%Y%m%d%H%M%S)-discord"
sed -i "/^host *synapse *synapse_user/a host mautrix_discord mautrix_discord $SRC/32 scram-sha-256" "$HBA"
fi
su postgres -c "psql -Atqc 'select pg_reload_conf()'" >/dev/null
echo "ok"
PG
ROLE_EXISTED=$(pct exec "$PG_CT" -- sh -c 'test -f /root/.mautrix-discord-role-existed && echo yes || echo no' </dev/null)
echo "== [2/3] Bridge (LXC $SYNAPSE_CT): binary, config, registration, service =="
pct exec "$SYNAPSE_CT" -- bash -s "$DB_PASSWORD" "$ROLE_EXISTED" "$VERSION" "$SHA256" <<'BR'
set -euo pipefail
PW="$1"; ROLE_EXISTED="$2"; VERSION="$3"; SHA256="$4"
DIR=/opt/mautrix-discord
SRC=/opt/matrix-config
id mautrix-discord >/dev/null 2>&1 || useradd --system --home "$DIR" --shell /usr/sbin/nologin mautrix-discord
mkdir -p "$DIR" /var/log/mautrix-discord
curl -fsSL -o "$DIR/mautrix-discord.new" "https://github.com/mautrix/discord/releases/download/$VERSION/mautrix-discord-amd64"
echo "$SHA256 $DIR/mautrix-discord.new" | sha256sum -c -
chmod 755 "$DIR/mautrix-discord.new"
mv "$DIR/mautrix-discord.new" "$DIR/mautrix-discord"
if [ ! -f "$DIR/config.yaml" ]; then
if [ "$ROLE_EXISTED" = yes ]; then
echo "ERROR: DB role already existed but there is no config; set the password in $DIR/config.yaml by hand" >&2
exit 1
fi
sed "s/DB_PASSWORD/$PW/" "$SRC/discord-bridge/config.yaml" > "$DIR/config.yaml"
fi
if [ ! -f "$DIR/registration.yaml" ]; then
(cd "$DIR" && ./mautrix-discord -g -c config.yaml -r registration.yaml)
fi
chown -R mautrix-discord:mautrix-discord "$DIR" /var/log/mautrix-discord
chmod 600 "$DIR/config.yaml"
install -m 640 -o root -g matrix-synapse "$DIR/registration.yaml" /etc/matrix-synapse/discord-registration.yaml
install -m 644 "$SRC/systemd/mautrix-discord.service" /etc/systemd/system/mautrix-discord.service
systemctl daemon-reload
HS=/etc/matrix-synapse/homeserver.yaml
if ! grep -q discord-registration.yaml "$HS"; then
cp "$HS" "$HS.bak-$(date +%Y%m%d%H%M%S)-discord"
sed -i '/^ - \/etc\/matrix-synapse\/hookshot-registration.yaml/a \ - /etc/matrix-synapse/discord-registration.yaml' "$HS"
fi
grep -A3 '^app_service_config_files' "$HS"
echo "ok"
BR
echo "== [3/3] Restart Synapse, start bridge =="
pct exec "$SYNAPSE_CT" -- bash -s <<'RS'
set -euo pipefail
systemctl restart matrix-synapse
for i in $(seq 1 60); do
curl -fsS -o /dev/null http://127.0.0.1:8008/health && break
sleep 2
done
curl -fsS http://127.0.0.1:8008/health && echo " (synapse healthy)"
systemctl enable --now mautrix-discord
sleep 5
systemctl --no-pager --lines=15 status mautrix-discord
RS
pct exec "$PG_CT" -- rm -f /root/.mautrix-discord-role-existed </dev/null
echo
echo "Done. In Lotus Chat, start a DM with @discordbot:matrix.lotusguild.org and send:"
echo " login-token bot <your new bot token>"
+335
View File
@@ -0,0 +1,335 @@
#!/usr/bin/env python3
"""Discord presence -> Matrix ghosts.
mautrix-discord doesn't bridge presence, so this small service does: it watches
the Discord server over the gateway (GUILD_PRESENCES) and sets presence and a
status message on each member's @discord_<id> ghost, which Lotus already shows.
- Discord bot: the text bridge's own bot, read from its database at startup, so
there is no extra bot. This is a second gateway session asking only for the
intents it needs; the bridge's session is untouched.
- Matrix: its own appservice registration (non-exclusive on the ghost namespace;
the bridge still owns the ghosts). Only ghosts that already exist are updated.
- Synapse drops a user nobody refreshes to offline, so live states are re-sent
on a timer. That is also the failure mode we want: if this service dies, the
ghosts go offline rather than showing a stale "online" forever.
Member statuses and activities are never logged.
"""
import asyncio
import logging
import os
import re
import subprocess
import sys
import time
import urllib.parse
import aiohttp
log = logging.getLogger("lotus-presence")
GATEWAY = "wss://gateway.discord.gg/?v=10&encoding=json"
INTENT_GUILDS = 1 << 0
INTENT_GUILD_PRESENCES = 1 << 8
# Close codes where reconnecting cannot help: bad token, disallowed intents.
FATAL_CLOSE = {4004, 4013, 4014}
BUSY = "org.matrix.msc3026.busy"
STATUS_MAP = {"online": "online", "idle": "unavailable", "dnd": BUSY}
MAX_STATUS = 120
# A ghost that doesn't exist yet (never spoke in a bridged room) is retried this often.
UNKNOWN_RETRY_S = 600
def describe_activity(activity):
"""One Discord activity as text, or None. Types per Discord's Activity object."""
kind = activity.get("type")
name = (activity.get("name") or "").strip()
if kind == 4: # custom status
emoji = (activity.get("emoji") or {})
# Custom server emoji have an id and can't be shown as text.
glyph = emoji.get("name") if not emoji.get("id") else None
text = (activity.get("state") or "").strip()
return " ".join(p for p in (glyph, text) if p) or None
if not name:
return None
if kind == 0:
return f"Playing {name}"
if kind == 1:
return f"Streaming {activity.get('details') or name}"
if kind == 2:
song, artist = activity.get("details"), activity.get("state")
if song and artist:
return f"Listening to {song} by {artist}"
return f"Listening to {song or name}"
if kind == 3:
return f"Watching {name}"
if kind == 5:
return f"Competing in {name}"
return None
def map_presence(status, activities):
"""Discord status + activities -> (Matrix presence, status message)."""
presence = STATUS_MAP.get(status)
if presence is None: # offline / invisible: show nothing about them
return "offline", ""
custom = [a for a in activities or [] if a.get("type") == 4]
others = [a for a in activities or [] if a.get("type") != 4]
parts = []
for activity in custom + others[:1]:
text = describe_activity(activity)
if text:
parts.append(text)
message = " · ".join(parts)
if len(message) > MAX_STATUS:
message = message[: MAX_STATUS - 1].rstrip() + "…"
return presence, message
def ghost_id(discord_id, server_name):
return f"@discord_{discord_id}:{server_name}"
def bridge_db_env(config_text):
"""libpq settings from the bridge config's database URI (password stays out of argv)."""
match = re.search(r"^\s*uri:\s*[\"']?(postgres(?:ql)?://[^\"'\s]+)", config_text, re.M)
if not match:
raise RuntimeError("no postgres uri in the bridge config")
uri = urllib.parse.urlparse(match.group(1))
env = {
"PGHOST": uri.hostname or "",
"PGPORT": str(uri.port or 5432),
"PGUSER": urllib.parse.unquote(uri.username or ""),
"PGPASSWORD": urllib.parse.unquote(uri.password or ""),
"PGDATABASE": uri.path.lstrip("/"),
}
sslmode = urllib.parse.parse_qs(uri.query).get("sslmode")
if sslmode:
env["PGSSLMODE"] = sslmode[0]
return env
def read_bot_token(config_path):
"""The text bridge's bot token, from its database. Never logged."""
with open(config_path, encoding="utf-8") as handle:
env = {**os.environ, **bridge_db_env(handle.read())}
out = subprocess.run(
["psql", "-Atc", "select discord_token from \"user\" where discord_token like 'Bot %'"],
env=env,
capture_output=True,
text=True,
timeout=30,
check=False,
)
if out.returncode != 0:
raise RuntimeError("could not read the bot token from the bridge database")
tokens = [line for line in out.stdout.splitlines() if line]
if len(tokens) != 1:
raise RuntimeError(f"expected exactly one bridge bot login, found {len(tokens)}")
return tokens[0].removeprefix("Bot ")
class GhostPresence:
"""What each ghost should show, pushed to Synapse on change and on a timer."""
def __init__(self, session, homeserver, as_token, server_name, clock=time.monotonic):
self.session = session
self.homeserver = homeserver.rstrip("/")
self.as_token = as_token
self.server_name = server_name
self.clock = clock
self.live = {} # discord id -> (presence, message), everything not offline
self.unknown_until = {} # discord id -> when to try a missing ghost again
async def _put(self, discord_id, presence, message):
user = urllib.parse.quote(ghost_id(discord_id, self.server_name), safe="")
url = f"{self.homeserver}/_matrix/client/v3/presence/{user}/status?user_id={user}"
body = {"presence": presence, "status_msg": message}
headers = {"Authorization": f"Bearer {self.as_token}"}
try:
async with self.session.put(
url, json=body, headers=headers, timeout=aiohttp.ClientTimeout(total=10)
) as resp:
# Synapse answers 403 for a user the appservice hasn't registered.
if resp.status == 403:
self.unknown_until[discord_id] = self.clock() + UNKNOWN_RETRY_S
elif resp.status >= 400:
log.warning("presence update failed: HTTP %s", resp.status)
else:
self.unknown_until.pop(discord_id, None)
except (aiohttp.ClientError, asyncio.TimeoutError) as err:
log.warning("presence update failed: %s", type(err).__name__)
def _skip(self, discord_id):
return self.unknown_until.get(discord_id, 0) > self.clock()
async def update(self, discord_id, presence, message):
known = self.live.get(discord_id)
if presence == "offline":
if known is None:
return
del self.live[discord_id]
else:
if known == (presence, message):
return
self.live[discord_id] = (presence, message)
if not self._skip(discord_id):
await self._put(discord_id, presence, message)
async def resync(self, current):
"""A full member snapshot: `current` replaces what we had."""
for discord_id in set(self.live) - set(current):
await self.update(discord_id, "offline", "")
for discord_id, (presence, message) in current.items():
await self.update(discord_id, presence, message)
async def refresh(self):
for discord_id, (presence, message) in list(self.live.items()):
if not self._skip(discord_id):
await self._put(discord_id, presence, message)
class FatalGatewayError(Exception):
pass
class Gateway:
def __init__(self, session, token, guild_id, ghosts):
self.session = session
self.token = token
self.guild_id = guild_id
self.ghosts = ghosts
self.seq = None
self.session_id = None
self.resume_url = None
self.acked = True
async def _heartbeat(self, ws, interval_s):
while True:
await asyncio.sleep(interval_s)
if not self.acked:
log.warning("gateway missed a heartbeat ack, reconnecting")
await ws.close(code=4000)
return
self.acked = False
await ws.send_json({"op": 1, "d": self.seq})
async def _dispatch(self, kind, data):
if kind == "READY":
self.session_id = data["session_id"]
self.resume_url = data.get("resume_gateway_url")
log.info("gateway ready")
elif kind == "GUILD_CREATE" and data.get("id") == self.guild_id:
current = {}
for item in data.get("presences") or []:
presence, message = map_presence(item.get("status"), item.get("activities"))
if presence != "offline":
current[item["user"]["id"]] = (presence, message)
await self.ghosts.resync(current)
log.info("synced %d online members", len(current))
elif kind == "PRESENCE_UPDATE" and data.get("guild_id") == self.guild_id:
presence, message = map_presence(data.get("status"), data.get("activities"))
await self.ghosts.update(data["user"]["id"], presence, message)
async def _run_once(self):
resuming = bool(self.session_id and self.resume_url)
url = f"{self.resume_url}/?v=10&encoding=json" if resuming else GATEWAY
beat = None
async with self.session.ws_connect(url, heartbeat=None, max_msg_size=0) as ws:
try:
async for frame in ws:
if frame.type != aiohttp.WSMsgType.TEXT:
break
msg = frame.json()
if msg.get("s") is not None:
self.seq = msg["s"]
op = msg.get("op")
if op == 10:
self.acked = True
interval = msg["d"]["heartbeat_interval"] / 1000
beat = asyncio.create_task(self._heartbeat(ws, interval))
if resuming:
await ws.send_json({"op": 6, "d": {
"token": self.token,
"session_id": self.session_id,
"seq": self.seq,
}})
else:
await ws.send_json({"op": 2, "d": {
"token": self.token,
"intents": INTENT_GUILDS | INTENT_GUILD_PRESENCES,
"properties": {
"os": "linux",
"browser": "lotus-presence",
"device": "lotus-presence",
},
}})
elif op == 11:
self.acked = True
elif op == 1:
await ws.send_json({"op": 1, "d": self.seq})
elif op == 7:
return
elif op == 9:
if not msg.get("d"):
self.session_id = None
return
elif op == 0:
await self._dispatch(msg.get("t"), msg.get("d") or {})
finally:
if beat:
beat.cancel()
if ws.close_code in FATAL_CLOSE:
raise FatalGatewayError(f"gateway refused us (close code {ws.close_code})")
if ws.close_code:
log.info("gateway closed (code %s)", ws.close_code)
async def run(self):
backoff = 1
while True:
started = time.monotonic()
try:
await self._run_once()
except (aiohttp.ClientError, asyncio.TimeoutError, OSError) as err:
log.warning("gateway connection failed: %s", type(err).__name__)
if time.monotonic() - started > 60:
backoff = 1
await asyncio.sleep(backoff)
backoff = min(backoff * 2, 60)
async def refresh_loop(ghosts, interval_s):
while True:
await asyncio.sleep(interval_s)
await ghosts.refresh()
async def amain():
token = read_bot_token(os.environ["BRIDGE_CONFIG"])
async with aiohttp.ClientSession() as session:
ghosts = GhostPresence(
session,
os.environ.get("HOMESERVER_URL", "http://127.0.0.1:8008"),
os.environ["AS_TOKEN"],
os.environ["SERVER_NAME"],
)
gateway = Gateway(session, token, os.environ["DISCORD_GUILD_ID"], ghosts)
refresh_s = float(os.environ.get("REFRESH_SECONDS", "20"))
await asyncio.gather(gateway.run(), refresh_loop(ghosts, refresh_s))
def main():
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
# Discord's own clients log request URLs at DEBUG; keep ours quiet regardless.
logging.getLogger("aiohttp").setLevel(logging.WARNING)
try:
asyncio.run(amain())
except FatalGatewayError as err:
log.error("%s", err)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,253 @@
diff --git a/lotus_retry.go b/lotus_retry.go
new file mode 100644
index 0000000..972b612
--- /dev/null
+++ b/lotus_retry.go
@@ -0,0 +1,95 @@
+package main
+
+// [lotus] Retry Matrix→Discord sends that failed before reaching Discord.
+//
+// Upstream sends each message exactly once; a momentary network blip (for
+// example "net/http: TLS handshake timeout") drops the message and only posts
+// a "may not have been bridged" notice. We retry, but only for failures that
+// happen before the HTTP request is written (DNS, dial, TLS handshake). Those
+// cannot have reached Discord, so a retry can never post a duplicate.
+// Failures after the request was sent (read timeouts, resets, 5xx) are left
+// alone because webhook sends have no server-side dedupe.
+//
+// The retry runs inline in the portal's Matrix event loop, which handles a
+// room's events one at a time, so later messages queue behind it and ordering
+// is preserved.
+
+import (
+ "errors"
+ "io"
+ "net"
+ "strings"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+ "github.com/rs/zerolog"
+)
+
+var lotusSendRetryDelays = []time.Duration{
+ 2 * time.Second,
+ 5 * time.Second,
+ 10 * time.Second,
+ 20 * time.Second,
+ 30 * time.Second,
+ 60 * time.Second,
+ 60 * time.Second,
+ 60 * time.Second,
+}
+
+// lotusSleep is swapped out in tests.
+var lotusSleep = time.Sleep
+
+// isPreSendNetworkError reports whether err happened before the request could
+// have been written to Discord.
+func isPreSendNetworkError(err error) bool {
+ if err == nil {
+ return false
+ }
+ var restErr *discordgo.RESTError
+ if errors.As(err, &restErr) {
+ // Discord answered, so the request was delivered.
+ return false
+ }
+ var dnsErr *net.DNSError
+ if errors.As(err, &dnsErr) {
+ return true
+ }
+ var opErr *net.OpError
+ if errors.As(err, &opErr) && opErr.Op == "dial" {
+ return true
+ }
+ msg := err.Error()
+ return strings.Contains(msg, "TLS handshake timeout") ||
+ strings.Contains(msg, "proxyconnect")
+}
+
+func lotusRewindFiles(files []*discordgo.File) {
+ for _, f := range files {
+ if f == nil {
+ continue
+ }
+ if s, ok := f.Reader.(io.Seeker); ok {
+ _, _ = s.Seek(0, io.SeekStart)
+ }
+ }
+}
+
+// lotusRetrySend calls send until it succeeds, fails with an error that is not
+// a pre-send network error, or the retry budget runs out.
+func lotusRetrySend[T any](log zerolog.Logger, files []*discordgo.File, send func() (T, error)) (T, error) {
+ res, err := send()
+ for attempt := 0; attempt < len(lotusSendRetryDelays) && isPreSendNetworkError(err); attempt++ {
+ delay := lotusSendRetryDelays[attempt]
+ log.Warn().Err(err).
+ Int("attempt", attempt+1).
+ Dur("retry_in", delay).
+ Msg("[lotus] Discord send failed before reaching Discord, retrying")
+ lotusSleep(delay)
+ lotusRewindFiles(files)
+ res, err = send()
+ if err == nil {
+ log.Info().Int("attempts", attempt+2).Msg("[lotus] Discord send succeeded after retry")
+ }
+ }
+ return res, err
+}
diff --git a/lotus_retry_test.go b/lotus_retry_test.go
new file mode 100644
index 0000000..efa262e
--- /dev/null
+++ b/lotus_retry_test.go
@@ -0,0 +1,108 @@
+package main
+
+import (
+ "bytes"
+ "errors"
+ "io"
+ "net"
+ "net/http"
+ "net/url"
+ "testing"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+ "github.com/rs/zerolog"
+)
+
+type tlsTimeout struct{}
+
+func (tlsTimeout) Error() string { return "net/http: TLS handshake timeout" }
+func (tlsTimeout) Timeout() bool { return true }
+func (tlsTimeout) Temporary() bool { return true }
+
+func urlErr(err error) error {
+ return &url.Error{Op: "Post", URL: "https://discord.com/api/v9/webhooks/1/x?wait=true", Err: err}
+}
+
+func TestIsPreSendNetworkError(t *testing.T) {
+ cases := []struct {
+ name string
+ err error
+ want bool
+ }{
+ {"nil", nil, false},
+ {"tls handshake timeout", urlErr(tlsTimeout{}), true},
+ {"dns", urlErr(&net.DNSError{Err: "no such host", Name: "discord.com"}), true},
+ {"dial refused", urlErr(&net.OpError{Op: "dial", Net: "tcp", Err: errors.New("connection refused")}), true},
+ {"read reset after send", urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")}), false},
+ {"response timeout", urlErr(errors.New("context deadline exceeded (Client.Timeout exceeded while awaiting headers)")), false},
+ {"discord rest error", &discordgo.RESTError{Response: &http.Response{StatusCode: 500}}, false},
+ {"other", errors.New("unknown message type"), false},
+ }
+ for _, c := range cases {
+ if got := isPreSendNetworkError(c.err); got != c.want {
+ t.Errorf("%s: got %v want %v", c.name, got, c.want)
+ }
+ }
+}
+
+func withNoSleep(t *testing.T) *[]time.Duration {
+ var slept []time.Duration
+ old := lotusSleep
+ lotusSleep = func(d time.Duration) { slept = append(slept, d) }
+ t.Cleanup(func() { lotusSleep = old })
+ return &slept
+}
+
+func TestRetrySendRecoversAndRewindsFiles(t *testing.T) {
+ slept := withNoSleep(t)
+ file := &discordgo.File{Name: "a.txt", Reader: bytes.NewReader([]byte("hello"))}
+ calls := 0
+ msg, err := lotusRetrySend(zerolog.Nop(), []*discordgo.File{file}, func() (*discordgo.Message, error) {
+ calls++
+ data, _ := io.ReadAll(file.Reader)
+ if string(data) != "hello" {
+ t.Fatalf("call %d read %q, file not rewound", calls, data)
+ }
+ if calls < 3 {
+ return nil, urlErr(tlsTimeout{})
+ }
+ return &discordgo.Message{ID: "42"}, nil
+ })
+ if err != nil || msg == nil || msg.ID != "42" {
+ t.Fatalf("got msg=%v err=%v", msg, err)
+ }
+ if calls != 3 {
+ t.Fatalf("calls=%d want 3", calls)
+ }
+ if len(*slept) != 2 || (*slept)[0] != 2*time.Second || (*slept)[1] != 5*time.Second {
+ t.Fatalf("slept=%v", *slept)
+ }
+}
+
+func TestRetrySendDoesNotRetryAfterSend(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ _, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")})
+ })
+ if err == nil || calls != 1 {
+ t.Fatalf("calls=%d err=%v, want exactly one attempt", calls, err)
+ }
+}
+
+func TestRetrySendGivesUpAfterBudget(t *testing.T) {
+ slept := withNoSleep(t)
+ calls := 0
+ _, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, urlErr(tlsTimeout{})
+ })
+ if err == nil {
+ t.Fatal("expected error after budget")
+ }
+ if calls != len(lotusSendRetryDelays)+1 || len(*slept) != len(lotusSendRetryDelays) {
+ t.Fatalf("calls=%d slept=%d", calls, len(*slept))
+ }
+}
diff --git a/portal.go b/portal.go
index db26a0e..cc6da0d 100644
--- a/portal.go
+++ b/portal.go
@@ -1743,18 +1743,24 @@ func (portal *Portal) handleMatrixMessage(sender *User, evt *event.Event) {
sendReq.Nonce = generateNonce()
var msg *discordgo.Message
var err error
+ // [lotus] retry sends that failed before reaching Discord (see lotus_retry.go)
+ sendLog := portal.log.With().Str("event_id", evt.ID.String()).Logger()
if !isWebhookSend {
- msg, err = sess.ChannelMessageSendComplex(channelID, &sendReq, portal.RefererOptIfUser(sess, threadID)...)
+ msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
+ return sess.ChannelMessageSendComplex(channelID, &sendReq, portal.RefererOptIfUser(sess, threadID)...)
+ })
} else {
username, avatarURL := portal.getRelayUserMeta(sender)
- msg, err = relayClient.WebhookThreadExecute(portal.RelayWebhookID, portal.RelayWebhookSecret, true, threadID, &discordgo.WebhookParams{
- Content: sendReq.Content,
- Username: username,
- AvatarURL: avatarURL,
- Files: sendReq.Files,
- Components: sendReq.Components,
- Embeds: sendReq.Embeds,
- AllowedMentions: sendReq.AllowedMentions,
+ msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
+ return relayClient.WebhookThreadExecute(portal.RelayWebhookID, portal.RelayWebhookSecret, true, threadID, &discordgo.WebhookParams{
+ Content: sendReq.Content,
+ Username: username,
+ AvatarURL: avatarURL,
+ Files: sendReq.Files,
+ Components: sendReq.Components,
+ Embeds: sendReq.Embeds,
+ AllowedMentions: sendReq.AllowedMentions,
+ })
})
}
sender.handlePossible40002(err)
@@ -0,0 +1,437 @@
diff --git a/lotus_echo.go b/lotus_echo.go
new file mode 100644
index 0000000..0cfdd95
--- /dev/null
+++ b/lotus_echo.go
@@ -0,0 +1,118 @@
+package main
+
+// [lotus] Detect whether a webhook send that errored actually reached Discord.
+//
+// Discord echoes every webhook post back over the gateway as MESSAGE_CREATE.
+// While a relay send is in flight we register a waiter for its channel; the
+// gateway handler (user.go) offers every new message to the waiters before
+// queueing it for the portal. If a send fails ambiguously (the connection broke
+// after the request may have gone out), lotusRetrySend asks the waiter whether
+// the post showed up. If it did, the send is treated as successful and the
+// message ID recorded, so it is neither resent to Discord nor bridged back into
+// Matrix as a new message.
+//
+// The waiter is offered messages from the gateway goroutine, which keeps
+// running while the portal's own loop is blocked in the send.
+
+import (
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+)
+
+// lotusEchoWait is how long to wait for the gateway echo after an ambiguous
+// failure. Echoes normally arrive well under a second after Discord accepts the
+// post.
+var lotusEchoWait = 6 * time.Second
+
+type lotusEchoWaiter struct {
+ match func(*discordgo.Message) bool
+ ch chan *discordgo.Message
+}
+
+var (
+ lotusEchoMu sync.Mutex
+ lotusEchoWaiters = map[string][]*lotusEchoWaiter{}
+)
+
+// lotusNoteGatewayMessage offers a newly created Discord message to any waiter
+// registered for its channel. Called for every MESSAGE_CREATE.
+func lotusNoteGatewayMessage(msg *discordgo.Message) {
+ if msg == nil || msg.WebhookID == "" {
+ return
+ }
+ lotusEchoMu.Lock()
+ defer lotusEchoMu.Unlock()
+ for _, w := range lotusEchoWaiters[msg.ChannelID] {
+ if w.match(msg) {
+ select {
+ case w.ch <- msg:
+ default:
+ }
+ }
+ }
+}
+
+// lotusExpectEcho registers a waiter for channelID. check(wait) returns the
+// matching message if one has arrived, waiting up to lotusEchoWait when wait
+// is true. done must be called when the send is finished.
+func lotusExpectEcho(channelID string, match func(*discordgo.Message) bool) (check func(wait bool) (*discordgo.Message, bool), done func()) {
+ w := &lotusEchoWaiter{match: match, ch: make(chan *discordgo.Message, 1)}
+ lotusEchoMu.Lock()
+ lotusEchoWaiters[channelID] = append(lotusEchoWaiters[channelID], w)
+ lotusEchoMu.Unlock()
+
+ check = func(wait bool) (*discordgo.Message, bool) {
+ if !wait {
+ select {
+ case msg := <-w.ch:
+ return msg, true
+ default:
+ return nil, false
+ }
+ }
+ timer := time.NewTimer(lotusEchoWait)
+ defer timer.Stop()
+ select {
+ case msg := <-w.ch:
+ return msg, true
+ case <-timer.C:
+ return nil, false
+ }
+ }
+ done = func() {
+ lotusEchoMu.Lock()
+ defer lotusEchoMu.Unlock()
+ list := lotusEchoWaiters[channelID]
+ for i, x := range list {
+ if x == w {
+ list = append(list[:i], list[i+1:]...)
+ break
+ }
+ }
+ if len(list) == 0 {
+ delete(lotusEchoWaiters, channelID)
+ } else {
+ lotusEchoWaiters[channelID] = list
+ }
+ }
+ return check, done
+}
+
+// lotusWebhookEchoMatcher matches the gateway echo of a relay post: same
+// webhook, same text (ignoring surrounding whitespace, which Discord trims),
+// same number of attachments, created no earlier than shortly before the send
+// started.
+func lotusWebhookEchoMatcher(webhookID, content string, numFiles int, started time.Time) func(*discordgo.Message) bool {
+ notBefore := started.Add(-2 * time.Second)
+ content = strings.TrimSpace(content)
+ return func(msg *discordgo.Message) bool {
+ if msg.WebhookID != webhookID || strings.TrimSpace(msg.Content) != content || len(msg.Attachments) != numFiles {
+ return false
+ }
+ ts, err := discordgo.SnowflakeTimestamp(msg.ID)
+ return err != nil || !ts.Before(notBefore)
+ }
+}
diff --git a/lotus_echo_test.go b/lotus_echo_test.go
new file mode 100644
index 0000000..6f5ad44
--- /dev/null
+++ b/lotus_echo_test.go
@@ -0,0 +1,153 @@
+package main
+
+import (
+ "errors"
+ "net"
+ "net/http"
+ "strconv"
+ "testing"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+ "github.com/rs/zerolog"
+)
+
+func resetErr() error {
+ return urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")})
+}
+
+func TestIsAmbiguousSendError(t *testing.T) {
+ cases := []struct {
+ name string
+ err error
+ want bool
+ }{
+ {"nil", nil, false},
+ {"tls handshake timeout is pre-send", urlErr(tlsTimeout{}), false},
+ {"read reset", resetErr(), true},
+ {"awaiting headers", urlErr(errors.New("context deadline exceeded (Client.Timeout exceeded while awaiting headers)")), true},
+ {"unexpected EOF", urlErr(errors.New("unexpected EOF")), true},
+ {"discord 502", &discordgo.RESTError{Response: &http.Response{StatusCode: 502}}, true},
+ {"discord 400", &discordgo.RESTError{Response: &http.Response{StatusCode: 400}}, false},
+ {"other", errors.New("unknown message type"), false},
+ }
+ for _, c := range cases {
+ if got := isAmbiguousSendError(c.err); got != c.want {
+ t.Errorf("%s: got %v want %v", c.name, got, c.want)
+ }
+ }
+}
+
+func TestRetrySendAmbiguousWithoutCheckerIsNotRetried(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ _, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, resetErr()
+ }, nil)
+ if err == nil || calls != 1 {
+ t.Fatalf("calls=%d err=%v", calls, err)
+ }
+}
+
+func TestRetrySendAmbiguousButDeliveredIsNotResent(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ msg, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, resetErr()
+ }, func(wait bool) (*discordgo.Message, bool) {
+ return &discordgo.Message{ID: "echo"}, true
+ })
+ if err != nil || msg == nil || msg.ID != "echo" || calls != 1 {
+ t.Fatalf("calls=%d msg=%v err=%v", calls, msg, err)
+ }
+}
+
+func TestRetrySendAmbiguousNotDeliveredIsResent(t *testing.T) {
+ slept := withNoSleep(t)
+ calls := 0
+ var waits []bool
+ msg, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ if calls == 1 {
+ return nil, resetErr()
+ }
+ return &discordgo.Message{ID: "2"}, nil
+ }, func(wait bool) (*discordgo.Message, bool) {
+ waits = append(waits, wait)
+ return nil, false
+ })
+ if err != nil || msg.ID != "2" || calls != 2 || len(*slept) != 1 {
+ t.Fatalf("calls=%d msg=%v err=%v slept=%v", calls, msg, err, *slept)
+ }
+ if len(waits) != 2 || !waits[0] || waits[1] {
+ t.Fatalf("waits=%v, want [true false]", waits)
+ }
+}
+
+func TestRetrySendLateEchoDuringBackoffStopsResend(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ checks := 0
+ msg, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, resetErr()
+ }, func(wait bool) (*discordgo.Message, bool) {
+ checks++
+ if checks == 2 { // the echo arrives during the backoff
+ return &discordgo.Message{ID: "late"}, true
+ }
+ return nil, false
+ })
+ if err != nil || msg.ID != "late" || calls != 1 {
+ t.Fatalf("calls=%d msg=%v err=%v", calls, msg, err)
+ }
+}
+
+func snowflakeAt(ts time.Time) string {
+ ms := ts.UnixMilli() - 1420070400000
+ return strconv.FormatInt(ms<<22, 10)
+}
+
+func TestEchoWaiterMatchesWebhookPost(t *testing.T) {
+ oldWait := lotusEchoWait
+ lotusEchoWait = 200 * time.Millisecond
+ t.Cleanup(func() { lotusEchoWait = oldWait })
+
+ start := time.Now()
+ check, done := lotusExpectEcho("chan1", lotusWebhookEchoMatcher("hook1", "hi", 1, start))
+ defer done()
+
+ if _, ok := check(false); ok {
+ t.Fatal("matched before any message")
+ }
+ // Wrong webhook, wrong content, wrong channel and an old message are ignored.
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start), ChannelID: "chan1", WebhookID: "other", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start), ChannelID: "chan1", WebhookID: "hook1", Content: "nope", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start), ChannelID: "chan2", WebhookID: "hook1", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start.Add(-time.Minute)), ChannelID: "chan1", WebhookID: "hook1", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ if _, ok := check(true); ok {
+ t.Fatal("matched a non-matching message")
+ }
+
+ want := snowflakeAt(start.Add(time.Second))
+ go func() {
+ time.Sleep(20 * time.Millisecond)
+ lotusNoteGatewayMessage(&discordgo.Message{ID: want, ChannelID: "chan1", WebhookID: "hook1", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ }()
+ got, ok := check(true)
+ if !ok || got.ID != want {
+ t.Fatalf("got %v ok=%v", got, ok)
+ }
+}
+
+func TestEchoWaiterDoneUnregisters(t *testing.T) {
+ _, done := lotusExpectEcho("chanX", func(*discordgo.Message) bool { return true })
+ done()
+ lotusEchoMu.Lock()
+ defer lotusEchoMu.Unlock()
+ if _, ok := lotusEchoWaiters["chanX"]; ok {
+ t.Fatal("waiter still registered after done")
+ }
+}
diff --git a/lotus_retry.go b/lotus_retry.go
index 972b612..0c5e147 100644
--- a/lotus_retry.go
+++ b/lotus_retry.go
@@ -74,17 +74,69 @@ func lotusRewindFiles(files []*discordgo.File) {
}
}
+// isAmbiguousSendError reports whether err happened after the request may
+// already have reached Discord: the connection broke or timed out while
+// sending or waiting for the reply, or Discord answered with a 5xx.
+func isAmbiguousSendError(err error) bool {
+ if err == nil || isPreSendNetworkError(err) {
+ return false
+ }
+ var restErr *discordgo.RESTError
+ if errors.As(err, &restErr) {
+ return restErr.Response != nil && restErr.Response.StatusCode >= 500
+ }
+ if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
+ return true
+ }
+ var netErr net.Error
+ if errors.As(err, &netErr) {
+ return true
+ }
+ msg := err.Error()
+ return strings.Contains(msg, "connection reset") ||
+ strings.Contains(msg, "broken pipe") ||
+ strings.Contains(msg, "Client.Timeout exceeded") ||
+ strings.Contains(msg, "server sent GOAWAY") ||
+ strings.Contains(msg, "EOF")
+}
+
// lotusRetrySend calls send until it succeeds, fails with an error that is not
-// a pre-send network error, or the retry budget runs out.
-func lotusRetrySend[T any](log zerolog.Logger, files []*discordgo.File, send func() (T, error)) (T, error) {
+// retryable, or the retry budget runs out.
+//
+// Pre-send network errors are always retried. Ambiguous errors (see
+// isAmbiguousSendError) are retried only when checkDelivered is set: it is
+// called first (wait=true lets it wait briefly for evidence) and, if it finds
+// that the message did reach Discord, its result is returned as a success
+// instead of sending again. It is called again with wait=false just before
+// each resend.
+func lotusRetrySend[T any](log zerolog.Logger, files []*discordgo.File, send func() (T, error), checkDelivered func(wait bool) (T, bool)) (T, error) {
res, err := send()
- for attempt := 0; attempt < len(lotusSendRetryDelays) && isPreSendNetworkError(err); attempt++ {
+ for attempt := 0; attempt < len(lotusSendRetryDelays); attempt++ {
+ switch {
+ case isPreSendNetworkError(err):
+ case checkDelivered != nil && isAmbiguousSendError(err):
+ if delivered, ok := checkDelivered(true); ok {
+ log.Info().Err(err).Int("attempts", attempt+1).
+ Msg("[lotus] Discord send errored but the message was delivered, not resending")
+ return delivered, nil
+ }
+ default:
+ return res, err
+ }
delay := lotusSendRetryDelays[attempt]
log.Warn().Err(err).
Int("attempt", attempt+1).
Dur("retry_in", delay).
- Msg("[lotus] Discord send failed before reaching Discord, retrying")
+ Msg("[lotus] Discord send failed and the message was not delivered, retrying")
lotusSleep(delay)
+ if checkDelivered != nil {
+ // A late echo of an earlier attempt can land during the backoff.
+ if delivered, ok := checkDelivered(false); ok {
+ log.Info().Int("attempts", attempt+1).
+ Msg("[lotus] earlier Discord send turned out to be delivered, not resending")
+ return delivered, nil
+ }
+ }
lotusRewindFiles(files)
res, err = send()
if err == nil {
diff --git a/lotus_retry_test.go b/lotus_retry_test.go
index efa262e..e35585b 100644
--- a/lotus_retry_test.go
+++ b/lotus_retry_test.go
@@ -68,7 +68,7 @@ func TestRetrySendRecoversAndRewindsFiles(t *testing.T) {
return nil, urlErr(tlsTimeout{})
}
return &discordgo.Message{ID: "42"}, nil
- })
+ }, nil)
if err != nil || msg == nil || msg.ID != "42" {
t.Fatalf("got msg=%v err=%v", msg, err)
}
@@ -86,7 +86,7 @@ func TestRetrySendDoesNotRetryAfterSend(t *testing.T) {
_, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
calls++
return nil, urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")})
- })
+ }, nil)
if err == nil || calls != 1 {
t.Fatalf("calls=%d err=%v, want exactly one attempt", calls, err)
}
@@ -98,7 +98,7 @@ func TestRetrySendGivesUpAfterBudget(t *testing.T) {
_, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
calls++
return nil, urlErr(tlsTimeout{})
- })
+ }, nil)
if err == nil {
t.Fatal("expected error after budget")
}
diff --git a/portal.go b/portal.go
index cc6da0d..57ba7f9 100644
--- a/portal.go
+++ b/portal.go
@@ -1748,9 +1748,18 @@ func (portal *Portal) handleMatrixMessage(sender *User, evt *event.Event) {
if !isWebhookSend {
msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
return sess.ChannelMessageSendComplex(channelID, &sendReq, portal.RefererOptIfUser(sess, threadID)...)
- })
+ }, nil)
} else {
username, avatarURL := portal.getRelayUserMeta(sender)
+ // [lotus] watch for the gateway echo so an ambiguous failure can be
+ // checked instead of resent blindly (see lotus_echo.go)
+ echoChannel := channelID
+ if threadID != "" {
+ echoChannel = threadID
+ }
+ checkEcho, echoDone := lotusExpectEcho(echoChannel,
+ lotusWebhookEchoMatcher(portal.RelayWebhookID, sendReq.Content, len(sendReq.Files), time.Now()))
+ defer echoDone()
msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
return relayClient.WebhookThreadExecute(portal.RelayWebhookID, portal.RelayWebhookSecret, true, threadID, &discordgo.WebhookParams{
Content: sendReq.Content,
@@ -1761,7 +1770,7 @@ func (portal *Portal) handleMatrixMessage(sender *User, evt *event.Event) {
Embeds: sendReq.Embeds,
AllowedMentions: sendReq.AllowedMentions,
})
- })
+ }, checkEcho)
}
sender.handlePossible40002(err)
go portal.sendMessageMetrics(evt, err, "Error sending")
diff --git a/user.go b/user.go
index 8c0d6ec..16cd8c8 100644
--- a/user.go
+++ b/user.go
@@ -704,6 +704,7 @@ func (user *User) eventHandler(rawEvt any) {
case *discordgo.RelationshipUpdate:
user.relationshipUpdateHandler(evt)
case *discordgo.MessageCreate:
+ lotusNoteGatewayMessage(evt.Message) // [lotus] see lotus_echo.go
user.pushPortalMessage(evt, "message create", evt.ChannelID, evt.GuildID)
case *discordgo.MessageDelete:
user.pushPortalMessage(evt, "message delete", evt.ChannelID, evt.GuildID)
+34
View File
@@ -0,0 +1,34 @@
#!/bin/bash
# Gives @discordbot the Discord logo and the display name "Discord Bridge".
# Run inside LXC 151 (from the Proxmox node):
# pct exec 151 -- bash /opt/matrix-config/discord-bridge/set-bot-profile.sh
# Uploads bot-avatar.png as the bridge bot, points config.yaml at it and
# restarts the bridge, which applies the profile on startup. Safe to re-run.
set -euo pipefail
DIR=/opt/mautrix-discord
CFG="$DIR/config.yaml"
AVATAR=/opt/matrix-config/discord-bridge/bot-avatar.png
BOT="@discordbot:matrix.lotusguild.org"
AS_TOKEN=$(awk '/^as_token:/ {gsub(/"/, "", $2); print $2; exit}' "$DIR/registration.yaml")
[ -n "$AS_TOKEN" ] || { echo "no as_token in $DIR/registration.yaml" >&2; exit 1; }
MXC=$(curl -fsS -X POST \
-H "Authorization: Bearer $AS_TOKEN" \
-H "Content-Type: image/png" \
--data-binary "@$AVATAR" \
"http://127.0.0.1:8008/_matrix/media/v3/upload?filename=discord.png&user_id=${BOT/@/%40}" |
python3 -c 'import json, sys; print(json.load(sys.stdin)["content_uri"])')
echo "uploaded $MXC"
# Only the bot block's two lines; the 8-space indent is unique to appservice.bot.
sed -i -E \
-e 's|^( displayname: ).*$|\1Discord Bridge|' \
-e "s|^( avatar: )mxc://.*$|\1$MXC|" \
"$CFG"
grep -A7 '^ bot:' "$CFG"
systemctl restart mautrix-discord
sleep 3
systemctl is-active mautrix-discord
+123
View File
@@ -0,0 +1,123 @@
import asyncio
import unittest
import lotus_presence as lp
class MapPresence(unittest.TestCase):
def test_statuses(self):
self.assertEqual(lp.map_presence("online", []), ("online", ""))
self.assertEqual(lp.map_presence("idle", []), ("unavailable", ""))
self.assertEqual(lp.map_presence("dnd", []), (lp.BUSY, ""))
self.assertEqual(lp.map_presence("offline", []), ("offline", ""))
self.assertEqual(lp.map_presence("invisible", None), ("offline", ""))
def test_offline_never_carries_activity(self):
game = [{"type": 0, "name": "Valorant"}]
self.assertEqual(lp.map_presence("offline", game), ("offline", ""))
def test_activities(self):
for activity, expected in (
({"type": 0, "name": "Valorant"}, "Playing Valorant"),
({"type": 1, "name": "Twitch", "details": "Ranked"}, "Streaming Ranked"),
({"type": 2, "name": "Spotify", "details": "Song", "state": "Band"},
"Listening to Song by Band"),
({"type": 2, "name": "Spotify"}, "Listening to Spotify"),
({"type": 3, "name": "YouTube"}, "Watching YouTube"),
({"type": 5, "name": "Arena"}, "Competing in Arena"),
):
self.assertEqual(lp.map_presence("online", [activity])[1], expected)
def test_custom_status_comes_first_with_one_activity(self):
activities = [
{"type": 0, "name": "Valorant"},
{"type": 0, "name": "Second game"},
{"type": 4, "name": "Custom Status", "state": "afk", "emoji": {"name": "🍕"}},
]
self.assertEqual(lp.map_presence("online", activities)[1], "🍕 afk · Playing Valorant")
def test_custom_server_emoji_is_dropped(self):
custom = {"type": 4, "state": "hi", "emoji": {"name": "partyblob", "id": "123"}}
self.assertEqual(lp.map_presence("online", [custom])[1], "hi")
def test_long_status_is_truncated(self):
_, message = lp.map_presence("online", [{"type": 0, "name": "x" * 500}])
self.assertLessEqual(len(message), lp.MAX_STATUS)
self.assertTrue(message.endswith("…"))
class BridgeConfig(unittest.TestCase):
def test_reads_the_database_uri(self):
env = lp.bridge_db_env(
"appservice:\n database:\n type: postgres\n"
' uri: "postgres://u:p%40ss@10.0.0.1:5433/db?sslmode=disable"\n'
)
self.assertEqual(env["PGHOST"], "10.0.0.1")
self.assertEqual(env["PGPORT"], "5433")
self.assertEqual(env["PGPASSWORD"], "p@ss")
self.assertEqual(env["PGDATABASE"], "db")
self.assertEqual(env["PGSSLMODE"], "disable")
def test_missing_uri_is_an_error(self):
with self.assertRaises(RuntimeError):
lp.bridge_db_env("uri: sqlite:///x")
class FakeGhosts(lp.GhostPresence):
def __init__(self, reject=()):
super().__init__(None, "http://hs", "t", "s", clock=lambda: self.now)
self.now = 0
self.sent = []
self.reject = set(reject)
async def _put(self, discord_id, presence, message):
self.sent.append((discord_id, presence, message))
if discord_id in self.reject:
self.unknown_until[discord_id] = self.now + lp.UNKNOWN_RETRY_S
class Ghosts(unittest.TestCase):
def test_only_changes_are_sent(self):
g = FakeGhosts()
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.update("1", "online", "Playing X"))
self.assertEqual(len(g.sent), 2)
def test_offline_is_sent_once_and_only_after_being_live(self):
g = FakeGhosts()
asyncio.run(g.update("1", "offline", ""))
self.assertEqual(g.sent, [])
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.update("1", "offline", ""))
asyncio.run(g.update("1", "offline", ""))
self.assertEqual(g.sent[-1], ("1", "offline", ""))
self.assertEqual(len(g.sent), 2)
def test_resync_turns_off_whoever_left(self):
g = FakeGhosts()
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.resync({"2": ("online", "")}))
self.assertIn(("1", "offline", ""), g.sent)
self.assertEqual(set(g.live), {"2"})
def test_refresh_resends_live_states(self):
g = FakeGhosts()
asyncio.run(g.update("1", "unavailable", "x"))
g.sent.clear()
asyncio.run(g.refresh())
self.assertEqual(g.sent, [("1", "unavailable", "x")])
def test_missing_ghost_is_retried_later_not_every_refresh(self):
g = FakeGhosts(reject={"1"})
asyncio.run(g.update("1", "online", ""))
g.sent.clear()
asyncio.run(g.refresh())
self.assertEqual(g.sent, [])
g.now += lp.UNKNOWN_RETRY_S + 1
asyncio.run(g.refresh())
self.assertEqual(g.sent, [("1", "online", "")])
if __name__ == "__main__":
unittest.main()
-37
View File
@@ -1,37 +0,0 @@
var ev = data.eventType || 'Unknown';
if (ev === 'Test') {
result = { version: 'v2', plain: '🧪 Lidarr: Connection test successful', msgtype: 'm.notice' };
} else {
var artist = (data.artist && data.artist.name) || 'Unknown Artist';
var albums = data.albums || (data.album ? [data.album] : []);
var albumStr = albums.map(function(a){ return a.title || ''; }).filter(Boolean).join(', ');
var quality = (data.release && data.release.quality) || (data.trackFiles && data.trackFiles[0] && data.trackFiles[0].quality) || '';
var releaseGroup = (data.release && data.release.releaseGroup) || '';
var client = data.downloadClient || '';
var upgrade = data.isUpgrade ? ' \u2191upgrade' : '';
var healthMsg = data.message || '';
var healthWiki = data.wikiUrl || '';
var prevVer = data.previousVersion || '';
var newVer = data.newVersion || '';
var emojiMap = { 'Grab':'📥','Download':'✅','Rename':'✏️','ArtistAdd':'➕','ArtistDelete':'🗑️','AlbumAdd':'➕','AlbumDelete':'🗑️','TrackFileDelete':'🗑️','HealthIssue':'⚠️','HealthRestored':'💚','ApplicationUpdate':'🔄' };
var emoji = emojiMap[ev] || '🎵';
var plain, html;
if (ev === 'HealthIssue' || ev === 'HealthRestored') {
plain = emoji + ' Lidarr ' + ev + ': ' + healthMsg + (healthWiki ? '\n' + healthWiki : '');
html = '<b>' + emoji + ' Lidarr ' + ev + '</b>: ' + healthMsg + (healthWiki ? '<br><a href="' + healthWiki + '">Wiki</a>' : '');
} else if (ev === 'ApplicationUpdate') {
plain = emoji + ' Lidarr updated: ' + prevVer + ' \u2192 ' + newVer;
html = '<b>' + emoji + ' Lidarr updated</b>: ' + prevVer + ' \u2192 ' + newVer;
} else if (ev === 'ArtistAdd' || ev === 'ArtistDelete') {
plain = emoji + ' Lidarr ' + ev + ': ' + artist;
html = '<b>' + emoji + ' Lidarr ' + ev + '</b>: ' + artist;
} else {
var albumPart = albumStr ? ' \u2014 ' + albumStr : '';
var qualPart = quality ? ' [' + quality + ']' : '';
var groupPart = releaseGroup ? ' {' + releaseGroup + '}' : '';
var clientPart = client ? ' via ' + client : '';
plain = emoji + ' Lidarr ' + ev + ': ' + artist + albumPart + qualPart + groupPart + upgrade + clientPart;
html = '<b>' + emoji + ' Lidarr ' + ev + '</b>: ' + artist + (albumStr ? ' \u2014 <i>' + albumStr + '</i>' : '') + qualPart + groupPart + upgrade + clientPart;
}
result = { version: 'v2', plain: plain, html: html, msgtype: 'm.notice' };
}
+1 -1
View File
@@ -1616,7 +1616,7 @@
"cells": [
{
"mark": "part",
"note": "OS notifications while the app is open; no background push yet"
"note": "OS notifications while the web or desktop app is open (the desktop app keeps running in the tray); background web push is not planned, so use Element X or FluffyChat on phones"
},
{
"mark": "part",
+1 -1
View File
@@ -1342,7 +1342,7 @@ Client comparison last reviewed <time datetime="2026-09-26">September 26, 2026</
</tr>
<tr>
<td>Push notifications</td>
<td class="ours"><span class="part">~</span><small>OS notifications while the app is open; no background push yet</small></td>
<td class="ours"><span class="part">~</span><small>OS notifications while the web or desktop app is open (the desktop app keeps running in the tray); background web push is not planned, so use Element X or FluffyChat on phones</small></td>
<td><span class="part">~</span><small>only while the tab/app is open; optional email</small></td>
<td><span class="yes">✓</span></td>
<td><span class="yes">✓</span></td>
+37
View File
@@ -495,3 +495,40 @@ class TestReconcileRoomLimit(unittest.TestCase):
guard.livekit_list_participants = lambda a: (_ for _ in ()).throw(AssertionError("listed"))
guard.reconcile_room("alias", "!room:x")
self.assertEqual(self.removed, [])
class LimitExemptTests(unittest.TestCase):
"""A voice bridge's bot listens in the call but isn't a person."""
BOT = "@lotus-voice_bot:matrix.lotusguild.org"
GHOST = "@lotus-voice_1565220670265360514:matrix.lotusguild.org"
def setUp(self):
self._saved = guard.LIMIT_EXEMPT
guard.LIMIT_EXEMPT = guard.re.compile(r"@lotus-voice_bot:matrix\.lotusguild\.org")
def tearDown(self):
guard.LIMIT_EXEMPT = self._saved
def test_bot_does_not_use_a_slot(self):
self.assertFalse(guard.should_block(2, {"@a:x", self.BOT}, "@b:x"))
self.assertTrue(guard.should_block(2, {"@a:x", "@c:x", self.BOT}, "@b:x"))
def test_bot_is_never_blocked(self):
self.assertFalse(guard.should_block(1, {"@a:x"}, self.BOT))
def test_relayed_discord_speakers_still_count(self):
self.assertTrue(guard.should_block(2, {"@a:x", self.GHOST}, "@b:x"))
def test_reconcile_never_removes_or_counts_the_bot(self):
parts = [
{"identity": "@a:x:D1", "joined_at_ms": 1},
{"identity": self.BOT + ":voice-1", "joined_at_ms": 3},
]
removals, users = guard.over_limit_removals(parts, 1, {"@a:x"})
self.assertEqual(removals, [])
self.assertEqual(users, {"@a:x"})
def test_no_exemption_configured_counts_everyone(self):
guard.LIMIT_EXEMPT = None
self.assertTrue(guard.should_block(2, {"@a:x", self.BOT}, "@b:x"))
+17 -3
View File
@@ -52,6 +52,7 @@ import hashlib
import hmac
import json
import os
import re
import threading
import time
import urllib.error
@@ -72,6 +73,11 @@ MATRIX_TOKEN = os.environ.get("MATRIX_TOKEN", "")
# from participants who joined BEFORE a policy tightened. New joins are already
# handled by the JWT re-sign at issue time; this closes the mid-call-flip gap.
RECONCILE_ENABLED = os.environ.get("GUARD_RECONCILE", "1") not in ("0", "false", "")
# Matrix users the participant limit ignores (full-match regex), e.g. a voice
# bridge's own bot, which joins the call to listen but is not a person. The
# Discord speakers the bridge relays (its ghost users) are people and still count.
_EXEMPT = os.environ.get("VOICE_LIMIT_EXEMPT_USERS", "")
LIMIT_EXEMPT = re.compile(_EXEMPT) if _EXEMPT else None
# Floor the interval so a misconfigured 0 can't busy-loop the admin/Synapse APIs.
RECONCILE_INTERVAL = max(0.5, float(os.environ.get("GUARD_RECONCILE_INTERVAL", "3.0")))
@@ -131,18 +137,26 @@ def room_id_from_request(path: str, data: dict) -> str:
return data.get("room") or ""
def limit_exempt(user: str) -> bool:
"""True for users the participant limit ignores (VOICE_LIMIT_EXEMPT_USERS)."""
return bool(LIMIT_EXEMPT and user and LIMIT_EXEMPT.fullmatch(user))
def should_block(limit: int, present_users: set, requester: str) -> bool:
"""Pure decision: should this token be refused?
- limit <= 0 -> never block (no limit configured)
- requester exempt -> never block (e.g. a bridge's bot)
- requester already present -> never block (rejoin / extra device)
- distinct users >= limit -> block
- distinct non-exempt users >= limit -> block
"""
if limit <= 0:
return False
if limit_exempt(requester):
return False
if requester and requester in present_users:
return False
return len(present_users) >= limit
return len({u for u in present_users if not limit_exempt(u)}) >= limit
# [matrix #25] Tokens issued but not yet connected weren't counted, so two
@@ -480,7 +494,7 @@ def over_limit_removals(participants: list, limit: int, seen):
for p in participants:
identity = p.get("identity") or ""
user = matrix_user(identity)
if not user:
if not user or limit_exempt(user):
continue
identities.setdefault(user, []).append(identity)
joined = _joined_at(p)
+5 -5
View File
@@ -27,11 +27,11 @@ Matrix bot and server improvements for the Lotus Guild homeserver (`matrix.lotus
| Service | Host | IP | LXC | Notes |
|---------|------|-----|-----|-------|
| Synapse | micro1 | 10.10.10.29 | 151 | Homeserver + coturn + LiveKit |
| PostgreSQL 17 | micro1 | 10.10.10.44 | 109 | Synapse database backend |
| NPM | large1 | 10.10.10.27 | 139 | Reverse proxy + landing page |
| Authelia | compute-storage-01 | 10.10.10.36 | 167 | SSO/OIDC provider |
| LLDAP | large1 | 10.10.10.39 | 147 | LDAP user directory |
| Synapse | HA (floats) | 10.10.10.29 | 151 | Homeserver + coturn + LiveKit |
| PostgreSQL 17 | HA (floats) | 10.10.10.44 | 109 | Synapse database backend |
| NPM | HA (floats) | 10.10.10.27 | 139 | Reverse proxy + landing page |
| Authelia | HA (floats) | 10.10.10.36 | 167 | SSO/OIDC provider |
| LLDAP | HA (floats) | 10.10.10.39 | 147 | LDAP user directory |
**Key paths on Synapse LXC (10.10.10.29):**
- Synapse config: `/etc/matrix-synapse/homeserver.yaml`
+38 -17
View File
@@ -120,8 +120,19 @@ def is_elevated(client: AsyncClient, room_id: str, user_id: str, min_level: int
# ==================== COMMANDS ====================
@command("help", "Show all available commands")
@command("help", "Show all commands, or details for one (!help <command>)")
async def cmd_help(client: AsyncClient, room_id: str, sender: str, args: str):
# Compact on purpose: the Discord bridge drops messages over 2000 characters.
query = args.strip().lstrip(BOT_PREFIX).lower()
if query:
entry = COMMANDS.get(query)
if entry is None:
await send_text(client, room_id, f"Unknown command: {BOT_PREFIX}{query}. Try {BOT_PREFIX}help for the list.")
return
_, desc = entry
await send_html(client, room_id, f"{BOT_PREFIX}{query} — {desc}", f"<strong>{BOT_PREFIX}{query}</strong> — {desc}")
return
elevated = is_elevated(client, room_id, sender)
categories = [
("🤖 AI / Fun", ["ask", "fortune", "8ball", "roast", "story", "debate"]),
@@ -148,18 +159,17 @@ async def cmd_help(client: AsyncClient, room_id: str, sender: str, args: str):
"setpl", "kick", "purge", "members", "whois", "announce", "syncspace",
]))
plain_lines = ["LotusBot Commands"]
html_parts = ['<font color="#a855f7"><strong>🌸 LotusBot — Commands</strong></font>']
plain_lines = [f"LotusBot commands ({BOT_PREFIX}help <command> for details)"]
html_parts = [
'<font color="#a855f7"><strong>🌸 LotusBot — Commands</strong></font>'
f"<br><sup><em>{BOT_PREFIX}help &lt;command&gt; for details</em></sup>"
]
for cat_name, cmd_names in categories:
plain_lines.append(f"\n{cat_name}")
html_parts.append(f"<br><strong>{cat_name}</strong><ul>")
for name in cmd_names:
if name in COMMANDS:
_, desc = COMMANDS[name]
plain_lines.append(f" {BOT_PREFIX}{name} — {desc}")
html_parts.append(f"<li><strong>{BOT_PREFIX}{name}</strong> — {desc}</li>")
html_parts.append("</ul>")
names = [f"{BOT_PREFIX}{name}" for name in cmd_names if name in COMMANDS]
if not names:
continue
plain_lines.append(f"{cat_name}: " + " ".join(names))
html_parts.append(f"<br><strong>{cat_name}:</strong> " + " ".join(f"<code>{n}</code>" for n in names))
await send_html(client, room_id, "\n".join(plain_lines), "".join(html_parts))
@@ -1313,6 +1323,11 @@ async def cmd_minecraft(client: AsyncClient, room_id: str, sender: str, args: st
await send_text(client, room_id, "Minecraft server is not configured.")
return
remaining = check_cooldown(sender, "minecraft", 60)
if remaining:
await send_text(client, room_id, f"Command on cooldown. Try again in {remaining}s.")
return
await send_text(client, room_id, f"Whitelisting {username}...")
try:
@@ -1327,7 +1342,11 @@ async def cmd_minecraft(client: AsyncClient, room_id: str, sender: str, args: st
loop = asyncio.get_running_loop()
response = await asyncio.wait_for(loop.run_in_executor(None, _rcon), timeout=RCON_TIMEOUT)
logger.info(f"RCON response: {response}")
logger.info(f"Minecraft whitelist: sender={sender} room={room_id} username={username} response={response!r}")
if "does not exist" in response.lower() or "unknown player" in response.lower():
await send_text(client, room_id, f"No Minecraft account named {username} exists. Check the spelling (Java Edition username).")
return
already = "already whitelisted" in response.lower()
status_line = (
@@ -1336,16 +1355,18 @@ async def cmd_minecraft(client: AsyncClient, room_id: str, sender: str, args: st
else f"✅ <strong>{username}</strong> is whitelisted and ready to join."
)
plain = (
f"✿ Minecraft — Lotus SMP ✿\n"
f"✿ Lotus Minecraft Server ✿\n"
f"{'Already whitelisted' if already else 'Whitelisted'}: {username}\n"
f"Server: minecraft.lotusguild.org"
f"Server: minecraft.lotusguild.org\n"
f"Modpack: All the Mods 10 (CurseForge), version 8.6"
)
html = (
f'<font color="#980000"><strong>✿ Minecraft — Lotus SMP ✿</strong></font><br>'
f'<font color="#980000"><strong>✿ Lotus Minecraft Server ✿</strong></font><br>'
f"<blockquote>"
f"{status_line}<br>"
f'<strong>Server:</strong> <code>minecraft.lotusguild.org</code><br>'
f"<sup><em>fresh world · hard mode · no takebacks</em></sup>"
f'<strong>Modpack:</strong> All the Mods 10 (CurseForge), version <code>8.6</code><br>'
f"<sup><em>install the same pack version or the server won't let you in</em></sup>"
f"</blockquote>"
)
await send_html(client, room_id, plain, html)
+177
View File
@@ -0,0 +1,177 @@
#!/usr/bin/env python3
"""Player count -> status message on the Lotus Minecraft bot.
Pings the Minecraft server the way the in-game server list does (no RCON, no
password) and sets the result as the Matrix status of the account the chat
bridge mod logs in as, for example "3 online: Daisybell, Honnehh, Wolf".
- Credentials: the homeserver and token come from the mod's own config, so there
is no extra account or secret. systemd hands this service a private copy.
- The mod syncs with set_presence=offline, so it doesn't keep the account online,
and Synapse drops an unrefreshed user to offline. So state is re-sent on a timer.
- This runs beside the game, not in it, so it can report the server being down.
One failed ping is ignored: a busy modded server can miss one.
"""
import json
import logging
import os
import re
import socket
import struct
import time
import tomllib
import urllib.error
import urllib.parse
import urllib.request
log = logging.getLogger("lotus-mc-status")
INTERVAL = 20 # seconds; Synapse's default limit for this endpoint is one per 10 s
PING_TIMEOUT = 8
MAX_LEN = 64 # what Cinny's own status field allows
FAILS_BEFORE_DOWN = 2
OFFLINE = "Server offline"
PLAYER_NAME = re.compile(r"[A-Za-z0-9_]{1,16}")
def varint(n: int) -> bytes:
out = b""
while True:
byte = n & 0x7F
n >>= 7
out += bytes([byte | (0x80 if n else 0)])
if not n:
return out
def read_varint(read) -> int:
result = 0
for shift in range(0, 35, 7):
byte = read(1)[0]
result |= (byte & 0x7F) << shift
if not byte & 0x80:
return result
raise ValueError("varint too long")
def parse_status(packet: bytes) -> tuple[int, list[str]]:
"""Return (players online, names) from a status response packet."""
pos = [0]
def read(n: int) -> bytes:
chunk = packet[pos[0] : pos[0] + n]
if len(chunk) < n:
raise ValueError("truncated status packet")
pos[0] += n
return chunk
if read_varint(read) != 0:
raise ValueError("not a status response")
players = json.loads(read(read_varint(read)))["players"]
return int(players["online"]), [str(p.get("name", "")) for p in players.get("sample") or []]
def ping(host: str, port: int) -> tuple[int, list[str]]:
with socket.create_connection((host, port), timeout=PING_TIMEOUT) as sock:
addr = host.encode()
hello = b"\0" + varint(767) + varint(len(addr)) + addr + struct.pack(">H", port) + varint(1)
sock.sendall(varint(len(hello)) + hello + b"\x01\0")
def read(n: int) -> bytes:
buf = b""
while len(buf) < n:
chunk = sock.recv(n - len(buf))
if not chunk:
raise ConnectionError("server closed the connection")
buf += chunk
return buf
return parse_status(read(read_varint(read)))
def format_status(online: int, names: list[str], max_len: int = MAX_LEN) -> str:
"""Count plus as many names as fit. Names are sorted so a re-ordered list
doesn't change the status, and anything that isn't a real player name
(servers can send placeholders) is dropped and counted under "more"."""
if online <= 0:
return "Nobody online"
head = f"{online} online"
real = sorted({n for n in names if PLAYER_NAME.fullmatch(n)}, key=str.casefold)[:online]
shown: list[str] = []
for name in real:
rest = online - len(shown) - 1
candidate = f"{head}: {', '.join([*shown, name])}" + (f" +{rest} more" if rest else "")
if len(candidate) > max_len:
break
shown.append(name)
if not shown:
return head
rest = online - len(shown)
return f"{head}: {', '.join(shown)}" + (f" +{rest} more" if rest else "")
def read_config(path: str) -> tuple[str, str]:
with open(path, "rb") as f:
matrix = tomllib.load(f)["matrix"]
return matrix["homeserver"].rstrip("/"), matrix["accessToken"]
def api(method: str, url: str, token: str, body: dict | None = None) -> dict:
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(url, data=data, method=method, headers={
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
"User-Agent": "lotus-mc-status",
})
with urllib.request.urlopen(req, timeout=15) as resp:
return json.load(resp)
def main() -> None:
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
creds = os.environ.get("CREDENTIALS_DIRECTORY")
path = os.path.join(creds, "bridge-config") if creds else os.environ["LOTUS_MC_CONFIG"]
host = os.environ["LOTUS_MC_HOST"]
port = int(os.environ.get("LOTUS_MC_PORT", "25565"))
homeserver, token = read_config(path)
user_id = api("GET", f"{homeserver}/_matrix/client/v3/account/whoami", token)["user_id"]
url = f"{homeserver}/_matrix/client/v3/presence/{urllib.parse.quote(user_id, safe='')}/status"
log.info("watching %s:%d for %s", host, port, user_id)
fails = 0
body = None
last_error = None
last_count = None
while True:
try:
online, names = ping(host, port)
fails = 0
body = {"presence": "online", "status_msg": format_status(online, names)}
if online != last_count:
log.info("%d online", online)
last_count = online
except (OSError, ValueError, KeyError, TypeError) as exc:
fails += 1
if fails == FAILS_BEFORE_DOWN:
log.warning("server not answering: %s", type(exc).__name__)
last_count = None
if fails >= FAILS_BEFORE_DOWN:
body = {"presence": "unavailable", "status_msg": OFFLINE}
if body:
try:
api("PUT", url, token, body)
if last_error:
log.info("Matrix reachable again")
last_error = None
except (OSError, ValueError) as exc:
# HTTPError and URLError are OSErrors. Their text has no token in it.
error = f"HTTP {exc.code}" if isinstance(exc, urllib.error.HTTPError) else type(exc).__name__
if error != last_error:
log.warning("could not set status: %s", error)
last_error = error
time.sleep(INTERVAL)
if __name__ == "__main__":
main()
+77
View File
@@ -0,0 +1,77 @@
import json
import unittest
import lotus_mc_status as s
def packet(players: dict) -> bytes:
body = json.dumps({"version": {"name": "1.21.1"}, "players": players}).encode()
return s.varint(0) + s.varint(len(body)) + body
class FormatStatus(unittest.TestCase):
def test_nobody(self):
self.assertEqual(s.format_status(0, []), "Nobody online")
def test_one(self):
self.assertEqual(s.format_status(1, ["Honnehh"]), "1 online: Honnehh")
def test_sorted_regardless_of_server_order(self):
self.assertEqual(s.format_status(3, ["b_x", "C", "a"]), "3 online: a, b_x, C")
def test_long_names_collapse_and_stay_within_the_limit(self):
names = [f"Player{i}_abcdefghi"[:16] for i in range(10)]
got = s.format_status(10, names)
self.assertLessEqual(len(got), s.MAX_LEN)
self.assertRegex(got, r"^10 online: .+ \+\d+ more$")
shown = got.split(": ")[1].split(" +")[0].split(", ")
self.assertEqual(int(got.split("+")[1].split()[0]), 10 - len(shown))
def test_exact_fit_has_no_more_suffix(self):
got = s.format_status(1, ["A" * 16], max_len=len("1 online: ") + 16)
self.assertEqual(got, "1 online: " + "A" * 16)
def test_more_players_than_the_sample(self):
self.assertEqual(s.format_status(14, ["a", "b"]), "14 online: a, b +12 more")
def test_placeholders_are_not_shown_as_players(self):
self.assertEqual(s.format_status(2, ["Anonymous Player", "a"]), "2 online: a +1 more")
self.assertEqual(s.format_status(1, ["§cinjected name"]), "1 online")
def test_count_only_when_nothing_fits(self):
self.assertEqual(s.format_status(2, ["abcdefgh"], max_len=9), "2 online")
def test_duplicates_and_stale_extras_do_not_overcount(self):
self.assertEqual(s.format_status(1, ["a", "a", "b"]), "1 online: a")
class ParseStatus(unittest.TestCase):
def test_with_sample(self):
p = packet({"max": 20, "online": 2, "sample": [{"name": "a", "id": "x"}, {"name": "b", "id": "y"}]})
self.assertEqual(s.parse_status(p), (2, ["a", "b"]))
def test_without_sample(self):
self.assertEqual(s.parse_status(packet({"max": 20, "online": 0})), (0, []))
def test_rejects_other_packets_and_truncation(self):
with self.assertRaises(ValueError):
s.parse_status(s.varint(1) + b"xx")
with self.assertRaises(ValueError):
s.parse_status(packet({"max": 20, "online": 0})[:-3])
def test_varint_round_trip(self):
for n in (0, 1, 127, 128, 300, 6087, 2097151):
data = iter([s.varint(n)])
buf = next(data)
pos = [0]
def read(k, buf=buf, pos=pos):
out = buf[pos[0] : pos[0] + k]
pos[0] += k
return out
self.assertEqual(s.read_varint(read), n)
if __name__ == "__main__":
unittest.main()
+37
View File
@@ -0,0 +1,37 @@
#!/bin/bash
# Install or update the now-playing service (cinny #491) on LXC 151.
# Run with this repo's main branch pulled to /opt/matrix-config:
# bash /opt/matrix-config/now-playing/install.sh
# First install: put the Navidrome login in /etc/lotus-now-playing/env first
# (NAVIDROME_USER=lotus-nowplaying / NAVIDROME_PASSWORD=...), mode 640 root:lotus-now-playing.
# Optional /etc/lotus-now-playing/users.json maps Matrix IDs whose SSO link has no
# email to their SSO username, e.g. {"@bigmanjeff:matrix.lotusguild.org": "jeff"}.
# Neither file is kept in this (public) repo. Safe to re-run.
set -euo pipefail
REPO=/opt/matrix-config
CONF=/etc/lotus-now-playing
id lotus-now-playing >/dev/null 2>&1 ||
useradd --system --no-create-home --home-dir /var/lib/lotus-now-playing --shell /usr/sbin/nologin lotus-now-playing
install -d -m750 -o root -g lotus-now-playing "$CONF"
install -D -m644 "$REPO/now-playing/lotus_now_playing.py" /opt/lotus-now-playing/lotus_now_playing.py
if [ ! -f "$CONF/env" ]; then
echo "missing $CONF/env (NAVIDROME_USER / NAVIDROME_PASSWORD); create it and re-run" >&2
exit 1
fi
chown root:lotus-now-playing "$CONF/env"
chmod 640 "$CONF/env"
if [ -f "$CONF/users.json" ]; then
chown root:lotus-now-playing "$CONF/users.json"
chmod 640 "$CONF/users.json"
fi
install -m644 "$REPO/systemd/lotus-now-playing.service" /etc/systemd/system/lotus-now-playing.service
systemctl daemon-reload
systemctl enable lotus-now-playing >/dev/null
systemctl restart lotus-now-playing
sleep 2
systemctl --no-pager --lines=5 status lotus-now-playing
curl -fsS http://127.0.0.1:8072/healthz && echo " (healthy)"
+493
View File
@@ -0,0 +1,493 @@
#!/usr/bin/env python3
"""Now playing (Navidrome) -> Lotus Chat "Listening to" cards. [cinny #491]
Navidrome is the scrobbling master: every play, whether from Dropped Needle or
any Subsonic app, shows up in its `getNowPlaying` under the listener's own
username, which is their SSO (LLDAP) username. This service polls that list and
serves it to Lotus Chat for users who opted in, keyed by Matrix user ID.
- Who is asking: Lotus Chat sends a Matrix OpenID token (Authorization: Bearer).
It is checked against Synapse's /_matrix/federation/v1/openid/userinfo, so
only signed-in users of this homeserver can read anything.
- Matrix user -> SSO username: an override file first (accounts whose SSO link
carries no email), then the Synapse admin API (email <name>@<domain>, or a
legacy external ID that is the plain username).
- Opt-in: off by default, stored here in STATE_DIR/state.json.
- Cover art is proxied only for covers that an opted-in user is playing right
now, so the endpoint can't be used to browse the library.
Track titles, artists and tokens are never logged.
"""
import hashlib
import json
import logging
import os
import re
import secrets
import tempfile
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
log = logging.getLogger("lotus-now-playing")
PREFIX = "/now-playing/v1"
MAX_IDS = 50
SUBSONIC_CLIENT = "lotus-now-playing"
SUBSONIC_VERSION = "1.16.1"
# Navidrome drops a session a while after the last report, so an entry older
# than this (no position updates, not paused) is treated as stale.
STALE_MINUTES = 30
COVER_SIZES = (64, 128, 256, 512)
UUID_RE = re.compile(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}")
# --- pure helpers (unit tested) ----------------------------------------------
def parse_now_playing(payload, ignore_users=()):
"""getNowPlaying JSON -> {username: entry}. Keeps the most recent session per user."""
resp = (payload or {}).get("subsonic-response") or {}
if resp.get("status") != "ok":
raise ValueError((resp.get("error") or {}).get("message") or "getNowPlaying failed")
entries = (resp.get("nowPlaying") or {}).get("entry") or []
if isinstance(entries, dict):
entries = [entries]
out = {}
for e in entries:
user = (e.get("username") or "").strip()
if not user or user in ignore_users:
continue
state = (e.get("state") or "playing").lower()
if state not in ("playing", "paused"):
continue
minutes = int(e.get("minutesAgo") or 0)
if state == "playing" and minutes > STALE_MINUTES and not e.get("positionMs"):
continue
entry = {
"title": e.get("title") or "",
"artist": e.get("displayArtist") or e.get("artist") or "",
"album": e.get("album") or "",
"durationS": int(e.get("duration") or 0),
"positionMs": int(e.get("positionMs") or 0),
"state": state,
"player": e.get("playerName") or "",
"coverArt": e.get("coverArt") or "",
"minutesAgo": minutes,
}
if not entry["title"]:
continue
prev = out.get(user)
if prev is None or minutes < prev["minutesAgo"]:
out[user] = entry
return out
def username_from_admin_user(user, email_domain):
"""Synapse admin v2 user JSON -> SSO username, or None."""
for ext in user.get("external_ids") or []:
provider = ext.get("auth_provider") or ""
ext_id = ext.get("external_id") or ""
# One legacy link stores the plain username instead of Authelia's opaque ID.
if (provider.startswith("http") and re.fullmatch(r"[a-z0-9._-]+", ext_id)
and not UUID_RE.fullmatch(ext_id)):
return ext_id
suffix = "@" + email_domain.lower()
for tp in user.get("threepids") or []:
addr = (tp.get("address") or "").strip().lower()
if tp.get("medium") == "email" and addr.endswith(suffix):
local = addr[: -len(suffix)]
if re.fullmatch(r"[a-z0-9._-]+", local):
return local
return None
def public_entry(entry, now_ms):
"""What Lotus Chat gets for one listener."""
out = {k: entry[k] for k in ("title", "artist", "album", "durationS", "positionMs", "state", "player")}
out["updatedAt"] = now_ms
out["cover"] = f"{PREFIX}/cover/{urllib.parse.quote(entry['coverArt'], safe='')}" if entry.get("coverArt") else None
return out
def parse_ids(raw, server_name):
ids = []
for part in (raw or "").split(","):
part = part.strip()
if part.startswith("@") and part.endswith(":" + server_name) and part not in ids:
ids.append(part)
return ids[:MAX_IDS]
# --- stateful pieces ---------------------------------------------------------
class OptIns:
"""{mxid: True} persisted atomically."""
def __init__(self, path):
self.path = path
self.lock = threading.Lock()
self.data = {}
try:
with open(path) as f:
loaded = json.load(f)
self.data = {k: True for k, v in loaded.items() if v is True}
except FileNotFoundError:
pass
except (OSError, ValueError) as exc:
log.error("could not read %s: %s", path, exc)
def get(self, mxid):
with self.lock:
return self.data.get(mxid, False)
def all(self):
with self.lock:
return list(self.data)
def set(self, mxid, share):
with self.lock:
if share:
self.data[mxid] = True
else:
self.data.pop(mxid, None)
snapshot = dict(self.data)
d = os.path.dirname(self.path) or "."
fd, tmp = tempfile.mkstemp(dir=d, prefix=".state-")
try:
with os.fdopen(fd, "w") as f:
json.dump(snapshot, f, indent=1, sort_keys=True)
os.replace(tmp, self.path)
except BaseException:
try:
os.unlink(tmp)
except OSError:
pass
raise
class TTLCache:
def __init__(self, max_items=2000):
self.lock = threading.Lock()
self.items = {}
self.max_items = max_items
def get(self, key, now=None):
now = time.monotonic() if now is None else now
with self.lock:
hit = self.items.get(key)
if hit and hit[1] > now:
return hit[0], True
self.items.pop(key, None)
return None, False
def put(self, key, value, ttl, now=None):
now = time.monotonic() if now is None else now
with self.lock:
if len(self.items) >= self.max_items:
for k in [k for k, (_, exp) in self.items.items() if exp <= now] or list(self.items)[:100]:
self.items.pop(k, None)
self.items[key] = (value, now + ttl)
class Service:
def __init__(self, cfg, http_get=None):
self.cfg = cfg
self.http_get = http_get or _http_get
self.optins = OptIns(os.path.join(cfg["state_dir"], "state.json"))
self.tokens = TTLCache()
self.users = TTLCache()
self.lock = threading.Lock()
self.playing = {}
self.overrides = {}
self.overrides_mtime = None
self.load_overrides()
# Navidrome
def subsonic_url(self, method, **params):
salt = secrets.token_hex(8)
token = hashlib.md5((self.cfg["nd_password"] + salt).encode()).hexdigest() # noqa: S324 (Subsonic protocol)
q = {"u": self.cfg["nd_user"], "t": token, "s": salt, "v": SUBSONIC_VERSION,
"c": SUBSONIC_CLIENT, "f": "json", **params}
return f"{self.cfg['nd_url']}/rest/{method}?{urllib.parse.urlencode(q)}"
def poll_once(self):
status, body, _ = self.http_get(self.subsonic_url("getNowPlaying"), {}, 10)
if status != 200:
raise ValueError(f"getNowPlaying HTTP {status}")
parsed = parse_now_playing(json.loads(body), self.cfg["ignore_users"])
with self.lock:
self.playing = parsed
return len(parsed)
def poll_forever(self):
failures = 0
while True:
try:
n = self.poll_once()
if failures:
log.info("navidrome reachable again (%d sessions)", n)
failures = 0
except Exception as exc: # keep polling whatever happens
failures += 1
if failures in (1, 10) or failures % 100 == 0:
log.warning("poll failed (%d in a row): %s", failures, exc)
if failures >= 3:
with self.lock:
self.playing = {}
time.sleep(self.cfg["poll_s"])
# identity
def verify_token(self, token):
if not token or len(token) > 512:
return None
cached, hit = self.tokens.get(token)
if hit:
return cached
url = (f"{self.cfg['synapse']}/_matrix/federation/v1/openid/userinfo?"
+ urllib.parse.urlencode({"access_token": token}))
try:
status, body, _ = self.http_get(url, {}, 5)
except Exception as exc:
log.warning("openid check failed: %s", exc)
return None
mxid = None
if status == 200:
sub = (json.loads(body) or {}).get("sub") or ""
if sub.startswith("@") and sub.endswith(":" + self.cfg["server_name"]):
mxid = sub
# Tokens live about an hour; a rejected one is remembered briefly.
self.tokens.put(token, mxid, 600 if mxid else 30)
return mxid
def load_overrides(self):
path = self.cfg["users_file"]
try:
mtime = os.path.getmtime(path)
except OSError:
self.overrides = {}
return
if mtime == self.overrides_mtime:
return
try:
with open(path) as f:
data = json.load(f)
self.overrides = {k: v for k, v in data.items() if isinstance(v, str) and v}
self.overrides_mtime = mtime
except (OSError, ValueError) as exc:
log.error("could not read %s: %s", path, exc)
def resolve(self, mxid):
self.load_overrides()
if mxid in self.overrides:
return self.overrides[mxid]
cached, hit = self.users.get(mxid)
if hit:
return cached
name = None
url = f"{self.cfg['synapse']}/_synapse/admin/v2/users/{urllib.parse.quote(mxid)}"
try:
status, body, _ = self.http_get(url, {"Authorization": f"Bearer {self.cfg['matrix_token']}"}, 5)
if status == 200:
name = username_from_admin_user(json.loads(body), self.cfg["email_domain"])
elif status != 404:
return None # don't cache transient failures
except Exception as exc:
log.warning("user lookup failed: %s", exc)
return None
self.users.put(mxid, name, 3600 if name else 300)
return name
# views
def entry_for(self, mxid):
if not self.optins.get(mxid):
return None
name = self.resolve(mxid)
if not name:
return None
with self.lock:
return self.playing.get(name)
def users_view(self, ids):
now_ms = int(time.time() * 1000)
out = {}
for mxid in ids:
entry = self.entry_for(mxid)
if entry:
out[mxid] = public_entry(entry, now_ms)
return out
def me_view(self, mxid):
name = self.resolve(mxid)
share = self.optins.get(mxid)
entry = None
if name:
with self.lock:
entry = self.playing.get(name)
return {
"share": share,
"linked": bool(name),
"playing": public_entry(entry, int(time.time() * 1000)) if entry else None,
}
def cover_allowed(self, cover_id):
with self.lock:
playing = dict(self.playing)
wanted = {name for name, e in playing.items() if e.get("coverArt") == cover_id}
if not wanted:
return False
return any(self.resolve(m) in wanted for m in self.optins.all())
# --- HTTP --------------------------------------------------------------------
def _http_get(url, headers, timeout):
req = urllib.request.Request(url, headers=headers)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.status, resp.read(), resp.headers.get("Content-Type", "")
except urllib.error.HTTPError as exc:
return exc.code, exc.read(), exc.headers.get("Content-Type", "")
class Handler(BaseHTTPRequestHandler):
server_version = "lotus-now-playing"
service: Service = None # set in main()
def log_message(self, fmt, *args): # no request logs: paths carry user IDs
pass
def _cors(self):
self.send_header("Access-Control-Allow-Origin", "*")
self.send_header("Access-Control-Allow-Headers", "Authorization, Content-Type")
self.send_header("Access-Control-Allow-Methods", "GET, PUT, OPTIONS")
self.send_header("Access-Control-Max-Age", "600")
def _json(self, status, obj):
body = json.dumps(obj).encode()
self.send_response(status)
self._cors()
self.send_header("Content-Type", "application/json")
self.send_header("Cache-Control", "no-store")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def _caller(self):
auth = self.headers.get("Authorization") or ""
token = auth[7:].strip() if auth.lower().startswith("bearer ") else ""
return self.service.verify_token(token)
def do_OPTIONS(self):
self.send_response(204)
self._cors()
self.end_headers()
def do_GET(self):
url = urllib.parse.urlsplit(self.path)
if url.path == "/healthz":
return self._json(200, {"ok": True})
if url.path.startswith(PREFIX + "/cover/"):
return self._cover(urllib.parse.unquote(url.path[len(PREFIX) + 7:]), url.query)
mxid = self._caller()
if not mxid:
return self._json(401, {"errcode": "M_UNKNOWN_TOKEN", "error": "Matrix OpenID token required"})
if url.path == PREFIX + "/me":
return self._json(200, self.service.me_view(mxid))
if url.path == PREFIX + "/users":
raw = urllib.parse.parse_qs(url.query).get("ids", [""])[0]
ids = parse_ids(raw, self.service.cfg["server_name"])
return self._json(200, {"users": self.service.users_view(ids)})
return self._json(404, {"errcode": "M_NOT_FOUND"})
def do_PUT(self):
url = urllib.parse.urlsplit(self.path)
if url.path != PREFIX + "/me":
return self._json(404, {"errcode": "M_NOT_FOUND"})
mxid = self._caller()
if not mxid:
return self._json(401, {"errcode": "M_UNKNOWN_TOKEN", "error": "Matrix OpenID token required"})
try:
length = int(self.headers.get("Content-Length") or 0)
body = json.loads(self.rfile.read(min(length, 4096)) or b"{}")
share = body["share"]
if not isinstance(share, bool):
raise ValueError
except (ValueError, KeyError, TypeError):
return self._json(400, {"errcode": "M_BAD_JSON", "error": "expected {\"share\": true|false}"})
try:
self.service.optins.set(mxid, share)
except OSError as exc:
log.error("could not save opt-in: %s", exc)
return self._json(500, {"errcode": "M_UNKNOWN", "error": "could not save"})
log.info("sharing %s (now %d users)", "on" if share else "off", len(self.service.optins.all()))
return self._json(200, self.service.me_view(mxid))
def _cover(self, cover_id, query):
if not cover_id or len(cover_id) > 128 or not self.service.cover_allowed(cover_id):
return self._json(404, {"errcode": "M_NOT_FOUND"})
try:
size = int(urllib.parse.parse_qs(query).get("size", ["256"])[0])
except ValueError:
size = 256
size = min(COVER_SIZES, key=lambda s: abs(s - size))
try:
status, body, ctype = self.service.http_get(
self.service.subsonic_url("getCoverArt", id=cover_id, size=size), {}, 10)
except Exception:
status, body, ctype = 502, b"", ""
if status != 200 or not ctype.startswith("image/"):
return self._json(502, {"errcode": "M_UNKNOWN", "error": "cover unavailable"})
self.send_response(200)
self._cors()
self.send_header("Content-Type", ctype)
self.send_header("Cache-Control", "public, max-age=300")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def load_config(env=os.environ):
def need(name):
value = env.get(name, "")
if not value:
raise SystemExit(f"missing {name}")
return value
return {
"bind": env.get("BIND_HOST", "0.0.0.0"),
"port": int(env.get("BIND_PORT", "8072")),
"server_name": env.get("SERVER_NAME", "matrix.lotusguild.org"),
"email_domain": env.get("EMAIL_DOMAIN", "lotusguild.org"),
"synapse": env.get("SYNAPSE_API", "http://127.0.0.1:8008").rstrip("/"),
"matrix_token": need("MATRIX_TOKEN"),
"nd_url": need("NAVIDROME_URL").rstrip("/"),
"nd_user": need("NAVIDROME_USER"),
"nd_password": need("NAVIDROME_PASSWORD"),
"ignore_users": tuple(u for u in env.get("IGNORE_USERS", "droppedneedle").split(",") if u),
"state_dir": env.get("STATE_DIR", "/var/lib/lotus-now-playing"),
"users_file": env.get("USERS_FILE", "/etc/lotus-now-playing/users.json"),
"poll_s": max(5, int(env.get("POLL_SECONDS", "10"))),
}
def main():
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
cfg = load_config()
service = Service(cfg)
try:
log.info("navidrome ok, %d sessions", service.poll_once())
except Exception as exc:
log.warning("first poll failed: %s", exc)
threading.Thread(target=service.poll_forever, daemon=True).start()
Handler.service = service
server = ThreadingHTTPServer((cfg["bind"], cfg["port"]), Handler)
server.daemon_threads = True
log.info("listening on %s:%d (%d users sharing)", cfg["bind"], cfg["port"], len(service.optins.all()))
server.serve_forever()
if __name__ == "__main__":
main()
+173
View File
@@ -0,0 +1,173 @@
"""Tests for lotus_now_playing.py. Run: python3 -m unittest now-playing/test_lotus_now_playing.py"""
import json
import os
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import lotus_now_playing as np # noqa: E402
def nd(entries, status="ok"):
return {"subsonic-response": {"status": status, "nowPlaying": {"entry": entries}}}
def entry(user, title="Song", minutes=0, state="playing", cover="al-1", **kw):
e = {"username": user, "title": title, "artist": "Band", "album": "LP", "duration": 200,
"minutesAgo": minutes, "state": state, "coverArt": cover, "playerName": "DN", "positionMs": 1000}
e.update(kw)
return e
class ParseTests(unittest.TestCase):
def test_basic_and_ignored_users(self):
out = np.parse_now_playing(nd([entry("jared"), entry("droppedneedle")]), ("droppedneedle",))
self.assertEqual(list(out), ["jared"])
self.assertEqual(out["jared"]["artist"], "Band")
self.assertEqual(out["jared"]["durationS"], 200)
def test_prefers_display_artist(self):
out = np.parse_now_playing(nd([entry("a", displayArtist="Band feat. X")]))
self.assertEqual(out["a"]["artist"], "Band feat. X")
def test_newest_session_wins(self):
out = np.parse_now_playing(nd([entry("a", "Old", minutes=5), entry("a", "New", minutes=0)]))
self.assertEqual(out["a"]["title"], "New")
def test_drops_stopped_untitled_and_stale(self):
out = np.parse_now_playing(nd([
entry("a", state="stopped"),
entry("b", title=""),
entry("c", minutes=90, positionMs=0),
entry("d", state="paused"),
]))
self.assertEqual(list(out), ["d"])
def test_single_entry_object_and_empty(self):
self.assertEqual(list(np.parse_now_playing(nd(entry("a")))), ["a"])
self.assertEqual(np.parse_now_playing({"subsonic-response": {"status": "ok"}}), {})
def test_error_raises(self):
with self.assertRaises(ValueError):
np.parse_now_playing({"subsonic-response": {"status": "failed", "error": {"message": "nope"}}})
class MappingTests(unittest.TestCase):
def test_email_on_domain(self):
user = {"threepids": [{"medium": "email", "address": "Tyler@LotusGuild.org"}]}
self.assertEqual(np.username_from_admin_user(user, "lotusguild.org"), "tyler")
def test_other_domain_ignored(self):
user = {"threepids": [{"medium": "email", "address": "x@gmail.com"}]}
self.assertIsNone(np.username_from_admin_user(user, "lotusguild.org"))
def test_legacy_external_id(self):
user = {"external_ids": [{"auth_provider": "https://auth.lotusguild.org", "external_id": "cole"}]}
self.assertEqual(np.username_from_admin_user(user, "lotusguild.org"), "cole")
def test_opaque_uuid_is_not_a_username(self):
user = {"external_ids": [
{"auth_provider": "oidc-authelia", "external_id": "b546660f-371c-45de-812c-bbae0e615148"},
{"auth_provider": "https://auth.lotusguild.org", "external_id": "b546660f-371c-45de-812c-bbae0e615148"},
]}
self.assertIsNone(np.username_from_admin_user(user, "lotusguild.org"))
def test_parse_ids(self):
ids = np.parse_ids("@a:hs, @b:other,@a:hs,junk,@c:hs", "hs")
self.assertEqual(ids, ["@a:hs", "@c:hs"])
self.assertEqual(len(np.parse_ids(",".join(f"@u{i}:hs" for i in range(80)), "hs")), np.MAX_IDS)
class FakeHttp:
def __init__(self):
self.calls = []
self.now_playing = nd([entry("jared", cover="al-9"), entry("jeff", cover="al-7")])
self.admin = {
"@jared:hs": {"threepids": [{"medium": "email", "address": "jared@lotusguild.org"}]},
"@nomail:hs": {"threepids": []},
}
self.openid = {"good": "@jared:hs", "foreign": "@x:elsewhere"}
def __call__(self, url, headers, timeout):
self.calls.append(url)
if "/rest/getNowPlaying" in url:
return 200, json.dumps(self.now_playing).encode(), "application/json"
if "/openid/userinfo" in url:
tok = url.split("access_token=")[1]
if tok in self.openid:
return 200, json.dumps({"sub": self.openid[tok]}).encode(), "application/json"
return 401, b"{}", "application/json"
if "/_synapse/admin/v2/users/" in url:
mxid = url.rsplit("/", 1)[1].replace("%40", "@").replace("%3A", ":")
if mxid in self.admin:
return 200, json.dumps(self.admin[mxid]).encode(), "application/json"
return 404, b"{}", "application/json"
raise AssertionError(url)
class ServiceTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
users = os.path.join(self.tmp.name, "users.json")
with open(users, "w") as f:
json.dump({"@bigmanjeff:hs": "jeff"}, f)
self.http = FakeHttp()
self.svc = np.Service({
"server_name": "hs", "email_domain": "lotusguild.org", "synapse": "http://syn",
"matrix_token": "admin", "nd_url": "http://nd", "nd_user": "svc", "nd_password": "pw",
"ignore_users": ("droppedneedle",), "state_dir": self.tmp.name, "users_file": users, "poll_s": 10,
}, http_get=self.http)
self.svc.poll_once()
def tearDown(self):
self.tmp.cleanup()
def test_token_check_and_cache(self):
self.assertEqual(self.svc.verify_token("good"), "@jared:hs")
self.assertEqual(self.svc.verify_token("good"), "@jared:hs")
self.assertEqual(sum("/openid/" in c for c in self.http.calls), 1)
self.assertIsNone(self.svc.verify_token("bad"))
self.assertIsNone(self.svc.verify_token("foreign"))
self.assertIsNone(self.svc.verify_token(""))
def test_opt_in_required(self):
self.assertEqual(self.svc.users_view(["@jared:hs", "@bigmanjeff:hs"]), {})
self.svc.optins.set("@jared:hs", True)
view = self.svc.users_view(["@jared:hs", "@bigmanjeff:hs"])
self.assertEqual(list(view), ["@jared:hs"])
self.assertEqual(view["@jared:hs"]["cover"], "/now-playing/v1/cover/al-9")
def test_override_mapping(self):
self.svc.optins.set("@bigmanjeff:hs", True)
self.assertIn("@bigmanjeff:hs", self.svc.users_view(["@bigmanjeff:hs"]))
def test_opt_in_persists_and_opt_out(self):
self.svc.optins.set("@jared:hs", True)
reloaded = np.OptIns(os.path.join(self.tmp.name, "state.json"))
self.assertTrue(reloaded.get("@jared:hs"))
self.svc.optins.set("@jared:hs", False)
self.assertEqual(self.svc.users_view(["@jared:hs"]), {})
def test_me_view(self):
me = self.svc.me_view("@jared:hs")
self.assertEqual((me["share"], me["linked"]), (False, True))
self.assertEqual(me["playing"]["title"], "Song")
self.assertEqual(self.svc.me_view("@nomail:hs"), {"share": False, "linked": False, "playing": None})
def test_cover_allow_list(self):
self.assertFalse(self.svc.cover_allowed("al-9")) # nobody opted in yet
self.svc.optins.set("@jared:hs", True)
self.assertTrue(self.svc.cover_allowed("al-9"))
self.assertFalse(self.svc.cover_allowed("al-7")) # jeff hasn't opted in
self.assertFalse(self.svc.cover_allowed("al-unknown"))
def test_subsonic_url_never_contains_password(self):
url = self.svc.subsonic_url("getNowPlaying")
self.assertNotIn("pw", url.split("?")[1].split("&t=")[0])
self.assertIn("u=svc", url)
self.assertNotIn("p=", url)
if __name__ == "__main__":
unittest.main()
+31
View File
@@ -0,0 +1,31 @@
[Unit]
Description=Discord presence to Matrix ghosts (Lotus)
After=network-online.target matrix-synapse.service mautrix-discord.service
Wants=network-online.target
[Service]
Type=simple
# The bridge's user: it reads the bot token from the bridge's own config + database.
User=mautrix-discord
Group=mautrix-discord
EnvironmentFile=/etc/lotus-presence/env
Environment=BRIDGE_CONFIG=/opt/mautrix-discord/config.yaml
Environment=SERVER_NAME=matrix.lotusguild.org
Environment=DISCORD_GUILD_ID=1565159371355197570
ExecStart=/usr/bin/python3 /opt/lotus-presence/lotus_presence.py
Restart=always
RestartSec=30
StandardOutput=journal
StandardError=journal
SyslogIdentifier=lotus-presence
MemoryMax=128M
CPUQuota=20%
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
+21
View File
@@ -0,0 +1,21 @@
[Unit]
Description=Lotus Minecraft player count as the bridge bot's Matrix status
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/python3 -I /opt/lotus-mc-status/lotus_mc_status.py
Environment=LOTUS_MC_HOST=10.10.10.78
Environment=LOTUS_MC_PORT=25565
LoadCredential=bridge-config:/var/lib/pelican/volumes/5476addf-384e-4f49-baa6-303e05719e9f/config/minecraftmatrixbridge.toml
DynamicUser=yes
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target
+38
View File
@@ -0,0 +1,38 @@
[Unit]
Description=Now playing (Navidrome) for Lotus Chat "Listening to" cards
After=network-online.target matrix-synapse.service
Wants=network-online.target
[Service]
Type=simple
User=lotus-now-playing
Group=lotus-now-playing
# MATRIX_TOKEN (Synapse admin API, user lookups only), shared with voice-limit-guard.
EnvironmentFile=/etc/matrix-deploy.env
# NAVIDROME_USER / NAVIDROME_PASSWORD for the read-only lotus-nowplaying account.
EnvironmentFile=/etc/lotus-now-playing/env
Environment=NAVIDROME_URL=http://10.10.10.28:4533
Environment=SERVER_NAME=matrix.lotusguild.org
Environment=BIND_HOST=0.0.0.0
Environment=BIND_PORT=8072
Environment=STATE_DIR=/var/lib/lotus-now-playing
Environment=USERS_FILE=/etc/lotus-now-playing/users.json
StateDirectory=lotus-now-playing
StateDirectoryMode=0750
ExecStart=/usr/bin/python3 /opt/lotus-now-playing/lotus_now_playing.py
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=lotus-now-playing
MemoryMax=96M
CPUQuota=20%
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
+32
View File
@@ -0,0 +1,32 @@
[Unit]
Description=Lotus voice bridge (Discord voice <-> Matrix calls, nether-voicebridge)
After=network-online.target matrix-synapse.service
Wants=network-online.target
[Service]
Type=simple
User=lotus-voice
Group=lotus-voice
Environment=CONFIG_PATH=/etc/lotus-voice/config.toml
Environment=RUST_LOG=info
ExecStart=/opt/lotus-voice/lotus-voice-bridge
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=lotus-voice-bridge
# Resource limits (Opus encode/decode + LiveKit + Discord voice per call)
MemoryMax=1G
CPUQuota=150%
# Hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadWritePaths=/var/lib/lotus-voice /etc/lotus-voice
[Install]
WantedBy=multi-user.target
+30
View File
@@ -0,0 +1,30 @@
[Unit]
Description=mautrix-discord Matrix-Discord bridge
After=network-online.target matrix-synapse.service
Wants=network-online.target
[Service]
Type=simple
User=mautrix-discord
Group=mautrix-discord
WorkingDirectory=/opt/mautrix-discord
ExecStart=/opt/mautrix-discord/mautrix-discord -c /opt/mautrix-discord/config.yaml -r /opt/mautrix-discord/registration.yaml
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=mautrix-discord
# Resource limits
MemoryMax=512M
CPUQuota=80%
# Hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
ReadWritePaths=/opt/mautrix-discord /var/log/mautrix-discord
[Install]
WantedBy=multi-user.target
+17
View File
@@ -0,0 +1,17 @@
[Unit]
Description=Compress Synapse state groups (synapse_auto_compressor)
Documentation=https://github.com/matrix-org/rust-synapse-compress-state
After=postgresql.service
[Service]
Type=oneshot
User=postgres
# 100 chunks of 500 state groups per night. Progress is stored in the
# state_compressor_* tables, so each run continues where the last one stopped.
# Every state group is rewritten in its own transaction, after the compressed
# state is checked against the original; a chunk that would not shrink is skipped.
ExecStart=/usr/local/bin/synapse_auto_compressor -p postgresql:///synapse?host=/var/run/postgresql&user=postgres -c 500 -n 100
Environment=RUST_LOG=info
Nice=10
IOSchedulingClass=idle
TimeoutStartSec=2h
+10
View File
@@ -0,0 +1,10 @@
[Unit]
Description=Nightly Synapse state compression (after the 03:00 vzdump backup)
[Timer]
OnCalendar=*-*-* 04:30
RandomizedDelaySec=10min
Persistent=true
[Install]
WantedBy=timers.target
+2
View File
@@ -19,6 +19,8 @@ Environment=GUARD_BIND_PORT=8070
Environment=GUARD_UPSTREAM=http://127.0.0.1:8071
Environment=LIVEKIT_API=http://127.0.0.1:7880
Environment=SYNAPSE_API=http://127.0.0.1:8008
# The voice bridge's bot listens in calls but is not a person (Discord speakers still count).
Environment=VOICE_LIMIT_EXEMPT_USERS=@lotus-voice_bot:matrix\\.lotusguild\\.org
[Install]
WantedBy=multi-user.target
+41
View File
@@ -0,0 +1,41 @@
# Lotus Guild voice bridge (nether-voicebridge, Lotus-patched build) — LXC 151.
# Rendered by voice-bridge/install.sh into /etc/lotus-voice/config.toml (0600).
# @AS_TOKEN@ and @PUPPET_TOKENS@ are filled in on the server; never commit them.
data_dir = "/var/lib/lotus-voice"
[server.lotus]
homeserver_url = "http://127.0.0.1:8008"
# MXIDs and .well-known discovery (https://matrix.lotusguild.org) use this domain.
matrix_domain = "matrix.lotusguild.org"
# Bot = @lotus-voice_bot, Discord speakers = @lotus-voice_<discord id>.
sender_localpart = "lotus-voice"
appservice_id = "lotus-voice"
appservice_token = "@AS_TOKEN@"
[puppets]
# Lotus Voice 1–3 (Discord apps 1565219639640858746 / 1565220427297722378 /
# 1565220532830470207). One per Matrix person speaking at the same time.
tokens = [@PUPPET_TOKENS@]
nickname_prefix = "[Matrix] "
[[bridge]]
name = "discord-voice-bridge"
server = "lotus"
# Matrix: "Discord Voice Bridge" voice room.
room_id = "!yS_ZIHL3sOKQJBt4NiIQyiHKt3cv6HrIDeq7OxDpSM0"
[bridge.discord]
# Lotus Guild server, voice channel "matrix-bridge".
guild_id = "1565159371355197570"
channel_id = "1565159372122882071"
[bridge.matrix]
# Shown as a call participant in encrypted calls (it receives the media keys).
bot_display_name = "Discord Voice Bridge"
bot_avatar_path = "/opt/lotus-voice/avatar.png"
# Lotus: leave Discord promptly after the last Matrix person leaves
# (upstream defaults 12 s + 90 s). 8 s still covers Element Call's 2-3 s membership flaps.
# The bridge logs a "NON-DEFAULT TIMING PROFILE" warning for this; it's expected.
[timing]
leave_grace_ms = 8000
anchor_idle_grace_ms = 10000
+87
View File
@@ -0,0 +1,87 @@
#!/bin/bash
# One-time install / upgrade of the Lotus voice bridge on LXC 151.
# Run on the Proxmox node hosting LXC 151 (and LXC 109 is not needed: the bridge
# keeps its state in sqlite under /var/lib/lotus-voice):
#
# bash install.sh <path to lotus-voice-bridge binary> <secrets file>
#
# The secrets file holds the three Lotus Voice bot tokens as BOT1=…, BOT2=…, BOT3=…
# (shell syntax); it is copied in, used to render the config, then deleted.
# Needs this repo's main pulled to /opt/matrix-config on LXC 151 (auto-deploy).
# First install restarts Synapse once (new appservice registration). Re-running
# with only a new binary just swaps it and restarts the bridge. Safe to re-run.
set -euo pipefail
BIN=${1:?usage: install.sh <binary> <secrets file>}
SECRETS=${2:?usage: install.sh <binary> <secrets file>}
CT=151
[ -x "$BIN" ] || { echo "binary not found/executable: $BIN" >&2; exit 1; }
echo "== [1/4] copy binary + secrets into LXC $CT"
pct push "$CT" "$BIN" /root/lotus-voice-bridge.new --perms 0755
pct push "$CT" "$SECRETS" /root/lotus-voice.secrets --perms 0600
echo "== [2/4] install files, user, config, registration"
pct exec "$CT" -- bash -s <<'IN'
set -euo pipefail
SRC=/opt/matrix-config
id lotus-voice >/dev/null 2>&1 || useradd --system --home /var/lib/lotus-voice --shell /usr/sbin/nologin lotus-voice
install -d -o lotus-voice -g lotus-voice -m 0750 /var/lib/lotus-voice /etc/lotus-voice
install -d -m 0755 /opt/lotus-voice
mv /root/lotus-voice-bridge.new /opt/lotus-voice/lotus-voice-bridge
install -m 0644 "$SRC/discord-bridge/bot-avatar.png" /opt/lotus-voice/avatar.png
REG=/etc/matrix-synapse/lotus-voice-registration.yaml
if [ ! -f "$REG" ]; then
AS=$(openssl rand -hex 32); HS=$(openssl rand -hex 32)
sed -e "s/@AS_TOKEN@/$AS/" -e "s/@HS_TOKEN@/$HS/" "$SRC/voice-bridge/registration.yaml.template" > "$REG"
chown root:matrix-synapse "$REG"; chmod 0640 "$REG"
echo "registration written"
fi
AS=$(awk '/^as_token:/ {gsub(/"/, "", $2); print $2; exit}' "$REG")
set -a; . /root/lotus-voice.secrets; set +a
rm -f /root/lotus-voice.secrets
for v in BOT1 BOT2 BOT3; do [ -n "${!v:-}" ] || { echo "secrets file is missing $v" >&2; exit 1; }; done
TOKENS="\"$BOT1\", \"$BOT2\", \"$BOT3\""
umask 077
sed -e "s/@AS_TOKEN@/$AS/" -e "s|@PUPPET_TOKENS@|$TOKENS|" "$SRC/voice-bridge/config.toml.template" > /etc/lotus-voice/config.toml
unset BOT1 BOT2 BOT3 TOKENS
chown lotus-voice:lotus-voice /etc/lotus-voice/config.toml
install -m 0644 "$SRC/systemd/lotus-voice-bridge.service" /etc/systemd/system/lotus-voice-bridge.service
systemctl daemon-reload
echo "files ok"
IN
echo "== [3/4] register with Synapse (restart only if newly added)"
pct exec "$CT" -- bash -s <<'RS'
set -euo pipefail
HSY=/etc/matrix-synapse/homeserver.yaml
if grep -q lotus-voice-registration.yaml "$HSY"; then
echo "already registered — no Synapse restart"
else
cp "$HSY" "$HSY.bak-$(date +%Y%m%d%H%M%S)-voicebridge"
sed -i '/^ - \/etc\/matrix-synapse\/discord-registration.yaml/a \ - /etc/matrix-synapse/lotus-voice-registration.yaml' "$HSY"
grep -A4 '^app_service_config_files' "$HSY"
systemctl restart matrix-synapse
for i in $(seq 1 60); do curl -fsS -o /dev/null http://127.0.0.1:8008/health && break; sleep 2; done
curl -fsS http://127.0.0.1:8008/health && echo " (synapse healthy)"
fi
# Create the bot account now so it can be invited before the bridge first starts.
AS=$(awk '/^as_token:/ {gsub(/"/, "", $2); print $2; exit}' /etc/matrix-synapse/lotus-voice-registration.yaml)
curl -sS -X POST -H "Authorization: Bearer $AS" -H 'Content-Type: application/json' \
-d '{"type":"m.login.application_service","username":"lotus-voice_bot","inhibit_login":true}' \
http://127.0.0.1:8008/_matrix/client/v3/register | grep -qE 'user_id|M_USER_IN_USE' && echo "bot account @lotus-voice_bot ready"
RS
echo "== [4/4] (re)start the bridge"
pct exec "$CT" -- bash -s <<'ST'
set -euo pipefail
systemctl enable lotus-voice-bridge >/dev/null 2>&1
systemctl restart lotus-voice-bridge
sleep 8
systemctl --no-pager --lines=0 status lotus-voice-bridge | head -5
journalctl -u lotus-voice-bridge --since "-15 sec" --no-pager -o cat | grep -E "puppet connected|initial call snapshot|LiveKit focus|bridge failed|ERROR" | sed -E 's/\x1b\[[0-9;]*m//g' | cut -c1-160 | head -10
ST
echo
echo "Done. In Lotus, invite @lotus-voice_bot:matrix.lotusguild.org to the Discord Voice Bridge room."
+15
View File
@@ -0,0 +1,15 @@
# Synapse appservice registration for the voice bridge. Rendered by install.sh to
# /etc/matrix-synapse/lotus-voice-registration.yaml with fresh random tokens.
# The bridge talks to Synapse only as a client (as_token + ?user_id=), so no URL.
id: lotus-voice
url: null
as_token: "@AS_TOKEN@"
hs_token: "@HS_TOKEN@"
sender_localpart: lotus-voice_as
namespaces:
users:
- exclusive: true
regex: "@lotus-voice_.*:matrix\\.lotusguild\\.org"
aliases: []
rooms: []
rate_limited: false