Compare commits

..
24 Commits
Author SHA1 Message Date
jared 8bf2e6d828 Merge pull request 'fix(matrixbot): bring live ATM10 edits into the repo' (#70) from fix/lotusbot-atm10-live-edits into main
Lint / JS (eslint) (push) Successful in 12s
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 13s
Lint / Secret scan (gitleaks) (push) Successful in 11s
Lint / Python deps (pip-audit) (push) Successful in 41s
Lint / Shell (shellcheck) (push) Successful in 11s
Lint / No Hookshot webhook IDs in README (push) Successful in 18s
2026-11-19 20:57:48 -05:00
jaredandClaude Opus 5.5 71054d2778 fix(matrixbot): bring live ATM10 edits into the repo
Lint / Shell (shellcheck) (pull_request) Successful in 8s
Lint / JS (eslint) (pull_request) Successful in 8s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 7s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Secret scan (gitleaks) (pull_request) Successful in 13s
Lint / JS (eslint) (push) Successful in 8s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 10s
Lint / Python (ruff) (push) Successful in 10s
Lint / Secret scan (gitleaks) (push) Successful in 12s
Lint / Python deps (pip-audit) (push) Successful in 38s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m2s
Lint / Shell (shellcheck) (push) Successful in 11s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
These changes were made directly on CT151 (2026-10-30 to 11-01) and never
committed, so the auto-deploy triggered by b18aae8 on 2026-11-18 overwrote
them. Restored live on 2026-11-19; this commits them so the next deploy
keeps them.

- !minecraft: 60 s per-user cooldown, audit log line, clear message when the
  Minecraft account doesn't exist
- !minecraft reply: Lotus Minecraft Server / All the Mods 10 8.6 (was the
  old Paper "Lotus SMP, fresh world, hard mode" text)
- !help: compact list with !help <command> for details, since the Discord
  bridge drops messages over 2000 characters

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KQ9u9cUsWuUQN791N3JzFG
2026-11-19 20:49:15 -05:00
jared 7c497d2a0a Merge pull request 'docs: large1 → compute-storage-02; no fixed node for HA guests' (#69) from docs/large1-leftovers into main
Lint / Python (ruff) (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 45s
Lint / JS (eslint) (push) Successful in 7s
Lint / No Hookshot webhook IDs in README (push) Successful in 19s
Lint / Shell (shellcheck) (push) Successful in 8s
2026-11-18 16:22:13 -05:00
jaredandClaude Opus 5.5 b18aae89d9 docs: remove large1 leftovers; no node names for HA guests
Lint / JS (eslint) (push) Successful in 8s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Python (ruff) (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 11s
Lint / Python deps (pip-audit) (push) Successful in 1m6s
Lint / Shell (shellcheck) (pull_request) Successful in 12s
Lint / JS (eslint) (pull_request) Successful in 7s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 9s
Lint / No secrets in webhook configs (pull_request) Successful in 5s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Secret scan (gitleaks) (pull_request) Successful in 8s
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python (ruff) (pull_request) Successful in 7s
Lint / Python deps (pip-audit) (pull_request) Successful in 37s
Lint / Shell (shellcheck) (push) Successful in 9s
large1 was renamed to compute-storage-02 on 2026-11-18. Updates text that still
named large1. HA guests no longer carry a fixed node (they move between nodes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 16:15:33 -05:00
jared 3a0a4e15a8 now-playing: Navidrome "Listening to" service for Lotus Chat (cinny #491) (#68)
Lint / JS (eslint) (push) Successful in 7s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 11s
Lint / Python deps (pip-audit) (push) Successful in 36s
Lint / Secret scan (gitleaks) (push) Successful in 5s
Lint / Shell (shellcheck) (push) Successful in 10s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
2026-11-16 23:51:07 -05:00
jared b3fec4af2a discord-bridge: check the gateway echo before retrying ambiguous webhook sends (#67)
Lint / JS (eslint) (push) Successful in 6s
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Python (ruff) (push) Successful in 4s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 36s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 4s
2026-11-15 21:46:31 -05:00
jared a57c91860b Merge pull request 'Docs: drop stale node names from the infrastructure tables' (#66) from rename/micro1-to-monitor-01 into main
Lint / JS (eslint) (push) Successful in 8s
Lint / No secrets in webhook configs (push) Successful in 9s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 1m28s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
2026-11-14 17:57:54 -05:00
jaredandClaude Opus 5.5 c8b7b2b049 Docs: drop stale node names from the infrastructure tables
Lint / JS (eslint) (push) Successful in 11s
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 12s
Lint / Python (ruff) (push) Successful in 23s
Lint / Python deps (pip-audit) (push) Successful in 1m34s
Lint / JS (eslint) (pull_request) Successful in 10s
Lint / Shell (shellcheck) (pull_request) Successful in 13s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 5s
Lint / No secrets in webhook configs (pull_request) Successful in 7s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Secret scan (gitleaks) (pull_request) Successful in 11s
Lint / Secret scan (gitleaks) (push) Successful in 4s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 19s
Lint / Python (ruff) (pull_request) Successful in 6s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m19s
These guests are HA and move between nodes; micro1 no longer exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:46:56 -05:00
jared c2ffb8907d discord-bridge: retry Matrix→Discord sends that failed before reaching Discord (#65)
Lint / JS (eslint) (push) Successful in 8s
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Python (ruff) (push) Successful in 4s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 34s
Lint / Python deps (pip-audit) (push) Successful in 49s
Lint / Shell (shellcheck) (push) Successful in 8s
Lint / No Hookshot webhook IDs in README (push) Successful in 4s
2026-11-14 14:31:23 -05:00
jared 16eca51e4c hookshot: remove the Lidarr webhook (#64)
Lint / JS (eslint) (push) Successful in 8s
Lint / No secrets in webhook configs (push) Successful in 4s
Lint / Python (ruff) (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / Python deps (pip-audit) (push) Successful in 39s
Lint / Shell (shellcheck) (push) Successful in 12s
Lint / No Hookshot webhook IDs in README (push) Successful in 5s
Co-authored-by: Jared Vititoe <jjvititoe1@gmail.com>
2026-11-10 13:04:34 -05:00
jared 3c28ec16dd Merge pull request 'docs: background web push is not planned (cinny #202)' (#63) from docs/no-web-push into main
Lint / No secrets in webhook configs (push) Successful in 18s
Lint / Python (ruff) (push) Successful in 11s
Lint / Secret scan (gitleaks) (push) Successful in 19s
Lint / Python deps (pip-audit) (push) Successful in 1m50s
Lint / Landing page is rendered (matrix (push) Successful in 9s
Lint / JS (eslint) (push) Successful in 16s
Lint / No Hookshot webhook IDs in README (push) Successful in 32s
Lint / Shell (shellcheck) (push) Successful in 18s
2026-11-09 19:20:54 -05:00
jaredandClaude Opus 5.5 3e1659224b docs: background web push is not planned (cinny #202)
Lint / JS (eslint) (push) Successful in 18s
Lint / No secrets in webhook configs (push) Successful in 11s
Lint / Python deps (pip-audit) (push) Successful in 59s
Lint / Python (ruff) (push) Successful in 25s
Lint / Shell (shellcheck) (pull_request) Successful in 28s
Lint / JS (eslint) (pull_request) Successful in 25s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 11s
Lint / No secrets in webhook configs (pull_request) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 9s
Lint / Landing page is rendered (matrix (pull_request) Successful in 16s
Lint / Secret scan (gitleaks) (push) Successful in 18s
Lint / Secret scan (gitleaks) (pull_request) Successful in 12s
Lint / No Hookshot webhook IDs in README (push) Successful in 17s
Lint / Python (ruff) (pull_request) Successful in 13s
Lint / Shell (shellcheck) (push) Successful in 20s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m25s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-09 19:12:48 -05:00
jared a73afea6b9 Merge pull request 'minecraft: show players online as the bridge bot's status' (#62) from feat/mc-player-status into main
Lint / JS (eslint) (push) Successful in 14s
Lint / No secrets in webhook configs (push) Successful in 9s
Lint / Landing page is rendered (matrix (push) Successful in 13s
Lint / Python (ruff) (push) Successful in 12s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m11s
Lint / Shell (shellcheck) (push) Successful in 15s
Lint / No Hookshot webhook IDs in README (push) Successful in 10s
2026-11-04 22:42:03 -05:00
jaredandClaude Opus 5.5 2d2ebf64a5 minecraft: show players online as the bridge bot's status
Lint / Shell (shellcheck) (pull_request) Successful in 26s
Lint / JS (eslint) (pull_request) Successful in 23s
Lint / No secrets in webhook configs (pull_request) Successful in 9s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 11s
Lint / Landing page is rendered (matrix (pull_request) Successful in 13s
Lint / Secret scan (gitleaks) (pull_request) Successful in 24s
Lint / JS (eslint) (push) Successful in 23s
Lint / No secrets in webhook configs (push) Successful in 6s
Lint / Landing page is rendered (matrix (push) Successful in 5s
Lint / Python (ruff) (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 56s
Lint / Python (ruff) (pull_request) Successful in 12s
Lint / Shell (shellcheck) (push) Successful in 27s
Lint / No Hookshot webhook IDs in README (push) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m43s
A small service on CT 154 pings the Lotus Minecraft server and sets the result
as the Matrix status of @lotus-mc-bridge, re-sent every 20 s because the mod
syncs with set_presence=offline and Synapse would otherwise drop it to offline.
Uses the mod's own token, so no new account or secret.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-04 22:37:55 -05:00
jared c793d4fb29 Merge pull request 'cinny: point the status banner at Kuma's public page' (#61) from fix/status-banner-public-page into main
Lint / JS (eslint) (push) Successful in 13s
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Landing page is rendered (matrix (push) Successful in 15s
Lint / Python (ruff) (push) Successful in 9s
Lint / Secret scan (gitleaks) (push) Successful in 17s
Lint / Python deps (pip-audit) (push) Successful in 1m44s
Lint / Shell (shellcheck) (push) Successful in 30s
Lint / No Hookshot webhook IDs in README (push) Successful in 9s
2026-11-04 22:21:53 -05:00
jaredandClaude Opus 5.5 5268156e48 cinny: point the status banner at Kuma's public page
Lint / JS (eslint) (pull_request) Successful in 30s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 15s
Lint / No secrets in webhook configs (pull_request) Successful in 15s
Lint / Landing page is rendered (matrix (pull_request) Successful in 5s
Lint / Secret scan (gitleaks) (pull_request) Successful in 9s
Lint / JS (eslint) (push) Successful in 11s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Python (ruff) (push) Successful in 6s
Lint / Shell (shellcheck) (pull_request) Successful in 22s
Lint / Python deps (pip-audit) (push) Successful in 1m34s
Lint / No secrets in webhook configs (push) Successful in 7s
Lint / Secret scan (gitleaks) (push) Successful in 7s
Lint / Python (ruff) (pull_request) Successful in 8s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m9s
Lint / Shell (shellcheck) (push) Successful in 13s
Lint / No Hookshot webhook IDs in README (push) Successful in 16s
The "matrix" status page was merged into "public" on 2026-10-11, so the banner
has read a 404 and stayed silent since. Kuma's Matrix group on "public" is now
split into Homeserver / Voice calls / Discord, and notices are scoped with
noticePrefix "Matrix:" (cinny #486, already live), so other services'
maintenance and incidents don't reach Lotus users.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-04 21:53:23 -05:00
jared dde36cdcd2 Merge pull request 'discord-bridge: Discord presence into Matrix ghosts' (#60) from feat/discord-presence into main
Lint / JS (eslint) (push) Successful in 27s
Lint / No secrets in webhook configs (push) Successful in 15s
Lint / Landing page is rendered (matrix (push) Successful in 17s
Lint / Python (ruff) (push) Successful in 21s
Lint / Secret scan (gitleaks) (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m15s
Lint / Shell (shellcheck) (push) Successful in 29s
Lint / No Hookshot webhook IDs in README (push) Successful in 17s
2026-11-04 19:41:07 -05:00
jaredandClaude Opus 5.5 f8f0ca4549 discord-bridge: Discord presence into Matrix ghosts
Lint / JS (eslint) (pull_request) Successful in 10s
Lint / Shell (shellcheck) (pull_request) Successful in 11s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 6s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Landing page is rendered (matrix (pull_request) Successful in 6s
Lint / Secret scan (gitleaks) (pull_request) Successful in 13s
Lint / JS (eslint) (push) Successful in 18s
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 17s
Lint / Python (ruff) (push) Successful in 16s
Lint / Secret scan (gitleaks) (push) Successful in 31s
Lint / Python (ruff) (pull_request) Successful in 7s
Lint / Shell (shellcheck) (push) Successful in 26s
Lint / No Hookshot webhook IDs in README (push) Successful in 21s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m54s
Lint / Python deps (pip-audit) (push) Successful in 1m37s
mautrix-discord doesn't bridge presence. lotus_presence.py watches the Discord
server (a second gateway session on the bridge's own bot, Guilds + Presence
intents only) and sets presence and a status message on each @discord_* ghost:
online / idle / DND / offline -> online / unavailable / msc3026 busy / offline;
custom status and the first activity become the status message.

Its own appservice registration is non-exclusive on the ghost namespace, so it
can update ghosts but not create them. Synapse drops an unrefreshed user to
offline after about a minute, so live states are re-sent every 20 s; if the
service dies, ghosts go offline rather than stale. The installer generates the
credentials on the server and doesn't restart Synapse. Statuses and activities
are never logged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
2026-11-04 19:36:38 -05:00
jared ed1300f146 Merge pull request 'feat(synapse): nightly state compression on CT 109' (#59) from feat/synapse-state-compressor into main
Lint / JS (eslint) (push) Successful in 8s
Lint / No secrets in webhook configs (push) Successful in 5s
Lint / Python (ruff) (push) Successful in 5s
Lint / Landing page is rendered (matrix (push) Successful in 6s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Shell (shellcheck) (push) Successful in 9s
Lint / No Hookshot webhook IDs in README (push) Successful in 6s
Lint / Python deps (pip-audit) (push) Successful in 39s
2026-11-03 19:10:33 -05:00
Lotus CIandClaude Opus 5.5 b2877323c3 feat(synapse): nightly state compression on CT 109
Lint / No secrets in webhook configs (push) Successful in 12s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 5s
Lint / Secret scan (gitleaks) (push) Successful in 6s
Lint / Shell (shellcheck) (pull_request) Successful in 8s
Lint / JS (eslint) (pull_request) Successful in 7s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 5s
Lint / No secrets in webhook configs (pull_request) Successful in 6s
Lint / Landing page is rendered (matrix (pull_request) Successful in 4s
Lint / Python deps (pip-audit) (push) Successful in 55s
Lint / Secret scan (gitleaks) (pull_request) Successful in 7s
Lint / JS (eslint) (push) Successful in 6s
Lint / No Hookshot webhook IDs in README (push) Successful in 12s
Lint / Shell (shellcheck) (push) Successful in 9s
Lint / Python (ruff) (pull_request) Successful in 5s
Lint / Python deps (pip-audit) (pull_request) Successful in 42s
The Synapse database grew from 2.2 GB (after the July cleanup) to
6.2 GB, almost all of it state_groups_state from large federated rooms
with one or two local members.

synapse_auto_compressor (matrix-org/rust-synapse-compress-state, built
from main) now runs nightly at 04:30, after the 03:00 vzdump backup.
It verifies compressed state against the original, writes each state
group in its own transaction, skips chunks that would not shrink, and
resumes from stored progress. The initial backlog took ~8 minutes with
no Synapse errors: state_groups_state 11.0 M -> 4.25 M rows (-61%).

Also documents the new "Synapse Database Growing Fast" Grafana alert
(> 500 MB in 7 days).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 19:08:01 -05:00
jared ded4a30530 Merge pull request 'docs: all 13 Hookshot webhooks rotated' (#58) from docs/rotated-remaining-webhooks into main
Lint / JS (eslint) (push) Successful in 13s
Lint / No secrets in webhook configs (push) Successful in 8s
Lint / Landing page is rendered (matrix (push) Successful in 7s
Lint / Python (ruff) (push) Successful in 10s
Lint / Secret scan (gitleaks) (push) Successful in 25s
Lint / Shell (shellcheck) (push) Successful in 14s
Lint / No Hookshot webhook IDs in README (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m10s
2026-11-03 17:03:09 -05:00
Lotus CIandClaude Opus 5.5 f3eb3ea406 docs: all 13 Hookshot webhooks rotated; old IDs in history are dead
Lint / JS (eslint) (push) Successful in 17s
Lint / No secrets in webhook configs (push) Successful in 14s
Lint / Python (ruff) (push) Successful in 22s
Lint / Landing page is rendered (matrix (push) Successful in 24s
Lint / Secret scan (gitleaks) (push) Successful in 12s
Lint / Shell (shellcheck) (pull_request) Successful in 15s
Lint / JS (eslint) (pull_request) Successful in 17s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 16s
Lint / Landing page is rendered (matrix (pull_request) Successful in 7s
Lint / No secrets in webhook configs (pull_request) Successful in 8s
Lint / Secret scan (gitleaks) (pull_request) Successful in 13s
Lint / Shell (shellcheck) (push) Successful in 17s
Lint / No Hookshot webhook IDs in README (push) Successful in 8s
Lint / Python deps (pip-audit) (push) Successful in 1m7s
Lint / Python (ruff) (pull_request) Successful in 15s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m4s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 16:59:31 -05:00
jared 7b01a9fcdc Merge pull request 'docs: call stack upgraded (lk-jwt-service 0.9.0, LiveKit 1.13.11)' (#57) from docs/call-stack-upgraded into main
Lint / JS (eslint) (push) Successful in 24s
Lint / No secrets in webhook configs (push) Successful in 17s
Lint / Landing page is rendered (matrix (push) Successful in 17s
Lint / Python (ruff) (push) Successful in 23s
Lint / Secret scan (gitleaks) (push) Successful in 20s
Lint / Python deps (pip-audit) (push) Successful in 1m11s
Lint / Shell (shellcheck) (push) Successful in 18s
Lint / No Hookshot webhook IDs in README (push) Successful in 17s
2026-11-03 16:42:04 -05:00
Lotus CIandClaude Opus 5.5 bc504a881d docs: call stack upgraded to lk-jwt-service 0.9.0 and LiveKit 1.13.11
Lint / JS (eslint) (push) Successful in 1m31s
Lint / No secrets in webhook configs (push) Successful in 22s
Lint / Landing page is rendered (matrix (push) Successful in 14s
Lint / Python (ruff) (push) Successful in 14s
Lint / Secret scan (gitleaks) (push) Successful in 28s
Lint / Shell (shellcheck) (pull_request) Successful in 22s
Lint / Python deps (pip-audit) (push) Successful in 1m7s
Lint / JS (eslint) (pull_request) Successful in 17s
Lint / No Hookshot webhook IDs in README (pull_request) Successful in 19s
Lint / No secrets in webhook configs (pull_request) Successful in 16s
Lint / Landing page is rendered (matrix (pull_request) Successful in 22s
Lint / Secret scan (gitleaks) (pull_request) Successful in 25s
Lint / Shell (shellcheck) (push) Successful in 1m31s
Lint / No Hookshot webhook IDs in README (push) Successful in 13s
Lint / Python (ruff) (pull_request) Successful in 15s
Lint / Python deps (pip-audit) (pull_request) Successful in 1m25s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017PCb59uY3UKqDqt7B6hzd5
2026-11-03 16:36:28 -05:00
25 changed files with 2508 additions and 94 deletions

No files matched your search

+2 -2
View File
@@ -85,10 +85,10 @@ jobs:
mv ruff /usr/local/bin/ruff
- name: Check syntax errors
run: ruff check matrixbot/ --select E9,F63,F7,F82 --output-format=github
run: ruff check matrixbot/ now-playing/ --select E9,F63,F7,F82 --output-format=github
- name: Run full lint
run: ruff check matrixbot/ --output-format=github
run: ruff check matrixbot/ now-playing/ --output-format=github
python-audit:
name: Python deps (pip-audit)
+124 -30
View File
@@ -29,6 +29,10 @@ matrix/
├── discord-bridge/
│ ├── config.yaml # mautrix-discord config (DB password filled in on the server, never committed)
│ └── install.sh # One-time installer, run on the Proxmox node hosting LXC 109 + 151
├── now-playing/ # "Listening to" service for Lotus Chat (cinny #491), LXC 151 :8072
│ ├── lotus_now_playing.py
│ ├── test_lotus_now_playing.py
│ └── install.sh
├── deploy/ # Auto-deployment infrastructure
│ ├── lxc151-hookshot.sh # Deploy script for LXC 151 (matrix/hookshot/livekit)
│ ├── lxc106-cinny.sh # Deploy script for LXC 106 (cinny)
@@ -45,6 +49,7 @@ matrix/
├── livekit-graceful-restart.timer # Runs every 5 min
├── draupnir.service
├── mautrix-discord.service # Discord bridge (LXC 151)
├── lotus-now-playing.service # "Listening to" service (LXC 151)
└── cinny-upstream-check.cron # Installed to /etc/cron.d/ on LXC 106 — runs daily at noon
```
@@ -63,7 +68,7 @@ matrix/
| NPM | 10.10.10.27 | 139 | — | — | Nginx Proxy Manager + matrix landing page |
| Authelia | 10.10.10.36 | 167 | — | — | SSO/OIDC provider |
| LLDAP | 10.10.10.39 | 147 | — | — | LDAP user directory |
| Uptime Kuma | 10.10.10.25 | 101 | — | — | Uptime monitoring (micro1 node) |
| Uptime Kuma | 10.10.10.25 | 101 | — | — | Uptime monitoring |
**Key paths on Synapse LXC (151):**
- Synapse config: `/etc/matrix-synapse/homeserver.yaml`
@@ -304,6 +309,7 @@ The token in `draupnir/production.yaml` in this repo is **intentionally redacted
| 7881 | LiveKit RTC TCP | 0.0.0.0 |
| 8070 | voice-limit-guard (fronts lk-jwt-service) | 0.0.0.0 |
| 8071 | lk-jwt-service (behind guard) | 127.0.0.1 (matrix #24; was 0.0.0.0) |
| 8072 | lotus-now-playing ("Listening to", cinny #491) | 0.0.0.0 |
| 8080 | synapse-admin (nginx) | 0.0.0.0 |
| 3478 | coturn STUN/TURN | 0.0.0.0 |
| 5349 | coturn TURNS/TLS | 0.0.0.0 |
@@ -358,31 +364,30 @@ Webhook URL format: `https://matrix.lotusguild.org/webhook/<uuid>`
> the table lists only the first 8 characters (enough to match Hookshot's logs); the full URLs
> live in each sender's settings. Full UUIDs were in this file from 2026-02-20 to 2026-11-03,
> so the IDs below are exposed in git history until rotated (see *Webhook secrets — rotating*).
> Grafana, Proxmox, Uptime Kuma and Ci-Alert were rotated on 2026-11-03 (the old IDs now return 404).
> All 13 were rotated on 2026-11-03, so the IDs in git history are dead (they return 404).
> CI (`readme-no-webhook-ids`) rejects full UUIDs in this README.
| Service | Webhook UUID | Notes |
|---------|-------------|-------|
| Grafana | `e713b383…` | Unified alerting contact point |
| Proxmox | `ca27432c…` | Notification system (8.1+), Discord embed format |
| Sonarr | `aeffc311…` | All event types |
| Radarr | `34913454…` | All event types |
| Readarr | `e57ab4f3…` | All event types |
| Lidarr | `66ac6fdd…` | All event types |
| Sonarr | `c3dde7ba…` | All event types |
| Radarr | `a7584167…` | All event types |
| Readarr | `3b30c0df…` | All event types |
| Uptime Kuma | `4b3fb94b…` | Status change notifications |
| Seerr | `555185af…` | Request/approval events |
| Owncast (Livestream) | `9993e911…` | STREAM_STARTED / STREAM_STOPPED |
| Bazarr | `470fb267…` | Subtitle events (Apprise JSON notifier) |
| Tinker-Tickets | `6e306faf…` | Custom transformation code |
| Seerr | `9c56d489…` | Request/approval events |
| Owncast (Livestream) | `ff37824a…` | STREAM_STARTED / STREAM_STOPPED |
| Bazarr | `263a8301…` | Subtitle events (Apprise JSON notifier) |
| Tinker-Tickets | `135b040b…` | Custom transformation code |
| Ci-Alert | `66c1aa17…` | Gitea Actions `notify-failure` jobs (org secret `MATRIX_WEBHOOK_URL`); transform `hookshot/ci-alert.js` |
| Devdesk | `4a3ccfc9…` | DevDesk "Claude needs you" (CT 126 `devdesk-notify-matrix`, URL in `/etc/devdesk/matrix-webhook`); transform `hookshot/devdesk.js`; avatar set via the appservice token |
| Devdesk | `db7df1a1…` | DevDesk "Claude needs you" (CT 126 `devdesk-notify-matrix`, URL in `/etc/devdesk/matrix-webhook`); transform `hookshot/devdesk.js`; avatar set via the appservice token |
**Hookshot notes:**
- Spam and Stuff is intentionally **unencrypted** — hookshot bridges cannot join E2EE rooms
- JS transformation functions use hookshot v2 API: `result = { version: "v2", plain, html, msgtype }`
- The `result` variable must be assigned without `var`/`let`/`const` (QuickJS IIFE sandbox)
- NPM proxies `https://matrix.lotusguild.org/webhook/*` → `http://10.10.10.29:9003`
- NPM proxies `/sfu/get` and `/get_token` → `http://10.10.10.29:8070` (lk-jwt-service). Both paths are in `/data/nginx/proxy_host/49.conf` on LXC 139 — **NPM will overwrite these if proxy host 49 is re-saved via the UI; re-add both location blocks after any NPM save**
- NPM proxies `/sfu/get`, `/get_token` and `/delegate_delayed_leave` → `http://10.10.10.29:8070` (voice-limit-guard → lk-jwt-service), and `/now-playing/` → `:8072`. Since 2026-11-12 all of proxy host 49's custom locations live in its **Advanced** config (NPM sqlite `proxy_host.advanced_config`), so a UI save keeps them. Change them there (or in both the DB and `/data/nginx/proxy_host/49.conf`), never only in the conf file.
- Proxmox sends Discord embed format: `data.embeds[0].{title,description,fields}` — NOT flat fields
- Transform functions are stored as Matrix room state (`uk.half-shot.matrix-hookshot.generic.hook`) and deployed via `hookshot/deploy.sh`
- Hook UUIDs are **not** in that state. Hookshot creates them and keeps a `{uuid: name}` map in the `@hookshot` bot's room account data (same event type). Deploying a new transform creates the hook. To look a UUID up on LXC 151, use the appservice token from `/etc/matrix-synapse/hookshot-registration.yaml`: `GET /_matrix/client/v3/user/@hookshot:matrix.lotusguild.org/rooms/<room>/account_data/uk.half-shot.matrix-hookshot.generic.hook?user_id=@hookshot:matrix.lotusguild.org`
@@ -413,6 +418,19 @@ channel webhook with their own display name.
- Database: `mautrix_discord` on LXC 109, role `mautrix_discord`, allowed from 10.10.10.29 in `pg_hba.conf`
- Service: `systemctl status mautrix-discord`, logs in `journalctl -u mautrix-discord` and `/var/log/mautrix-discord/`
- End-to-bridge encryption is allowed, so it can join encrypted rooms
- **Patched build.** The deployed binary is upstream v0.7.7 plus `discord-bridge/patches/`, built with
`discord-bridge/build-patched.sh` (deploy steps are in the script's header). `install.sh` still
downloads the stock release, so rebuild and redeploy the patched binary after any (re)install or upgrade.
- `0001`: when a Matrix → Discord send fails before the request reaches Discord (DNS, connect, or
`TLS handshake timeout`), the bridge retries after 2, 5, 10, 20, 30, 60, 60 and 60 s instead of
dropping the message with "⚠ Your message may not have been bridged". The room's later messages wait
in order behind the retry. Edits, reactions and deletes are not retried.
- `0002`: when a relay (webhook) send fails after the request may already have reached Discord
(connection reset, EOF, timeout waiting for the reply, Discord 5xx), the bridge waits up to 6 s for
its own post to come back over the Discord gateway. If it does, the send counts as delivered (no
resend, no copy back into Matrix); if not, it retries with the 0001 backoff, checking again before
each resend. A webhook post has no dedupe, so this echo check is what keeps those retries from
double-posting.
**Install (one time):** after this repo's `main` has reached `/opt/matrix-config` on LXC 151, run this
on the Proxmox node hosting LXC 109 and 151. It restarts Synapse once and is safe to re-run.
@@ -436,6 +454,8 @@ Skip `guilds bridge`: it sets the guild to create a Matrix room for every channe
old room). The relay webhook stays with the channel. Keep `@discordbot` at the default power level so it
can't overwrite the room's name and icon with the Discord channel's.
**Presence (`discord-bridge/lotus_presence.py`, `lotus-discord-presence.service`):** the bridge doesn't carry Discord status, so this small service does. It reuses the bridge's bot (token read from the bridge database at startup; a second gateway session with only the Guilds and Presence intents; **Presence Intent** must stay on in the developer portal) and has its own appservice registration, `/etc/matrix-synapse/discord-presence-registration.yaml`, non-exclusive on `@discord_*`, so it can update ghosts but not create them. Online / idle / DND / offline map to online / unavailable / `org.matrix.msc3026.busy` / offline; custom status and the first activity become the status message. Only people who already have a ghost show up. Synapse drops an unrefreshed user to offline after about a minute, so live states are re-sent every 20 s; if the service dies, ghosts go offline rather than stale. Install once with `discord-bridge/install-presence.sh`, then restart Synapse. Statuses and activities are never logged.
**Bot profile:** `@discordbot` shows as "Discord Bridge" with the Discord logo
(`discord-bridge/bot-avatar.png`). To reapply it: `pct exec 151 -- bash /opt/matrix-config/discord-bridge/set-bot-profile.sh`.
@@ -460,6 +480,18 @@ while the bridge or Synapse was down are bridged when it starts again.
---
## Now Playing ("Listening to", cinny #491)
`now-playing/lotus_now_playing.py` (`lotus-now-playing.service`, LXC 151, `:8072`, NPM `/now-playing/`) gives Lotus Chat what people are playing, for a Discord/Spotify-style "Listening to" card.
- **Source:** Navidrome `getNowPlaying`, polled every 10 s as the non-admin Navidrome user `lotus-nowplaying` (direct to `http://10.10.10.28:4533`). Navidrome is the scrobbling master and Dropped Needle passes the real listener through, so every play shows up under the listener's own Navidrome username, which is their SSO username. Dropped Needle's own `droppedneedle` service user is ignored.
- **Who's asking:** Lotus Chat sends a Matrix OpenID token; the service checks it with Synapse (`/_matrix/federation/v1/openid/userinfo`). Only users of this homeserver can read anything.
- **Matrix user → SSO username:** `/etc/lotus-now-playing/users.json` first (server-only, not in this public repo; for accounts whose SSO link has no email), then the Synapse admin API: an email `<name>@lotusguild.org`, or the legacy plain-username external ID. Uses `MATRIX_TOKEN` from `/etc/matrix-deploy.env`.
- **Opt-in:** off by default; each person turns on "Share what I'm listening to" in Lotus Chat. Stored in `/var/lib/lotus-now-playing/state.json`.
- **API:** `GET`/`PUT /now-playing/v1/me` (`{share, linked, playing}`), `GET /now-playing/v1/users?ids=@a,@b` (up to 50; only opted-in, matched, playing users), and `GET /now-playing/v1/cover/<id>?size=`. The cover endpoint has no auth (for `<img>`) but only serves covers an opted-in user is playing right now.
- Titles, artists and tokens are never logged.
- **Install / update:** create `/etc/lotus-now-playing/env` (`NAVIDROME_USER`, `NAVIDROME_PASSWORD`) once, then `bash /opt/matrix-config/now-playing/install.sh`. Tests: `python3 -m unittest now-playing/test_lotus_now_playing.py`.
## Voice Bridge (nether-voicebridge, Lotus-patched)
Bridges Discord voice into a Lotus voice room. People on Discord appear **inside the Matrix call**
@@ -511,6 +543,7 @@ Every bot account has a proper name and an avatar in one style (white glyph on a
| `@draupnir` | Draupnir | Profile API with Draupnir's token (Draupnir doesn't manage its own profile) |
| `@lotusbot` | Lotus Bot | Already set |
| `@notices` | Server Notices | `server_notices.system_mxid_avatar_url` in `homeserver.yaml` (Lotus logo); needs a Synapse restart |
| `@lotus-mc-bridge` | Lotus Minecraft | Profile API with its own token, from the Minecraft mod's config (voxel block with a lotus, not the white-glyph style; image not in `avatars/`). Its status is the player count: `minecraft-bridge/lotus_mc_status.py`, which runs on CT 154 as `lotus-mc-status.service` (not on LXC 151; installed by hand) |
---
@@ -654,6 +687,27 @@ Infra notes for THIS repo:
**Full handoff & step-by-step plan:** `LotusGuild/cinny` →
[`HANDOFF_ELEMENT_CALL_FORK.md`](https://code.lotusguild.org/LotusGuild/cinny/src/branch/lotus/HANDOFF_ELEMENT_CALL_FORK.md).
### Not planned: background web push
**Decision (2026-11-09):** the web app does **not** get background push
notifications (notifications while no Lotus tab is open), and no push gateway
(Sygnal) is run on our infrastructure.
- **Why:** people who want notifications keep the **desktop app** open (it
notifies natively and keeps running in the tray), and on phones people use
**Element X** or **FluffyChat**, which already push through their own
gateways. A gateway would only help someone using the website with every tab
closed, which doesn't justify another service to run, monitor and secure on
LXC 151.
- **What users get:** OS notifications while the web or desktop app is open,
in-app toasts, quiet hours and pause, email notifications (optional, Synapse).
- **If this changes:** the plan (Sygnal on LXC 151, VAPID key, NPM route locked
to Synapse, config.json `webPush` entry) is in
[cinny #202](https://code.lotusguild.org/LotusGuild/cinny/issues/202#issuecomment-6221),
and the finished-but-unmerged client side (service-worker `push` handler,
pusher registration, setting, tests) is in closed
[cinny PR #489](https://code.lotusguild.org/LotusGuild/cinny/pulls/489).
### Custom Features
All custom code lives in `src/app/` on the `lotus` branch of `code.lotusguild.org/LotusGuild/cinny`. Changes survive upstream merges as long as they don't conflict with the same files upstream touched.
@@ -750,7 +804,7 @@ Periodic `TLS/TCP socket error: Connection reset by peer` in coturn logs. Normal
`net.ipv4.tcp_congestion_control = bbr` must be set on the Proxmox **host**, not inside an unprivileged LXC. All other sysctl tuning (TCP/UDP buffers, fin_timeout) is applied inside LXC 151.
Applied 2026-07-26 via `/etc/sysctl.d/99-bbr.conf` on `compute-storage-01` and `large1`, and
Applied 2026-07-26 via `/etc/sysctl.d/99-bbr.conf` on `compute-storage-01` and `large1` (now `compute-storage-02`), and
**2026-08-16 on `compute-storage-gpu-01`** (which was found on `cubic`/`fq_codel` with `tcp_bbr` not
even loaded). All three nodes that may host LXC 151 now report `bbr`/`fq`. Because the file lives on
the host, **a node that has never hosted 151 will not have it** — check before allowing 151 to migrate
@@ -760,9 +814,9 @@ somewhere new:
sysctl net.ipv4.tcp_congestion_control net.core.default_qdisc # want: bbr / fq
```
### ⚠️ `large1` has a failing DIMM — keep Synapse and PostgreSQL off it
### ⚠️ `compute-storage-02` has a failing DIMM — keep Synapse and PostgreSQL off it
`large1` logs **corrected ECC errors** concentrated on one module (**P0 CHANNEL B, DIMM 1** —
`compute-storage-02` (called `large1` until the 2026-11-18 rename) logs **corrected ECC errors** concentrated on one module (**P0 CHANNEL B, DIMM 1** —
EDAC `mc#0csrow#2channel#1`, 421 corrected errors observed, with the MCE `Over`/overflow flag set
meaning more occurred than were logged). It has hard-crashed **on its own** twice — 2026-07-24 14:19
and 2026-07-26 00:20 — with no clean-shutdown markers and no OOM events, which is the classic
@@ -773,17 +827,18 @@ signature of an uncorrectable ECC error.
Consequences for Matrix:
- On the 2026-07-26 crash, HA relocated **LXC 109 (PostgreSQL)** and **LXC 151 (Synapse)** to
`compute-storage-01`, which has **zero ECC errors across all boots**. They should stay there.
- The HA rule `ha-rule-43cb668b-becf` in `/etc/pve/ha/rules.cfg` currently weights
`compute-storage-01:1,large1:1` — **equal priority, non-strict** — so a future failover can put
Synapse straight back onto the bad-RAM node. Re-weighting to `compute-storage-01:2,large1:1` would
prefer the healthy node while keeping large1 as a fallback. Not changed automatically because that
rule governs 10 resources and `compute-storage-01` has only 46GB RAM vs large1's 128GB.
- LXC 109 (PostgreSQL) and LXC 151 (Synapse) are HA guests and move between nodes; keep them
off compute-storage-02 through the HA rule weights, not by pinning.
- The HA rule `ha-rule-43cb668b-becf` in `/etc/pve/ha/rules.cfg` (checked 2026-11-18) weights
`compute-storage-01:1,compute-storage-02:2,storage-01:2`, so the bad-RAM node is one of the two
**preferred** nodes, and a failover can put Synapse or PostgreSQL onto it. Lowering
`compute-storage-02` below `storage-01` would keep it as a fallback only. Not changed automatically,
because that rule governs 9 CTs and a VM. Look up where 151 and 109 run now with
`ha-manager status | grep -E 'ct:(151|109) '`; they move.
- **Never run a large `VACUUM FULL` or state-compression job on a host with failing RAM** — those
rewrite the whole table through memory, and a bit-flip becomes silent, backup-propagated corruption.
- Ceph's **`noout` flag is intentionally left set.** With 10 OSDs on an intermittently-crashing node,
`noout` prevents a full 10-OSD rebalance every time it bounces. Do not clear it until large1 is stable.
`noout` prevents a full 10-OSD rebalance every time it bounces. Do not clear it until compute-storage-02 is stable.
A replacement DIMM is not currently in budget. Free mitigations worth evaluating: physically removing
the bad module (96GB in flex mode, or 64GB symmetric dual-channel), kernel page soft-offlining via
@@ -867,6 +922,43 @@ SELECT room_id, count(*) FROM state_groups_state GROUP BY room_id ORDER BY 2 DES
Spaces (`m.space`) are the worst offenders per byte — they carry huge membership with zero messages.
Leaving a space does **not** leave its child rooms; they are independent rooms with their own IDs.
**It grows back.** From 2183 MB after the July cleanup, the database reached 6.2 GB by 2026-11-03
(`state_groups_state` 2.75 GB, 11 M rows). Almost all of that came from big federated rooms, each with
one or two local members: *Jellyfin General Chat* (16,754 members, 5.3 M rows), an unnamed 1,213-member
room (2.4 M), *Homelabbing*, *Element X Android*, the *Cinny community* space.
#### Nightly state compression (CT 109)
`synapse_auto_compressor` (from [rust-synapse-compress-state](https://github.com/matrix-org/rust-synapse-compress-state),
built from `main`) runs every night at 04:30, after the 03:00 vzdump backup. It does not delete anything:
it rewrites how a room's state history is stored as smaller deltas.
- **Safety.** It checks that the compressed state matches the original before writing, writes each
state group in its own transaction, and skips any chunk that would not shrink.
- **Progress.** It stores its progress in `state_compressor_*` tables, so each night continues where
the last one stopped.
- **Cost.** No downtime. Synapse's DB p99 stayed at ~10 ms while the initial backlog ran.
- **First run (2026-11-03).** The initial backlog took ~8 minutes, with 0 Synapse errors.
`state_groups_state` went from **11.0 M to 4.25 M rows (−61%)**.
- **Disk.** The table file stays ~2.77 GB on disk: the freed ~1.75 GB is reused space inside it, so
the database stops growing until that is used up. To give it back to the filesystem you need
`VACUUM FULL state_groups_state`. That takes an exclusive lock and stalls Synapse for a minute or
two, so it is only worth doing if CT 109's disk gets tight (it is at 14%).
Units: `systemd/synapse-state-compressor.{service,timer}` (CT 109 has no auto-deploy hook, so install them by
hand). Binary: `/usr/local/bin/synapse_auto_compressor`.
```bash
# on CT 109
install -m644 systemd/synapse-state-compressor.{service,timer} /etc/systemd/system/
systemctl daemon-reload && systemctl enable --now synapse-state-compressor.timer
journalctl -u synapse-state-compressor -n 5 # "Finished running compressor. Saved N rows."
```
The Grafana alert **Synapse Database Growing Fast** (Matrix → Database) warns if the database grows by
more than 500 MB in 7 days. That means either the timer stopped running, or a room is growing faster
than nightly compression can keep up with.
---
## Server Checklist
@@ -926,7 +1018,7 @@ Leaving a space does **not** leave its child rooms; they are independent rooms w
- [x] TCP retransmit timeout lowered (`tcp_retries2=5`, `tcp_syn_retries=4`, `tcp_keepalive_probes=3`) — stalled outbound federation connections now fail in ~15-30s instead of ~15 min
- [x] Unreachable routes added for servers with asymmetric connectivity (can reach us but we can't reach their federation port) — prevents 90s TCP hangs from being added to lag; defined in `/etc/network/interfaces` post-up hooks and survive reboots (bark.lgbt ×2, parodia.dev, chat.ohaa.xyz, matrix.k8ekat.dev)
- [x] Stuck `device_lists_remote_resync` entries cleared for dead-server users (@dalite:bark.lgbt, @arndot:matrix.goch.social) — device list resync was firing every 30s
- [x] BBR congestion control — applied 2026-07-26 on `compute-storage-01` and `large1`, and 2026-08-16 on `compute-storage-gpu-01`, via `/etc/sysctl.d/99-bbr.conf` (`net.ipv4.tcp_congestion_control=bbr`, `net.core.default_qdisc=fq`). Persists across reboots. All three nodes that may host LXC 151 are now covered. If 151 is ever allowed to run on another node, apply it there too.
- [x] BBR congestion control — applied 2026-07-26 on `compute-storage-01` and `large1` (now `compute-storage-02`), and 2026-08-16 on `compute-storage-gpu-01`, via `/etc/sysctl.d/99-bbr.conf` (`net.ipv4.tcp_congestion_control=bbr`, `net.core.default_qdisc=fq`). Persists across reboots. All three nodes that may host LXC 151 are now covered. If 151 is ever allowed to run on another node, apply it there too.
- [x] LXC 109 rootfs grown 30GB → 80GB (2026-07-26). It was at **89% full with only 3.4GB free**, and `pg_wal` shares that volume — PostgreSQL stops outright at 100%. Also a hard prerequisite for any `VACUUM FULL`, which must rewrite the table alongside the original.
- [x] **Purged the `#community:matrix.org` space** (`!iMZEhwCvbfeAYUxAjZ:t2l.io`) 2026-07-26 — 81,724,747 rows / 17GB for a directory wrapper with **zero messages** and one local member. Admin API `DELETE /_synapse/admin/v2/rooms/` with `purge:true`, then `VACUUM FULL`. ~2.5 min downtime.
- [x] **Compressed room state** with `rust-synapse-compress-state` (built at `/opt/rust-synapse-compress-state/` on LXC 109). Every run verified `New state map matches old one` before commit:
@@ -942,7 +1034,7 @@ Leaving a space does **not** leave its child rooms; they are independent rooms w
> **Do NOT compress `Spam and Stuff`.** Its 107,763 state groups hold only 107,799 rows (~1 row each) — already minimal. The compressor's leveled structure would *add* rows (dry run: **110.16%**) and it refuses to run. High state-group count is not the same as compressible bloat.
- [x] PostgreSQL: `jit=off`, `wal_compression=lz4`, `max_worker_processes` 3→8, `maintenance_work_mem` 256MB→512MB, `shared_buffers` 1500MB→2GB in `/etc/postgresql/17/main/conf.d/synapse_tuning.conf`. Previous version kept as `synapse_tuning.conf.bak-20260726`. **That file is not in this repo — the tuning is unversioned.**
- [ ] `synchronous_commit = off` — **deliberately NOT applied.** Would be a real write-latency win, but this cluster hard-resets every few hours (HA self-fences on corosync quorum loss + the large1 DIMM), so it would drop up to ~600ms of committed messages *per reset* rather than once in a blue moon. Revisit once the cluster is stable.
- [ ] `synchronous_commit = off` — **deliberately NOT applied.** Would be a real write-latency win, but this cluster hard-resets every few hours (HA self-fences on corosync quorum loss + the compute-storage-02 DIMM), so it would drop up to ~600ms of committed messages *per reset* rather than once in a blue moon. Revisit once the cluster is stable.
### Auth & SSO
- [x] Open registration disabled (`enable_registration: false`, 2026-10-16): accounts come from LLDAP and are created on first sign-in through Lotus SSO (Authelia OIDC, which keeps its own registration on). The old registration token was deleted.
@@ -1012,10 +1104,12 @@ Only `config.yml` and `passkey.pem` in it are not generated; storage is in-memor
- 2026-11-03: 7.3.2 → 7.5.0 (GHSA-5c49-6fvm-2xvc: `/metrics` exposed webhook URLs, plus two DoS fixes).
Old build kept at `/opt/hookshot-7.3.2`.
## Call Stack Upgrade (lk-jwt-service 0.4.1 → 0.9.0, LiveKit 1.9.11 → 1.13.11) — planned
## Call Stack Upgrade (lk-jwt-service 0.4.1 → 0.9.0, LiveKit 1.9.11 → 1.13.11) — done 2026-11-03
Tested on the local stack on 2026-11-03 (calls through the guard, voice-bridge e2e audio both ways,
encrypted and not). Required changes, all on LXC 151:
Tested on the local stack, then applied on LXC 151 on 2026-11-03 with release binaries (sha256-checked).
Rollback: `/usr/local/bin/lk-jwt-service.0.4.1`, `/usr/local/bin/livekit-server.1.9.11`,
`/etc/livekit/config.yaml.bak-20261103`, and drop the `LIVEKIT_SANITY_CHECK_INTERVAL_SECONDS` line from
`/etc/systemd/system/lk-jwt-service.service.d/override.conf`. The changes:
1. **lk-jwt-service 0.9.0** (a Rust rewrite; fixes GHSA-rq24-fh66-rq7x). Existing env vars carry over.
Add `LIVEKIT_SANITY_CHECK_INTERVAL_SECONDS=30`.
@@ -1228,8 +1322,8 @@ All commands use the `!` prefix. Run `!help` in any room for the full list.
| Homeserver | Synapse | 1.159.0 |
| Database | PostgreSQL | 17.9 |
| TURN | coturn | latest |
| Video/voice calls | LiveKit SFU | 1.9.11 |
| LiveKit JWT | lk-jwt-service | 0.4.1 (0.9.0 planned, see *Call stack upgrade*) |
| Video/voice calls | LiveKit SFU | 1.13.11 |
| LiveKit JWT | lk-jwt-service | 0.9.0 |
| Moderation | Draupnir | 2.9.0 |
| SSO | Authelia (OIDC) + LLDAP | — |
| Webhook bridge | matrix-hookshot | 7.5.0 |
+7 -1
View File
@@ -28,12 +28,18 @@
"statusPages": {
"matrix.lotusguild.org": {
"url": "https://isitup.lotusguild.org",
"slug": "matrix",
"slug": "public",
"noticePrefix": "Matrix:",
"groups": {
"homeserver": "Homeserver",
"calls": "Voice calls",
"login": "Login"
}
}
},
"nowPlaying": {
"matrix.lotusguild.org": {
"url": "https://matrix.lotusguild.org/now-playing"
}
}
}
+13
View File
@@ -54,6 +54,19 @@ else
&& install -m755 "$REPO_DIR/deploy/livekit-graceful-restart.sh" /usr/local/bin/livekit-graceful-restart.sh
fi
# Discord presence service: only once installed (discord-bridge/install-presence.sh).
if [ -f /etc/lotus-presence/env ] && echo "$CHANGED" | grep -qE '^discord-bridge/lotus_presence\.py|^systemd/lotus-discord-presence\.service'; then
echo "Deploying lotus-discord-presence..."
if python3 -m py_compile "$REPO_DIR/discord-bridge/lotus_presence.py"; then
install -D -m644 "$REPO_DIR/discord-bridge/lotus_presence.py" /opt/lotus-presence/lotus_presence.py
install -m644 "$REPO_DIR/systemd/lotus-discord-presence.service" /etc/systemd/system/lotus-discord-presence.service
systemctl daemon-reload
systemctl restart lotus-discord-presence || true
else
echo "ERROR: lotus_presence.py has a syntax error, keeping the running version."
fi
fi
# Voice-limit / quality guard (fronts lk-jwt-service on :8070)
if echo "$CHANGED" | grep -qE '^livekit/voice-limit-guard\.py|^systemd/voice-limit-guard\.service'; then
echo "Deploying voice-limit-guard..."
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Build mautrix-discord with the Lotus patches in discord-bridge/patches/.
#
# Needs: git, Go (version from upstream go.mod or newer), a C compiler and libolm-dev.
# Output: ./mautrix-discord-lotus (dynamically linked against libolm.so.3, which
# LXC 151 has from the libolm3 package).
#
# Deploy (on the Proxmox node hosting LXC 151):
# pct push 151 mautrix-discord-lotus /opt/mautrix-discord/mautrix-discord.new
# pct exec 151 -- bash -c 'cd /opt/mautrix-discord && cp -a mautrix-discord mautrix-discord.bak-$(date +%Y%m%d) \
# && chown mautrix-discord: mautrix-discord.new && chmod 755 mautrix-discord.new \
# && mv mautrix-discord.new mautrix-discord && systemctl restart mautrix-discord'
set -euo pipefail
VERSION="${VERSION:-v0.7.7}"
HERE="$(cd "$(dirname "$0")" && pwd)"
OUT="${OUT:-$PWD/mautrix-discord-lotus}"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
git clone -q --depth 1 --branch "$VERSION" https://github.com/mautrix/discord.git "$WORK/src"
cd "$WORK/src"
for p in "$HERE"/patches/*.patch; do
echo "applying $(basename "$p")"
git apply "$p"
done
go test -race -count=1 -run 'Retry|PreSend|Ambiguous|Echo' .
COMMIT="$(git rev-parse HEAD)"
MAUTRIX="$(go list -m -f '{{.Version}}' maunium.net/go/mautrix)"
CGO_ENABLED=1 go build -o "$OUT" -ldflags "-s -w \
-X main.Tag=$VERSION \
-X main.Commit=$COMMIT+lotus \
-X 'main.BuildTime=$(date '+%b %_d %Y, %H:%M:%S')' \
-X 'maunium.net/go/mautrix.GoModVersion=$MAUTRIX'" .
"$OUT" --version
echo "built $OUT"
+56
View File
@@ -0,0 +1,56 @@
#!/bin/bash
# One-time install of the Discord presence service (lotus_presence.py).
# Run on LXC 151 with this repo's main branch pulled to /opt/matrix-config:
# bash /opt/matrix-config/discord-bridge/install-presence.sh
# Generates its own appservice credentials on this machine and never prints them.
# Does NOT restart Synapse: until you do, Discord works but Matrix answers 401.
# Safe to re-run: existing credentials are kept.
set -euo pipefail
REPO=/opt/matrix-config
REG=/etc/matrix-synapse/discord-presence-registration.yaml
ENV_FILE=/etc/lotus-presence/env
HS=/etc/matrix-synapse/homeserver.yaml
install -d -m750 -o root -g mautrix-discord /etc/lotus-presence
install -D -m644 "$REPO/discord-bridge/lotus_presence.py" /opt/lotus-presence/lotus_presence.py
if [ ! -f "$ENV_FILE" ]; then
AS_TOKEN="$(openssl rand -hex 32)"
HS_TOKEN="$(openssl rand -hex 32)"
(umask 027; printf 'AS_TOKEN=%s\n' "$AS_TOKEN" > "$ENV_FILE")
chown root:mautrix-discord "$ENV_FILE"
(umask 027; cat > "$REG" <<YAML
# Discord presence service. Non-exclusive on the ghost namespace: it updates
# presence on ghosts the text bridge owns, and cannot create them.
id: lotus-discord-presence
url: null
as_token: $AS_TOKEN
hs_token: $HS_TOKEN
sender_localpart: discordpresence
rate_limited: false
namespaces:
users:
- regex: '^@discord_[0-9]+:matrix\.lotusguild\.org$'
exclusive: false
rooms: []
aliases: []
YAML
)
chown root:matrix-synapse "$REG"
echo "generated appservice credentials"
else
echo "keeping existing credentials"
fi
if ! grep -qF "$REG" "$HS"; then
cp -p "$HS" "$HS.bak-presence-$(date +%Y%m%d%H%M%S)"
sed -i "/^app_service_config_files:/a\\ - $REG" "$HS"
echo "added the registration to homeserver.yaml (takes effect on the next Synapse restart)"
fi
install -m644 "$REPO/systemd/lotus-discord-presence.service" /etc/systemd/system/lotus-discord-presence.service
systemctl daemon-reload
systemctl enable --now lotus-discord-presence
systemctl restart lotus-discord-presence
echo "installed. Restart Synapse once to activate the registration: systemctl restart matrix-synapse"
+335
View File
@@ -0,0 +1,335 @@
#!/usr/bin/env python3
"""Discord presence -> Matrix ghosts.
mautrix-discord doesn't bridge presence, so this small service does: it watches
the Discord server over the gateway (GUILD_PRESENCES) and sets presence and a
status message on each member's @discord_<id> ghost, which Lotus already shows.
- Discord bot: the text bridge's own bot, read from its database at startup, so
there is no extra bot. This is a second gateway session asking only for the
intents it needs; the bridge's session is untouched.
- Matrix: its own appservice registration (non-exclusive on the ghost namespace;
the bridge still owns the ghosts). Only ghosts that already exist are updated.
- Synapse drops a user nobody refreshes to offline, so live states are re-sent
on a timer. That is also the failure mode we want: if this service dies, the
ghosts go offline rather than showing a stale "online" forever.
Member statuses and activities are never logged.
"""
import asyncio
import logging
import os
import re
import subprocess
import sys
import time
import urllib.parse
import aiohttp
log = logging.getLogger("lotus-presence")
GATEWAY = "wss://gateway.discord.gg/?v=10&encoding=json"
INTENT_GUILDS = 1 << 0
INTENT_GUILD_PRESENCES = 1 << 8
# Close codes where reconnecting cannot help: bad token, disallowed intents.
FATAL_CLOSE = {4004, 4013, 4014}
BUSY = "org.matrix.msc3026.busy"
STATUS_MAP = {"online": "online", "idle": "unavailable", "dnd": BUSY}
MAX_STATUS = 120
# A ghost that doesn't exist yet (never spoke in a bridged room) is retried this often.
UNKNOWN_RETRY_S = 600
def describe_activity(activity):
"""One Discord activity as text, or None. Types per Discord's Activity object."""
kind = activity.get("type")
name = (activity.get("name") or "").strip()
if kind == 4: # custom status
emoji = (activity.get("emoji") or {})
# Custom server emoji have an id and can't be shown as text.
glyph = emoji.get("name") if not emoji.get("id") else None
text = (activity.get("state") or "").strip()
return " ".join(p for p in (glyph, text) if p) or None
if not name:
return None
if kind == 0:
return f"Playing {name}"
if kind == 1:
return f"Streaming {activity.get('details') or name}"
if kind == 2:
song, artist = activity.get("details"), activity.get("state")
if song and artist:
return f"Listening to {song} by {artist}"
return f"Listening to {song or name}"
if kind == 3:
return f"Watching {name}"
if kind == 5:
return f"Competing in {name}"
return None
def map_presence(status, activities):
"""Discord status + activities -> (Matrix presence, status message)."""
presence = STATUS_MAP.get(status)
if presence is None: # offline / invisible: show nothing about them
return "offline", ""
custom = [a for a in activities or [] if a.get("type") == 4]
others = [a for a in activities or [] if a.get("type") != 4]
parts = []
for activity in custom + others[:1]:
text = describe_activity(activity)
if text:
parts.append(text)
message = " · ".join(parts)
if len(message) > MAX_STATUS:
message = message[: MAX_STATUS - 1].rstrip() + "…"
return presence, message
def ghost_id(discord_id, server_name):
return f"@discord_{discord_id}:{server_name}"
def bridge_db_env(config_text):
"""libpq settings from the bridge config's database URI (password stays out of argv)."""
match = re.search(r"^\s*uri:\s*[\"']?(postgres(?:ql)?://[^\"'\s]+)", config_text, re.M)
if not match:
raise RuntimeError("no postgres uri in the bridge config")
uri = urllib.parse.urlparse(match.group(1))
env = {
"PGHOST": uri.hostname or "",
"PGPORT": str(uri.port or 5432),
"PGUSER": urllib.parse.unquote(uri.username or ""),
"PGPASSWORD": urllib.parse.unquote(uri.password or ""),
"PGDATABASE": uri.path.lstrip("/"),
}
sslmode = urllib.parse.parse_qs(uri.query).get("sslmode")
if sslmode:
env["PGSSLMODE"] = sslmode[0]
return env
def read_bot_token(config_path):
"""The text bridge's bot token, from its database. Never logged."""
with open(config_path, encoding="utf-8") as handle:
env = {**os.environ, **bridge_db_env(handle.read())}
out = subprocess.run(
["psql", "-Atc", "select discord_token from \"user\" where discord_token like 'Bot %'"],
env=env,
capture_output=True,
text=True,
timeout=30,
check=False,
)
if out.returncode != 0:
raise RuntimeError("could not read the bot token from the bridge database")
tokens = [line for line in out.stdout.splitlines() if line]
if len(tokens) != 1:
raise RuntimeError(f"expected exactly one bridge bot login, found {len(tokens)}")
return tokens[0].removeprefix("Bot ")
class GhostPresence:
"""What each ghost should show, pushed to Synapse on change and on a timer."""
def __init__(self, session, homeserver, as_token, server_name, clock=time.monotonic):
self.session = session
self.homeserver = homeserver.rstrip("/")
self.as_token = as_token
self.server_name = server_name
self.clock = clock
self.live = {} # discord id -> (presence, message), everything not offline
self.unknown_until = {} # discord id -> when to try a missing ghost again
async def _put(self, discord_id, presence, message):
user = urllib.parse.quote(ghost_id(discord_id, self.server_name), safe="")
url = f"{self.homeserver}/_matrix/client/v3/presence/{user}/status?user_id={user}"
body = {"presence": presence, "status_msg": message}
headers = {"Authorization": f"Bearer {self.as_token}"}
try:
async with self.session.put(
url, json=body, headers=headers, timeout=aiohttp.ClientTimeout(total=10)
) as resp:
# Synapse answers 403 for a user the appservice hasn't registered.
if resp.status == 403:
self.unknown_until[discord_id] = self.clock() + UNKNOWN_RETRY_S
elif resp.status >= 400:
log.warning("presence update failed: HTTP %s", resp.status)
else:
self.unknown_until.pop(discord_id, None)
except (aiohttp.ClientError, asyncio.TimeoutError) as err:
log.warning("presence update failed: %s", type(err).__name__)
def _skip(self, discord_id):
return self.unknown_until.get(discord_id, 0) > self.clock()
async def update(self, discord_id, presence, message):
known = self.live.get(discord_id)
if presence == "offline":
if known is None:
return
del self.live[discord_id]
else:
if known == (presence, message):
return
self.live[discord_id] = (presence, message)
if not self._skip(discord_id):
await self._put(discord_id, presence, message)
async def resync(self, current):
"""A full member snapshot: `current` replaces what we had."""
for discord_id in set(self.live) - set(current):
await self.update(discord_id, "offline", "")
for discord_id, (presence, message) in current.items():
await self.update(discord_id, presence, message)
async def refresh(self):
for discord_id, (presence, message) in list(self.live.items()):
if not self._skip(discord_id):
await self._put(discord_id, presence, message)
class FatalGatewayError(Exception):
pass
class Gateway:
def __init__(self, session, token, guild_id, ghosts):
self.session = session
self.token = token
self.guild_id = guild_id
self.ghosts = ghosts
self.seq = None
self.session_id = None
self.resume_url = None
self.acked = True
async def _heartbeat(self, ws, interval_s):
while True:
await asyncio.sleep(interval_s)
if not self.acked:
log.warning("gateway missed a heartbeat ack, reconnecting")
await ws.close(code=4000)
return
self.acked = False
await ws.send_json({"op": 1, "d": self.seq})
async def _dispatch(self, kind, data):
if kind == "READY":
self.session_id = data["session_id"]
self.resume_url = data.get("resume_gateway_url")
log.info("gateway ready")
elif kind == "GUILD_CREATE" and data.get("id") == self.guild_id:
current = {}
for item in data.get("presences") or []:
presence, message = map_presence(item.get("status"), item.get("activities"))
if presence != "offline":
current[item["user"]["id"]] = (presence, message)
await self.ghosts.resync(current)
log.info("synced %d online members", len(current))
elif kind == "PRESENCE_UPDATE" and data.get("guild_id") == self.guild_id:
presence, message = map_presence(data.get("status"), data.get("activities"))
await self.ghosts.update(data["user"]["id"], presence, message)
async def _run_once(self):
resuming = bool(self.session_id and self.resume_url)
url = f"{self.resume_url}/?v=10&encoding=json" if resuming else GATEWAY
beat = None
async with self.session.ws_connect(url, heartbeat=None, max_msg_size=0) as ws:
try:
async for frame in ws:
if frame.type != aiohttp.WSMsgType.TEXT:
break
msg = frame.json()
if msg.get("s") is not None:
self.seq = msg["s"]
op = msg.get("op")
if op == 10:
self.acked = True
interval = msg["d"]["heartbeat_interval"] / 1000
beat = asyncio.create_task(self._heartbeat(ws, interval))
if resuming:
await ws.send_json({"op": 6, "d": {
"token": self.token,
"session_id": self.session_id,
"seq": self.seq,
}})
else:
await ws.send_json({"op": 2, "d": {
"token": self.token,
"intents": INTENT_GUILDS | INTENT_GUILD_PRESENCES,
"properties": {
"os": "linux",
"browser": "lotus-presence",
"device": "lotus-presence",
},
}})
elif op == 11:
self.acked = True
elif op == 1:
await ws.send_json({"op": 1, "d": self.seq})
elif op == 7:
return
elif op == 9:
if not msg.get("d"):
self.session_id = None
return
elif op == 0:
await self._dispatch(msg.get("t"), msg.get("d") or {})
finally:
if beat:
beat.cancel()
if ws.close_code in FATAL_CLOSE:
raise FatalGatewayError(f"gateway refused us (close code {ws.close_code})")
if ws.close_code:
log.info("gateway closed (code %s)", ws.close_code)
async def run(self):
backoff = 1
while True:
started = time.monotonic()
try:
await self._run_once()
except (aiohttp.ClientError, asyncio.TimeoutError, OSError) as err:
log.warning("gateway connection failed: %s", type(err).__name__)
if time.monotonic() - started > 60:
backoff = 1
await asyncio.sleep(backoff)
backoff = min(backoff * 2, 60)
async def refresh_loop(ghosts, interval_s):
while True:
await asyncio.sleep(interval_s)
await ghosts.refresh()
async def amain():
token = read_bot_token(os.environ["BRIDGE_CONFIG"])
async with aiohttp.ClientSession() as session:
ghosts = GhostPresence(
session,
os.environ.get("HOMESERVER_URL", "http://127.0.0.1:8008"),
os.environ["AS_TOKEN"],
os.environ["SERVER_NAME"],
)
gateway = Gateway(session, token, os.environ["DISCORD_GUILD_ID"], ghosts)
refresh_s = float(os.environ.get("REFRESH_SECONDS", "20"))
await asyncio.gather(gateway.run(), refresh_loop(ghosts, refresh_s))
def main():
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
# Discord's own clients log request URLs at DEBUG; keep ours quiet regardless.
logging.getLogger("aiohttp").setLevel(logging.WARNING)
try:
asyncio.run(amain())
except FatalGatewayError as err:
log.error("%s", err)
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,253 @@
diff --git a/lotus_retry.go b/lotus_retry.go
new file mode 100644
index 0000000..972b612
--- /dev/null
+++ b/lotus_retry.go
@@ -0,0 +1,95 @@
+package main
+
+// [lotus] Retry Matrix→Discord sends that failed before reaching Discord.
+//
+// Upstream sends each message exactly once; a momentary network blip (for
+// example "net/http: TLS handshake timeout") drops the message and only posts
+// a "may not have been bridged" notice. We retry, but only for failures that
+// happen before the HTTP request is written (DNS, dial, TLS handshake). Those
+// cannot have reached Discord, so a retry can never post a duplicate.
+// Failures after the request was sent (read timeouts, resets, 5xx) are left
+// alone because webhook sends have no server-side dedupe.
+//
+// The retry runs inline in the portal's Matrix event loop, which handles a
+// room's events one at a time, so later messages queue behind it and ordering
+// is preserved.
+
+import (
+ "errors"
+ "io"
+ "net"
+ "strings"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+ "github.com/rs/zerolog"
+)
+
+var lotusSendRetryDelays = []time.Duration{
+ 2 * time.Second,
+ 5 * time.Second,
+ 10 * time.Second,
+ 20 * time.Second,
+ 30 * time.Second,
+ 60 * time.Second,
+ 60 * time.Second,
+ 60 * time.Second,
+}
+
+// lotusSleep is swapped out in tests.
+var lotusSleep = time.Sleep
+
+// isPreSendNetworkError reports whether err happened before the request could
+// have been written to Discord.
+func isPreSendNetworkError(err error) bool {
+ if err == nil {
+ return false
+ }
+ var restErr *discordgo.RESTError
+ if errors.As(err, &restErr) {
+ // Discord answered, so the request was delivered.
+ return false
+ }
+ var dnsErr *net.DNSError
+ if errors.As(err, &dnsErr) {
+ return true
+ }
+ var opErr *net.OpError
+ if errors.As(err, &opErr) && opErr.Op == "dial" {
+ return true
+ }
+ msg := err.Error()
+ return strings.Contains(msg, "TLS handshake timeout") ||
+ strings.Contains(msg, "proxyconnect")
+}
+
+func lotusRewindFiles(files []*discordgo.File) {
+ for _, f := range files {
+ if f == nil {
+ continue
+ }
+ if s, ok := f.Reader.(io.Seeker); ok {
+ _, _ = s.Seek(0, io.SeekStart)
+ }
+ }
+}
+
+// lotusRetrySend calls send until it succeeds, fails with an error that is not
+// a pre-send network error, or the retry budget runs out.
+func lotusRetrySend[T any](log zerolog.Logger, files []*discordgo.File, send func() (T, error)) (T, error) {
+ res, err := send()
+ for attempt := 0; attempt < len(lotusSendRetryDelays) && isPreSendNetworkError(err); attempt++ {
+ delay := lotusSendRetryDelays[attempt]
+ log.Warn().Err(err).
+ Int("attempt", attempt+1).
+ Dur("retry_in", delay).
+ Msg("[lotus] Discord send failed before reaching Discord, retrying")
+ lotusSleep(delay)
+ lotusRewindFiles(files)
+ res, err = send()
+ if err == nil {
+ log.Info().Int("attempts", attempt+2).Msg("[lotus] Discord send succeeded after retry")
+ }
+ }
+ return res, err
+}
diff --git a/lotus_retry_test.go b/lotus_retry_test.go
new file mode 100644
index 0000000..efa262e
--- /dev/null
+++ b/lotus_retry_test.go
@@ -0,0 +1,108 @@
+package main
+
+import (
+ "bytes"
+ "errors"
+ "io"
+ "net"
+ "net/http"
+ "net/url"
+ "testing"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+ "github.com/rs/zerolog"
+)
+
+type tlsTimeout struct{}
+
+func (tlsTimeout) Error() string { return "net/http: TLS handshake timeout" }
+func (tlsTimeout) Timeout() bool { return true }
+func (tlsTimeout) Temporary() bool { return true }
+
+func urlErr(err error) error {
+ return &url.Error{Op: "Post", URL: "https://discord.com/api/v9/webhooks/1/x?wait=true", Err: err}
+}
+
+func TestIsPreSendNetworkError(t *testing.T) {
+ cases := []struct {
+ name string
+ err error
+ want bool
+ }{
+ {"nil", nil, false},
+ {"tls handshake timeout", urlErr(tlsTimeout{}), true},
+ {"dns", urlErr(&net.DNSError{Err: "no such host", Name: "discord.com"}), true},
+ {"dial refused", urlErr(&net.OpError{Op: "dial", Net: "tcp", Err: errors.New("connection refused")}), true},
+ {"read reset after send", urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")}), false},
+ {"response timeout", urlErr(errors.New("context deadline exceeded (Client.Timeout exceeded while awaiting headers)")), false},
+ {"discord rest error", &discordgo.RESTError{Response: &http.Response{StatusCode: 500}}, false},
+ {"other", errors.New("unknown message type"), false},
+ }
+ for _, c := range cases {
+ if got := isPreSendNetworkError(c.err); got != c.want {
+ t.Errorf("%s: got %v want %v", c.name, got, c.want)
+ }
+ }
+}
+
+func withNoSleep(t *testing.T) *[]time.Duration {
+ var slept []time.Duration
+ old := lotusSleep
+ lotusSleep = func(d time.Duration) { slept = append(slept, d) }
+ t.Cleanup(func() { lotusSleep = old })
+ return &slept
+}
+
+func TestRetrySendRecoversAndRewindsFiles(t *testing.T) {
+ slept := withNoSleep(t)
+ file := &discordgo.File{Name: "a.txt", Reader: bytes.NewReader([]byte("hello"))}
+ calls := 0
+ msg, err := lotusRetrySend(zerolog.Nop(), []*discordgo.File{file}, func() (*discordgo.Message, error) {
+ calls++
+ data, _ := io.ReadAll(file.Reader)
+ if string(data) != "hello" {
+ t.Fatalf("call %d read %q, file not rewound", calls, data)
+ }
+ if calls < 3 {
+ return nil, urlErr(tlsTimeout{})
+ }
+ return &discordgo.Message{ID: "42"}, nil
+ })
+ if err != nil || msg == nil || msg.ID != "42" {
+ t.Fatalf("got msg=%v err=%v", msg, err)
+ }
+ if calls != 3 {
+ t.Fatalf("calls=%d want 3", calls)
+ }
+ if len(*slept) != 2 || (*slept)[0] != 2*time.Second || (*slept)[1] != 5*time.Second {
+ t.Fatalf("slept=%v", *slept)
+ }
+}
+
+func TestRetrySendDoesNotRetryAfterSend(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ _, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")})
+ })
+ if err == nil || calls != 1 {
+ t.Fatalf("calls=%d err=%v, want exactly one attempt", calls, err)
+ }
+}
+
+func TestRetrySendGivesUpAfterBudget(t *testing.T) {
+ slept := withNoSleep(t)
+ calls := 0
+ _, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, urlErr(tlsTimeout{})
+ })
+ if err == nil {
+ t.Fatal("expected error after budget")
+ }
+ if calls != len(lotusSendRetryDelays)+1 || len(*slept) != len(lotusSendRetryDelays) {
+ t.Fatalf("calls=%d slept=%d", calls, len(*slept))
+ }
+}
diff --git a/portal.go b/portal.go
index db26a0e..cc6da0d 100644
--- a/portal.go
+++ b/portal.go
@@ -1743,18 +1743,24 @@ func (portal *Portal) handleMatrixMessage(sender *User, evt *event.Event) {
sendReq.Nonce = generateNonce()
var msg *discordgo.Message
var err error
+ // [lotus] retry sends that failed before reaching Discord (see lotus_retry.go)
+ sendLog := portal.log.With().Str("event_id", evt.ID.String()).Logger()
if !isWebhookSend {
- msg, err = sess.ChannelMessageSendComplex(channelID, &sendReq, portal.RefererOptIfUser(sess, threadID)...)
+ msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
+ return sess.ChannelMessageSendComplex(channelID, &sendReq, portal.RefererOptIfUser(sess, threadID)...)
+ })
} else {
username, avatarURL := portal.getRelayUserMeta(sender)
- msg, err = relayClient.WebhookThreadExecute(portal.RelayWebhookID, portal.RelayWebhookSecret, true, threadID, &discordgo.WebhookParams{
- Content: sendReq.Content,
- Username: username,
- AvatarURL: avatarURL,
- Files: sendReq.Files,
- Components: sendReq.Components,
- Embeds: sendReq.Embeds,
- AllowedMentions: sendReq.AllowedMentions,
+ msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
+ return relayClient.WebhookThreadExecute(portal.RelayWebhookID, portal.RelayWebhookSecret, true, threadID, &discordgo.WebhookParams{
+ Content: sendReq.Content,
+ Username: username,
+ AvatarURL: avatarURL,
+ Files: sendReq.Files,
+ Components: sendReq.Components,
+ Embeds: sendReq.Embeds,
+ AllowedMentions: sendReq.AllowedMentions,
+ })
})
}
sender.handlePossible40002(err)
@@ -0,0 +1,437 @@
diff --git a/lotus_echo.go b/lotus_echo.go
new file mode 100644
index 0000000..0cfdd95
--- /dev/null
+++ b/lotus_echo.go
@@ -0,0 +1,118 @@
+package main
+
+// [lotus] Detect whether a webhook send that errored actually reached Discord.
+//
+// Discord echoes every webhook post back over the gateway as MESSAGE_CREATE.
+// While a relay send is in flight we register a waiter for its channel; the
+// gateway handler (user.go) offers every new message to the waiters before
+// queueing it for the portal. If a send fails ambiguously (the connection broke
+// after the request may have gone out), lotusRetrySend asks the waiter whether
+// the post showed up. If it did, the send is treated as successful and the
+// message ID recorded, so it is neither resent to Discord nor bridged back into
+// Matrix as a new message.
+//
+// The waiter is offered messages from the gateway goroutine, which keeps
+// running while the portal's own loop is blocked in the send.
+
+import (
+ "strings"
+ "sync"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+)
+
+// lotusEchoWait is how long to wait for the gateway echo after an ambiguous
+// failure. Echoes normally arrive well under a second after Discord accepts the
+// post.
+var lotusEchoWait = 6 * time.Second
+
+type lotusEchoWaiter struct {
+ match func(*discordgo.Message) bool
+ ch chan *discordgo.Message
+}
+
+var (
+ lotusEchoMu sync.Mutex
+ lotusEchoWaiters = map[string][]*lotusEchoWaiter{}
+)
+
+// lotusNoteGatewayMessage offers a newly created Discord message to any waiter
+// registered for its channel. Called for every MESSAGE_CREATE.
+func lotusNoteGatewayMessage(msg *discordgo.Message) {
+ if msg == nil || msg.WebhookID == "" {
+ return
+ }
+ lotusEchoMu.Lock()
+ defer lotusEchoMu.Unlock()
+ for _, w := range lotusEchoWaiters[msg.ChannelID] {
+ if w.match(msg) {
+ select {
+ case w.ch <- msg:
+ default:
+ }
+ }
+ }
+}
+
+// lotusExpectEcho registers a waiter for channelID. check(wait) returns the
+// matching message if one has arrived, waiting up to lotusEchoWait when wait
+// is true. done must be called when the send is finished.
+func lotusExpectEcho(channelID string, match func(*discordgo.Message) bool) (check func(wait bool) (*discordgo.Message, bool), done func()) {
+ w := &lotusEchoWaiter{match: match, ch: make(chan *discordgo.Message, 1)}
+ lotusEchoMu.Lock()
+ lotusEchoWaiters[channelID] = append(lotusEchoWaiters[channelID], w)
+ lotusEchoMu.Unlock()
+
+ check = func(wait bool) (*discordgo.Message, bool) {
+ if !wait {
+ select {
+ case msg := <-w.ch:
+ return msg, true
+ default:
+ return nil, false
+ }
+ }
+ timer := time.NewTimer(lotusEchoWait)
+ defer timer.Stop()
+ select {
+ case msg := <-w.ch:
+ return msg, true
+ case <-timer.C:
+ return nil, false
+ }
+ }
+ done = func() {
+ lotusEchoMu.Lock()
+ defer lotusEchoMu.Unlock()
+ list := lotusEchoWaiters[channelID]
+ for i, x := range list {
+ if x == w {
+ list = append(list[:i], list[i+1:]...)
+ break
+ }
+ }
+ if len(list) == 0 {
+ delete(lotusEchoWaiters, channelID)
+ } else {
+ lotusEchoWaiters[channelID] = list
+ }
+ }
+ return check, done
+}
+
+// lotusWebhookEchoMatcher matches the gateway echo of a relay post: same
+// webhook, same text (ignoring surrounding whitespace, which Discord trims),
+// same number of attachments, created no earlier than shortly before the send
+// started.
+func lotusWebhookEchoMatcher(webhookID, content string, numFiles int, started time.Time) func(*discordgo.Message) bool {
+ notBefore := started.Add(-2 * time.Second)
+ content = strings.TrimSpace(content)
+ return func(msg *discordgo.Message) bool {
+ if msg.WebhookID != webhookID || strings.TrimSpace(msg.Content) != content || len(msg.Attachments) != numFiles {
+ return false
+ }
+ ts, err := discordgo.SnowflakeTimestamp(msg.ID)
+ return err != nil || !ts.Before(notBefore)
+ }
+}
diff --git a/lotus_echo_test.go b/lotus_echo_test.go
new file mode 100644
index 0000000..6f5ad44
--- /dev/null
+++ b/lotus_echo_test.go
@@ -0,0 +1,153 @@
+package main
+
+import (
+ "errors"
+ "net"
+ "net/http"
+ "strconv"
+ "testing"
+ "time"
+
+ "github.com/bwmarrin/discordgo"
+ "github.com/rs/zerolog"
+)
+
+func resetErr() error {
+ return urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")})
+}
+
+func TestIsAmbiguousSendError(t *testing.T) {
+ cases := []struct {
+ name string
+ err error
+ want bool
+ }{
+ {"nil", nil, false},
+ {"tls handshake timeout is pre-send", urlErr(tlsTimeout{}), false},
+ {"read reset", resetErr(), true},
+ {"awaiting headers", urlErr(errors.New("context deadline exceeded (Client.Timeout exceeded while awaiting headers)")), true},
+ {"unexpected EOF", urlErr(errors.New("unexpected EOF")), true},
+ {"discord 502", &discordgo.RESTError{Response: &http.Response{StatusCode: 502}}, true},
+ {"discord 400", &discordgo.RESTError{Response: &http.Response{StatusCode: 400}}, false},
+ {"other", errors.New("unknown message type"), false},
+ }
+ for _, c := range cases {
+ if got := isAmbiguousSendError(c.err); got != c.want {
+ t.Errorf("%s: got %v want %v", c.name, got, c.want)
+ }
+ }
+}
+
+func TestRetrySendAmbiguousWithoutCheckerIsNotRetried(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ _, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, resetErr()
+ }, nil)
+ if err == nil || calls != 1 {
+ t.Fatalf("calls=%d err=%v", calls, err)
+ }
+}
+
+func TestRetrySendAmbiguousButDeliveredIsNotResent(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ msg, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, resetErr()
+ }, func(wait bool) (*discordgo.Message, bool) {
+ return &discordgo.Message{ID: "echo"}, true
+ })
+ if err != nil || msg == nil || msg.ID != "echo" || calls != 1 {
+ t.Fatalf("calls=%d msg=%v err=%v", calls, msg, err)
+ }
+}
+
+func TestRetrySendAmbiguousNotDeliveredIsResent(t *testing.T) {
+ slept := withNoSleep(t)
+ calls := 0
+ var waits []bool
+ msg, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ if calls == 1 {
+ return nil, resetErr()
+ }
+ return &discordgo.Message{ID: "2"}, nil
+ }, func(wait bool) (*discordgo.Message, bool) {
+ waits = append(waits, wait)
+ return nil, false
+ })
+ if err != nil || msg.ID != "2" || calls != 2 || len(*slept) != 1 {
+ t.Fatalf("calls=%d msg=%v err=%v slept=%v", calls, msg, err, *slept)
+ }
+ if len(waits) != 2 || !waits[0] || waits[1] {
+ t.Fatalf("waits=%v, want [true false]", waits)
+ }
+}
+
+func TestRetrySendLateEchoDuringBackoffStopsResend(t *testing.T) {
+ withNoSleep(t)
+ calls := 0
+ checks := 0
+ msg, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
+ calls++
+ return nil, resetErr()
+ }, func(wait bool) (*discordgo.Message, bool) {
+ checks++
+ if checks == 2 { // the echo arrives during the backoff
+ return &discordgo.Message{ID: "late"}, true
+ }
+ return nil, false
+ })
+ if err != nil || msg.ID != "late" || calls != 1 {
+ t.Fatalf("calls=%d msg=%v err=%v", calls, msg, err)
+ }
+}
+
+func snowflakeAt(ts time.Time) string {
+ ms := ts.UnixMilli() - 1420070400000
+ return strconv.FormatInt(ms<<22, 10)
+}
+
+func TestEchoWaiterMatchesWebhookPost(t *testing.T) {
+ oldWait := lotusEchoWait
+ lotusEchoWait = 200 * time.Millisecond
+ t.Cleanup(func() { lotusEchoWait = oldWait })
+
+ start := time.Now()
+ check, done := lotusExpectEcho("chan1", lotusWebhookEchoMatcher("hook1", "hi", 1, start))
+ defer done()
+
+ if _, ok := check(false); ok {
+ t.Fatal("matched before any message")
+ }
+ // Wrong webhook, wrong content, wrong channel and an old message are ignored.
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start), ChannelID: "chan1", WebhookID: "other", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start), ChannelID: "chan1", WebhookID: "hook1", Content: "nope", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start), ChannelID: "chan2", WebhookID: "hook1", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ lotusNoteGatewayMessage(&discordgo.Message{ID: snowflakeAt(start.Add(-time.Minute)), ChannelID: "chan1", WebhookID: "hook1", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ if _, ok := check(true); ok {
+ t.Fatal("matched a non-matching message")
+ }
+
+ want := snowflakeAt(start.Add(time.Second))
+ go func() {
+ time.Sleep(20 * time.Millisecond)
+ lotusNoteGatewayMessage(&discordgo.Message{ID: want, ChannelID: "chan1", WebhookID: "hook1", Content: "hi", Attachments: make([]*discordgo.MessageAttachment, 1)})
+ }()
+ got, ok := check(true)
+ if !ok || got.ID != want {
+ t.Fatalf("got %v ok=%v", got, ok)
+ }
+}
+
+func TestEchoWaiterDoneUnregisters(t *testing.T) {
+ _, done := lotusExpectEcho("chanX", func(*discordgo.Message) bool { return true })
+ done()
+ lotusEchoMu.Lock()
+ defer lotusEchoMu.Unlock()
+ if _, ok := lotusEchoWaiters["chanX"]; ok {
+ t.Fatal("waiter still registered after done")
+ }
+}
diff --git a/lotus_retry.go b/lotus_retry.go
index 972b612..0c5e147 100644
--- a/lotus_retry.go
+++ b/lotus_retry.go
@@ -74,17 +74,69 @@ func lotusRewindFiles(files []*discordgo.File) {
}
}
+// isAmbiguousSendError reports whether err happened after the request may
+// already have reached Discord: the connection broke or timed out while
+// sending or waiting for the reply, or Discord answered with a 5xx.
+func isAmbiguousSendError(err error) bool {
+ if err == nil || isPreSendNetworkError(err) {
+ return false
+ }
+ var restErr *discordgo.RESTError
+ if errors.As(err, &restErr) {
+ return restErr.Response != nil && restErr.Response.StatusCode >= 500
+ }
+ if errors.Is(err, io.EOF) || errors.Is(err, io.ErrUnexpectedEOF) {
+ return true
+ }
+ var netErr net.Error
+ if errors.As(err, &netErr) {
+ return true
+ }
+ msg := err.Error()
+ return strings.Contains(msg, "connection reset") ||
+ strings.Contains(msg, "broken pipe") ||
+ strings.Contains(msg, "Client.Timeout exceeded") ||
+ strings.Contains(msg, "server sent GOAWAY") ||
+ strings.Contains(msg, "EOF")
+}
+
// lotusRetrySend calls send until it succeeds, fails with an error that is not
-// a pre-send network error, or the retry budget runs out.
-func lotusRetrySend[T any](log zerolog.Logger, files []*discordgo.File, send func() (T, error)) (T, error) {
+// retryable, or the retry budget runs out.
+//
+// Pre-send network errors are always retried. Ambiguous errors (see
+// isAmbiguousSendError) are retried only when checkDelivered is set: it is
+// called first (wait=true lets it wait briefly for evidence) and, if it finds
+// that the message did reach Discord, its result is returned as a success
+// instead of sending again. It is called again with wait=false just before
+// each resend.
+func lotusRetrySend[T any](log zerolog.Logger, files []*discordgo.File, send func() (T, error), checkDelivered func(wait bool) (T, bool)) (T, error) {
res, err := send()
- for attempt := 0; attempt < len(lotusSendRetryDelays) && isPreSendNetworkError(err); attempt++ {
+ for attempt := 0; attempt < len(lotusSendRetryDelays); attempt++ {
+ switch {
+ case isPreSendNetworkError(err):
+ case checkDelivered != nil && isAmbiguousSendError(err):
+ if delivered, ok := checkDelivered(true); ok {
+ log.Info().Err(err).Int("attempts", attempt+1).
+ Msg("[lotus] Discord send errored but the message was delivered, not resending")
+ return delivered, nil
+ }
+ default:
+ return res, err
+ }
delay := lotusSendRetryDelays[attempt]
log.Warn().Err(err).
Int("attempt", attempt+1).
Dur("retry_in", delay).
- Msg("[lotus] Discord send failed before reaching Discord, retrying")
+ Msg("[lotus] Discord send failed and the message was not delivered, retrying")
lotusSleep(delay)
+ if checkDelivered != nil {
+ // A late echo of an earlier attempt can land during the backoff.
+ if delivered, ok := checkDelivered(false); ok {
+ log.Info().Int("attempts", attempt+1).
+ Msg("[lotus] earlier Discord send turned out to be delivered, not resending")
+ return delivered, nil
+ }
+ }
lotusRewindFiles(files)
res, err = send()
if err == nil {
diff --git a/lotus_retry_test.go b/lotus_retry_test.go
index efa262e..e35585b 100644
--- a/lotus_retry_test.go
+++ b/lotus_retry_test.go
@@ -68,7 +68,7 @@ func TestRetrySendRecoversAndRewindsFiles(t *testing.T) {
return nil, urlErr(tlsTimeout{})
}
return &discordgo.Message{ID: "42"}, nil
- })
+ }, nil)
if err != nil || msg == nil || msg.ID != "42" {
t.Fatalf("got msg=%v err=%v", msg, err)
}
@@ -86,7 +86,7 @@ func TestRetrySendDoesNotRetryAfterSend(t *testing.T) {
_, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
calls++
return nil, urlErr(&net.OpError{Op: "read", Net: "tcp", Err: errors.New("connection reset by peer")})
- })
+ }, nil)
if err == nil || calls != 1 {
t.Fatalf("calls=%d err=%v, want exactly one attempt", calls, err)
}
@@ -98,7 +98,7 @@ func TestRetrySendGivesUpAfterBudget(t *testing.T) {
_, err := lotusRetrySend(zerolog.Nop(), nil, func() (*discordgo.Message, error) {
calls++
return nil, urlErr(tlsTimeout{})
- })
+ }, nil)
if err == nil {
t.Fatal("expected error after budget")
}
diff --git a/portal.go b/portal.go
index cc6da0d..57ba7f9 100644
--- a/portal.go
+++ b/portal.go
@@ -1748,9 +1748,18 @@ func (portal *Portal) handleMatrixMessage(sender *User, evt *event.Event) {
if !isWebhookSend {
msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
return sess.ChannelMessageSendComplex(channelID, &sendReq, portal.RefererOptIfUser(sess, threadID)...)
- })
+ }, nil)
} else {
username, avatarURL := portal.getRelayUserMeta(sender)
+ // [lotus] watch for the gateway echo so an ambiguous failure can be
+ // checked instead of resent blindly (see lotus_echo.go)
+ echoChannel := channelID
+ if threadID != "" {
+ echoChannel = threadID
+ }
+ checkEcho, echoDone := lotusExpectEcho(echoChannel,
+ lotusWebhookEchoMatcher(portal.RelayWebhookID, sendReq.Content, len(sendReq.Files), time.Now()))
+ defer echoDone()
msg, err = lotusRetrySend(sendLog, sendReq.Files, func() (*discordgo.Message, error) {
return relayClient.WebhookThreadExecute(portal.RelayWebhookID, portal.RelayWebhookSecret, true, threadID, &discordgo.WebhookParams{
Content: sendReq.Content,
@@ -1761,7 +1770,7 @@ func (portal *Portal) handleMatrixMessage(sender *User, evt *event.Event) {
Embeds: sendReq.Embeds,
AllowedMentions: sendReq.AllowedMentions,
})
- })
+ }, checkEcho)
}
sender.handlePossible40002(err)
go portal.sendMessageMetrics(evt, err, "Error sending")
diff --git a/user.go b/user.go
index 8c0d6ec..16cd8c8 100644
--- a/user.go
+++ b/user.go
@@ -704,6 +704,7 @@ func (user *User) eventHandler(rawEvt any) {
case *discordgo.RelationshipUpdate:
user.relationshipUpdateHandler(evt)
case *discordgo.MessageCreate:
+ lotusNoteGatewayMessage(evt.Message) // [lotus] see lotus_echo.go
user.pushPortalMessage(evt, "message create", evt.ChannelID, evt.GuildID)
case *discordgo.MessageDelete:
user.pushPortalMessage(evt, "message delete", evt.ChannelID, evt.GuildID)
+123
View File
@@ -0,0 +1,123 @@
import asyncio
import unittest
import lotus_presence as lp
class MapPresence(unittest.TestCase):
def test_statuses(self):
self.assertEqual(lp.map_presence("online", []), ("online", ""))
self.assertEqual(lp.map_presence("idle", []), ("unavailable", ""))
self.assertEqual(lp.map_presence("dnd", []), (lp.BUSY, ""))
self.assertEqual(lp.map_presence("offline", []), ("offline", ""))
self.assertEqual(lp.map_presence("invisible", None), ("offline", ""))
def test_offline_never_carries_activity(self):
game = [{"type": 0, "name": "Valorant"}]
self.assertEqual(lp.map_presence("offline", game), ("offline", ""))
def test_activities(self):
for activity, expected in (
({"type": 0, "name": "Valorant"}, "Playing Valorant"),
({"type": 1, "name": "Twitch", "details": "Ranked"}, "Streaming Ranked"),
({"type": 2, "name": "Spotify", "details": "Song", "state": "Band"},
"Listening to Song by Band"),
({"type": 2, "name": "Spotify"}, "Listening to Spotify"),
({"type": 3, "name": "YouTube"}, "Watching YouTube"),
({"type": 5, "name": "Arena"}, "Competing in Arena"),
):
self.assertEqual(lp.map_presence("online", [activity])[1], expected)
def test_custom_status_comes_first_with_one_activity(self):
activities = [
{"type": 0, "name": "Valorant"},
{"type": 0, "name": "Second game"},
{"type": 4, "name": "Custom Status", "state": "afk", "emoji": {"name": "🍕"}},
]
self.assertEqual(lp.map_presence("online", activities)[1], "🍕 afk · Playing Valorant")
def test_custom_server_emoji_is_dropped(self):
custom = {"type": 4, "state": "hi", "emoji": {"name": "partyblob", "id": "123"}}
self.assertEqual(lp.map_presence("online", [custom])[1], "hi")
def test_long_status_is_truncated(self):
_, message = lp.map_presence("online", [{"type": 0, "name": "x" * 500}])
self.assertLessEqual(len(message), lp.MAX_STATUS)
self.assertTrue(message.endswith("…"))
class BridgeConfig(unittest.TestCase):
def test_reads_the_database_uri(self):
env = lp.bridge_db_env(
"appservice:\n database:\n type: postgres\n"
' uri: "postgres://u:p%40ss@10.0.0.1:5433/db?sslmode=disable"\n'
)
self.assertEqual(env["PGHOST"], "10.0.0.1")
self.assertEqual(env["PGPORT"], "5433")
self.assertEqual(env["PGPASSWORD"], "p@ss")
self.assertEqual(env["PGDATABASE"], "db")
self.assertEqual(env["PGSSLMODE"], "disable")
def test_missing_uri_is_an_error(self):
with self.assertRaises(RuntimeError):
lp.bridge_db_env("uri: sqlite:///x")
class FakeGhosts(lp.GhostPresence):
def __init__(self, reject=()):
super().__init__(None, "http://hs", "t", "s", clock=lambda: self.now)
self.now = 0
self.sent = []
self.reject = set(reject)
async def _put(self, discord_id, presence, message):
self.sent.append((discord_id, presence, message))
if discord_id in self.reject:
self.unknown_until[discord_id] = self.now + lp.UNKNOWN_RETRY_S
class Ghosts(unittest.TestCase):
def test_only_changes_are_sent(self):
g = FakeGhosts()
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.update("1", "online", "Playing X"))
self.assertEqual(len(g.sent), 2)
def test_offline_is_sent_once_and_only_after_being_live(self):
g = FakeGhosts()
asyncio.run(g.update("1", "offline", ""))
self.assertEqual(g.sent, [])
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.update("1", "offline", ""))
asyncio.run(g.update("1", "offline", ""))
self.assertEqual(g.sent[-1], ("1", "offline", ""))
self.assertEqual(len(g.sent), 2)
def test_resync_turns_off_whoever_left(self):
g = FakeGhosts()
asyncio.run(g.update("1", "online", ""))
asyncio.run(g.resync({"2": ("online", "")}))
self.assertIn(("1", "offline", ""), g.sent)
self.assertEqual(set(g.live), {"2"})
def test_refresh_resends_live_states(self):
g = FakeGhosts()
asyncio.run(g.update("1", "unavailable", "x"))
g.sent.clear()
asyncio.run(g.refresh())
self.assertEqual(g.sent, [("1", "unavailable", "x")])
def test_missing_ghost_is_retried_later_not_every_refresh(self):
g = FakeGhosts(reject={"1"})
asyncio.run(g.update("1", "online", ""))
g.sent.clear()
asyncio.run(g.refresh())
self.assertEqual(g.sent, [])
g.now += lp.UNKNOWN_RETRY_S + 1
asyncio.run(g.refresh())
self.assertEqual(g.sent, [("1", "online", "")])
if __name__ == "__main__":
unittest.main()
-37
View File
@@ -1,37 +0,0 @@
var ev = data.eventType || 'Unknown';
if (ev === 'Test') {
result = { version: 'v2', plain: '🧪 Lidarr: Connection test successful', msgtype: 'm.notice' };
} else {
var artist = (data.artist && data.artist.name) || 'Unknown Artist';
var albums = data.albums || (data.album ? [data.album] : []);
var albumStr = albums.map(function(a){ return a.title || ''; }).filter(Boolean).join(', ');
var quality = (data.release && data.release.quality) || (data.trackFiles && data.trackFiles[0] && data.trackFiles[0].quality) || '';
var releaseGroup = (data.release && data.release.releaseGroup) || '';
var client = data.downloadClient || '';
var upgrade = data.isUpgrade ? ' \u2191upgrade' : '';
var healthMsg = data.message || '';
var healthWiki = data.wikiUrl || '';
var prevVer = data.previousVersion || '';
var newVer = data.newVersion || '';
var emojiMap = { 'Grab':'📥','Download':'✅','Rename':'✏️','ArtistAdd':'➕','ArtistDelete':'🗑️','AlbumAdd':'➕','AlbumDelete':'🗑️','TrackFileDelete':'🗑️','HealthIssue':'⚠️','HealthRestored':'💚','ApplicationUpdate':'🔄' };
var emoji = emojiMap[ev] || '🎵';
var plain, html;
if (ev === 'HealthIssue' || ev === 'HealthRestored') {
plain = emoji + ' Lidarr ' + ev + ': ' + healthMsg + (healthWiki ? '\n' + healthWiki : '');
html = '<b>' + emoji + ' Lidarr ' + ev + '</b>: ' + healthMsg + (healthWiki ? '<br><a href="' + healthWiki + '">Wiki</a>' : '');
} else if (ev === 'ApplicationUpdate') {
plain = emoji + ' Lidarr updated: ' + prevVer + ' \u2192 ' + newVer;
html = '<b>' + emoji + ' Lidarr updated</b>: ' + prevVer + ' \u2192 ' + newVer;
} else if (ev === 'ArtistAdd' || ev === 'ArtistDelete') {
plain = emoji + ' Lidarr ' + ev + ': ' + artist;
html = '<b>' + emoji + ' Lidarr ' + ev + '</b>: ' + artist;
} else {
var albumPart = albumStr ? ' \u2014 ' + albumStr : '';
var qualPart = quality ? ' [' + quality + ']' : '';
var groupPart = releaseGroup ? ' {' + releaseGroup + '}' : '';
var clientPart = client ? ' via ' + client : '';
plain = emoji + ' Lidarr ' + ev + ': ' + artist + albumPart + qualPart + groupPart + upgrade + clientPart;
html = '<b>' + emoji + ' Lidarr ' + ev + '</b>: ' + artist + (albumStr ? ' \u2014 <i>' + albumStr + '</i>' : '') + qualPart + groupPart + upgrade + clientPart;
}
result = { version: 'v2', plain: plain, html: html, msgtype: 'm.notice' };
}
+1 -1
View File
@@ -1616,7 +1616,7 @@
"cells": [
{
"mark": "part",
"note": "OS notifications while the app is open; no background push yet"
"note": "OS notifications while the web or desktop app is open (the desktop app keeps running in the tray); background web push is not planned, so use Element X or FluffyChat on phones"
},
{
"mark": "part",
+1 -1
View File
@@ -1342,7 +1342,7 @@ Client comparison last reviewed <time datetime="2026-09-26">September 26, 2026</
</tr>
<tr>
<td>Push notifications</td>
<td class="ours"><span class="part">~</span><small>OS notifications while the app is open; no background push yet</small></td>
<td class="ours"><span class="part">~</span><small>OS notifications while the web or desktop app is open (the desktop app keeps running in the tray); background web push is not planned, so use Element X or FluffyChat on phones</small></td>
<td><span class="part">~</span><small>only while the tab/app is open; optional email</small></td>
<td><span class="yes">✓</span></td>
<td><span class="yes">✓</span></td>
+5 -5
View File
@@ -27,11 +27,11 @@ Matrix bot and server improvements for the Lotus Guild homeserver (`matrix.lotus
| Service | Host | IP | LXC | Notes |
|---------|------|-----|-----|-------|
| Synapse | micro1 | 10.10.10.29 | 151 | Homeserver + coturn + LiveKit |
| PostgreSQL 17 | micro1 | 10.10.10.44 | 109 | Synapse database backend |
| NPM | large1 | 10.10.10.27 | 139 | Reverse proxy + landing page |
| Authelia | compute-storage-01 | 10.10.10.36 | 167 | SSO/OIDC provider |
| LLDAP | large1 | 10.10.10.39 | 147 | LDAP user directory |
| Synapse | HA (floats) | 10.10.10.29 | 151 | Homeserver + coturn + LiveKit |
| PostgreSQL 17 | HA (floats) | 10.10.10.44 | 109 | Synapse database backend |
| NPM | HA (floats) | 10.10.10.27 | 139 | Reverse proxy + landing page |
| Authelia | HA (floats) | 10.10.10.36 | 167 | SSO/OIDC provider |
| LLDAP | HA (floats) | 10.10.10.39 | 147 | LDAP user directory |
**Key paths on Synapse LXC (10.10.10.29):**
- Synapse config: `/etc/matrix-synapse/homeserver.yaml`
+38 -17
View File
@@ -120,8 +120,19 @@ def is_elevated(client: AsyncClient, room_id: str, user_id: str, min_level: int
# ==================== COMMANDS ====================
@command("help", "Show all available commands")
@command("help", "Show all commands, or details for one (!help <command>)")
async def cmd_help(client: AsyncClient, room_id: str, sender: str, args: str):
# Compact on purpose: the Discord bridge drops messages over 2000 characters.
query = args.strip().lstrip(BOT_PREFIX).lower()
if query:
entry = COMMANDS.get(query)
if entry is None:
await send_text(client, room_id, f"Unknown command: {BOT_PREFIX}{query}. Try {BOT_PREFIX}help for the list.")
return
_, desc = entry
await send_html(client, room_id, f"{BOT_PREFIX}{query} — {desc}", f"<strong>{BOT_PREFIX}{query}</strong> — {desc}")
return
elevated = is_elevated(client, room_id, sender)
categories = [
("🤖 AI / Fun", ["ask", "fortune", "8ball", "roast", "story", "debate"]),
@@ -148,18 +159,17 @@ async def cmd_help(client: AsyncClient, room_id: str, sender: str, args: str):
"setpl", "kick", "purge", "members", "whois", "announce", "syncspace",
]))
plain_lines = ["LotusBot Commands"]
html_parts = ['<font color="#a855f7"><strong>🌸 LotusBot — Commands</strong></font>']
plain_lines = [f"LotusBot commands ({BOT_PREFIX}help <command> for details)"]
html_parts = [
'<font color="#a855f7"><strong>🌸 LotusBot — Commands</strong></font>'
f"<br><sup><em>{BOT_PREFIX}help &lt;command&gt; for details</em></sup>"
]
for cat_name, cmd_names in categories:
plain_lines.append(f"\n{cat_name}")
html_parts.append(f"<br><strong>{cat_name}</strong><ul>")
for name in cmd_names:
if name in COMMANDS:
_, desc = COMMANDS[name]
plain_lines.append(f" {BOT_PREFIX}{name} — {desc}")
html_parts.append(f"<li><strong>{BOT_PREFIX}{name}</strong> — {desc}</li>")
html_parts.append("</ul>")
names = [f"{BOT_PREFIX}{name}" for name in cmd_names if name in COMMANDS]
if not names:
continue
plain_lines.append(f"{cat_name}: " + " ".join(names))
html_parts.append(f"<br><strong>{cat_name}:</strong> " + " ".join(f"<code>{n}</code>" for n in names))
await send_html(client, room_id, "\n".join(plain_lines), "".join(html_parts))
@@ -1313,6 +1323,11 @@ async def cmd_minecraft(client: AsyncClient, room_id: str, sender: str, args: st
await send_text(client, room_id, "Minecraft server is not configured.")
return
remaining = check_cooldown(sender, "minecraft", 60)
if remaining:
await send_text(client, room_id, f"Command on cooldown. Try again in {remaining}s.")
return
await send_text(client, room_id, f"Whitelisting {username}...")
try:
@@ -1327,7 +1342,11 @@ async def cmd_minecraft(client: AsyncClient, room_id: str, sender: str, args: st
loop = asyncio.get_running_loop()
response = await asyncio.wait_for(loop.run_in_executor(None, _rcon), timeout=RCON_TIMEOUT)
logger.info(f"RCON response: {response}")
logger.info(f"Minecraft whitelist: sender={sender} room={room_id} username={username} response={response!r}")
if "does not exist" in response.lower() or "unknown player" in response.lower():
await send_text(client, room_id, f"No Minecraft account named {username} exists. Check the spelling (Java Edition username).")
return
already = "already whitelisted" in response.lower()
status_line = (
@@ -1336,16 +1355,18 @@ async def cmd_minecraft(client: AsyncClient, room_id: str, sender: str, args: st
else f"✅ <strong>{username}</strong> is whitelisted and ready to join."
)
plain = (
f"✿ Minecraft — Lotus SMP ✿\n"
f"✿ Lotus Minecraft Server ✿\n"
f"{'Already whitelisted' if already else 'Whitelisted'}: {username}\n"
f"Server: minecraft.lotusguild.org"
f"Server: minecraft.lotusguild.org\n"
f"Modpack: All the Mods 10 (CurseForge), version 8.6"
)
html = (
f'<font color="#980000"><strong>✿ Minecraft — Lotus SMP ✿</strong></font><br>'
f'<font color="#980000"><strong>✿ Lotus Minecraft Server ✿</strong></font><br>'
f"<blockquote>"
f"{status_line}<br>"
f'<strong>Server:</strong> <code>minecraft.lotusguild.org</code><br>'
f"<sup><em>fresh world · hard mode · no takebacks</em></sup>"
f'<strong>Modpack:</strong> All the Mods 10 (CurseForge), version <code>8.6</code><br>'
f"<sup><em>install the same pack version or the server won't let you in</em></sup>"
f"</blockquote>"
)
await send_html(client, room_id, plain, html)
+177
View File
@@ -0,0 +1,177 @@
#!/usr/bin/env python3
"""Player count -> status message on the Lotus Minecraft bot.
Pings the Minecraft server the way the in-game server list does (no RCON, no
password) and sets the result as the Matrix status of the account the chat
bridge mod logs in as, for example "3 online: Daisybell, Honnehh, Wolf".
- Credentials: the homeserver and token come from the mod's own config, so there
is no extra account or secret. systemd hands this service a private copy.
- The mod syncs with set_presence=offline, so it doesn't keep the account online,
and Synapse drops an unrefreshed user to offline. So state is re-sent on a timer.
- This runs beside the game, not in it, so it can report the server being down.
One failed ping is ignored: a busy modded server can miss one.
"""
import json
import logging
import os
import re
import socket
import struct
import time
import tomllib
import urllib.error
import urllib.parse
import urllib.request
log = logging.getLogger("lotus-mc-status")
INTERVAL = 20 # seconds; Synapse's default limit for this endpoint is one per 10 s
PING_TIMEOUT = 8
MAX_LEN = 64 # what Cinny's own status field allows
FAILS_BEFORE_DOWN = 2
OFFLINE = "Server offline"
PLAYER_NAME = re.compile(r"[A-Za-z0-9_]{1,16}")
def varint(n: int) -> bytes:
out = b""
while True:
byte = n & 0x7F
n >>= 7
out += bytes([byte | (0x80 if n else 0)])
if not n:
return out
def read_varint(read) -> int:
result = 0
for shift in range(0, 35, 7):
byte = read(1)[0]
result |= (byte & 0x7F) << shift
if not byte & 0x80:
return result
raise ValueError("varint too long")
def parse_status(packet: bytes) -> tuple[int, list[str]]:
"""Return (players online, names) from a status response packet."""
pos = [0]
def read(n: int) -> bytes:
chunk = packet[pos[0] : pos[0] + n]
if len(chunk) < n:
raise ValueError("truncated status packet")
pos[0] += n
return chunk
if read_varint(read) != 0:
raise ValueError("not a status response")
players = json.loads(read(read_varint(read)))["players"]
return int(players["online"]), [str(p.get("name", "")) for p in players.get("sample") or []]
def ping(host: str, port: int) -> tuple[int, list[str]]:
with socket.create_connection((host, port), timeout=PING_TIMEOUT) as sock:
addr = host.encode()
hello = b"\0" + varint(767) + varint(len(addr)) + addr + struct.pack(">H", port) + varint(1)
sock.sendall(varint(len(hello)) + hello + b"\x01\0")
def read(n: int) -> bytes:
buf = b""
while len(buf) < n:
chunk = sock.recv(n - len(buf))
if not chunk:
raise ConnectionError("server closed the connection")
buf += chunk
return buf
return parse_status(read(read_varint(read)))
def format_status(online: int, names: list[str], max_len: int = MAX_LEN) -> str:
"""Count plus as many names as fit. Names are sorted so a re-ordered list
doesn't change the status, and anything that isn't a real player name
(servers can send placeholders) is dropped and counted under "more"."""
if online <= 0:
return "Nobody online"
head = f"{online} online"
real = sorted({n for n in names if PLAYER_NAME.fullmatch(n)}, key=str.casefold)[:online]
shown: list[str] = []
for name in real:
rest = online - len(shown) - 1
candidate = f"{head}: {', '.join([*shown, name])}" + (f" +{rest} more" if rest else "")
if len(candidate) > max_len:
break
shown.append(name)
if not shown:
return head
rest = online - len(shown)
return f"{head}: {', '.join(shown)}" + (f" +{rest} more" if rest else "")
def read_config(path: str) -> tuple[str, str]:
with open(path, "rb") as f:
matrix = tomllib.load(f)["matrix"]
return matrix["homeserver"].rstrip("/"), matrix["accessToken"]
def api(method: str, url: str, token: str, body: dict | None = None) -> dict:
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(url, data=data, method=method, headers={
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
"User-Agent": "lotus-mc-status",
})
with urllib.request.urlopen(req, timeout=15) as resp:
return json.load(resp)
def main() -> None:
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
creds = os.environ.get("CREDENTIALS_DIRECTORY")
path = os.path.join(creds, "bridge-config") if creds else os.environ["LOTUS_MC_CONFIG"]
host = os.environ["LOTUS_MC_HOST"]
port = int(os.environ.get("LOTUS_MC_PORT", "25565"))
homeserver, token = read_config(path)
user_id = api("GET", f"{homeserver}/_matrix/client/v3/account/whoami", token)["user_id"]
url = f"{homeserver}/_matrix/client/v3/presence/{urllib.parse.quote(user_id, safe='')}/status"
log.info("watching %s:%d for %s", host, port, user_id)
fails = 0
body = None
last_error = None
last_count = None
while True:
try:
online, names = ping(host, port)
fails = 0
body = {"presence": "online", "status_msg": format_status(online, names)}
if online != last_count:
log.info("%d online", online)
last_count = online
except (OSError, ValueError, KeyError, TypeError) as exc:
fails += 1
if fails == FAILS_BEFORE_DOWN:
log.warning("server not answering: %s", type(exc).__name__)
last_count = None
if fails >= FAILS_BEFORE_DOWN:
body = {"presence": "unavailable", "status_msg": OFFLINE}
if body:
try:
api("PUT", url, token, body)
if last_error:
log.info("Matrix reachable again")
last_error = None
except (OSError, ValueError) as exc:
# HTTPError and URLError are OSErrors. Their text has no token in it.
error = f"HTTP {exc.code}" if isinstance(exc, urllib.error.HTTPError) else type(exc).__name__
if error != last_error:
log.warning("could not set status: %s", error)
last_error = error
time.sleep(INTERVAL)
if __name__ == "__main__":
main()
+77
View File
@@ -0,0 +1,77 @@
import json
import unittest
import lotus_mc_status as s
def packet(players: dict) -> bytes:
body = json.dumps({"version": {"name": "1.21.1"}, "players": players}).encode()
return s.varint(0) + s.varint(len(body)) + body
class FormatStatus(unittest.TestCase):
def test_nobody(self):
self.assertEqual(s.format_status(0, []), "Nobody online")
def test_one(self):
self.assertEqual(s.format_status(1, ["Honnehh"]), "1 online: Honnehh")
def test_sorted_regardless_of_server_order(self):
self.assertEqual(s.format_status(3, ["b_x", "C", "a"]), "3 online: a, b_x, C")
def test_long_names_collapse_and_stay_within_the_limit(self):
names = [f"Player{i}_abcdefghi"[:16] for i in range(10)]
got = s.format_status(10, names)
self.assertLessEqual(len(got), s.MAX_LEN)
self.assertRegex(got, r"^10 online: .+ \+\d+ more$")
shown = got.split(": ")[1].split(" +")[0].split(", ")
self.assertEqual(int(got.split("+")[1].split()[0]), 10 - len(shown))
def test_exact_fit_has_no_more_suffix(self):
got = s.format_status(1, ["A" * 16], max_len=len("1 online: ") + 16)
self.assertEqual(got, "1 online: " + "A" * 16)
def test_more_players_than_the_sample(self):
self.assertEqual(s.format_status(14, ["a", "b"]), "14 online: a, b +12 more")
def test_placeholders_are_not_shown_as_players(self):
self.assertEqual(s.format_status(2, ["Anonymous Player", "a"]), "2 online: a +1 more")
self.assertEqual(s.format_status(1, ["§cinjected name"]), "1 online")
def test_count_only_when_nothing_fits(self):
self.assertEqual(s.format_status(2, ["abcdefgh"], max_len=9), "2 online")
def test_duplicates_and_stale_extras_do_not_overcount(self):
self.assertEqual(s.format_status(1, ["a", "a", "b"]), "1 online: a")
class ParseStatus(unittest.TestCase):
def test_with_sample(self):
p = packet({"max": 20, "online": 2, "sample": [{"name": "a", "id": "x"}, {"name": "b", "id": "y"}]})
self.assertEqual(s.parse_status(p), (2, ["a", "b"]))
def test_without_sample(self):
self.assertEqual(s.parse_status(packet({"max": 20, "online": 0})), (0, []))
def test_rejects_other_packets_and_truncation(self):
with self.assertRaises(ValueError):
s.parse_status(s.varint(1) + b"xx")
with self.assertRaises(ValueError):
s.parse_status(packet({"max": 20, "online": 0})[:-3])
def test_varint_round_trip(self):
for n in (0, 1, 127, 128, 300, 6087, 2097151):
data = iter([s.varint(n)])
buf = next(data)
pos = [0]
def read(k, buf=buf, pos=pos):
out = buf[pos[0] : pos[0] + k]
pos[0] += k
return out
self.assertEqual(s.read_varint(read), n)
if __name__ == "__main__":
unittest.main()
+37
View File
@@ -0,0 +1,37 @@
#!/bin/bash
# Install or update the now-playing service (cinny #491) on LXC 151.
# Run with this repo's main branch pulled to /opt/matrix-config:
# bash /opt/matrix-config/now-playing/install.sh
# First install: put the Navidrome login in /etc/lotus-now-playing/env first
# (NAVIDROME_USER=lotus-nowplaying / NAVIDROME_PASSWORD=...), mode 640 root:lotus-now-playing.
# Optional /etc/lotus-now-playing/users.json maps Matrix IDs whose SSO link has no
# email to their SSO username, e.g. {"@bigmanjeff:matrix.lotusguild.org": "jeff"}.
# Neither file is kept in this (public) repo. Safe to re-run.
set -euo pipefail
REPO=/opt/matrix-config
CONF=/etc/lotus-now-playing
id lotus-now-playing >/dev/null 2>&1 ||
useradd --system --no-create-home --home-dir /var/lib/lotus-now-playing --shell /usr/sbin/nologin lotus-now-playing
install -d -m750 -o root -g lotus-now-playing "$CONF"
install -D -m644 "$REPO/now-playing/lotus_now_playing.py" /opt/lotus-now-playing/lotus_now_playing.py
if [ ! -f "$CONF/env" ]; then
echo "missing $CONF/env (NAVIDROME_USER / NAVIDROME_PASSWORD); create it and re-run" >&2
exit 1
fi
chown root:lotus-now-playing "$CONF/env"
chmod 640 "$CONF/env"
if [ -f "$CONF/users.json" ]; then
chown root:lotus-now-playing "$CONF/users.json"
chmod 640 "$CONF/users.json"
fi
install -m644 "$REPO/systemd/lotus-now-playing.service" /etc/systemd/system/lotus-now-playing.service
systemctl daemon-reload
systemctl enable lotus-now-playing >/dev/null
systemctl restart lotus-now-playing
sleep 2
systemctl --no-pager --lines=5 status lotus-now-playing
curl -fsS http://127.0.0.1:8072/healthz && echo " (healthy)"
+493
View File
@@ -0,0 +1,493 @@
#!/usr/bin/env python3
"""Now playing (Navidrome) -> Lotus Chat "Listening to" cards. [cinny #491]
Navidrome is the scrobbling master: every play, whether from Dropped Needle or
any Subsonic app, shows up in its `getNowPlaying` under the listener's own
username, which is their SSO (LLDAP) username. This service polls that list and
serves it to Lotus Chat for users who opted in, keyed by Matrix user ID.
- Who is asking: Lotus Chat sends a Matrix OpenID token (Authorization: Bearer).
It is checked against Synapse's /_matrix/federation/v1/openid/userinfo, so
only signed-in users of this homeserver can read anything.
- Matrix user -> SSO username: an override file first (accounts whose SSO link
carries no email), then the Synapse admin API (email <name>@<domain>, or a
legacy external ID that is the plain username).
- Opt-in: off by default, stored here in STATE_DIR/state.json.
- Cover art is proxied only for covers that an opted-in user is playing right
now, so the endpoint can't be used to browse the library.
Track titles, artists and tokens are never logged.
"""
import hashlib
import json
import logging
import os
import re
import secrets
import tempfile
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
log = logging.getLogger("lotus-now-playing")
PREFIX = "/now-playing/v1"
MAX_IDS = 50
SUBSONIC_CLIENT = "lotus-now-playing"
SUBSONIC_VERSION = "1.16.1"
# Navidrome drops a session a while after the last report, so an entry older
# than this (no position updates, not paused) is treated as stale.
STALE_MINUTES = 30
COVER_SIZES = (64, 128, 256, 512)
UUID_RE = re.compile(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}")
# --- pure helpers (unit tested) ----------------------------------------------
def parse_now_playing(payload, ignore_users=()):
"""getNowPlaying JSON -> {username: entry}. Keeps the most recent session per user."""
resp = (payload or {}).get("subsonic-response") or {}
if resp.get("status") != "ok":
raise ValueError((resp.get("error") or {}).get("message") or "getNowPlaying failed")
entries = (resp.get("nowPlaying") or {}).get("entry") or []
if isinstance(entries, dict):
entries = [entries]
out = {}
for e in entries:
user = (e.get("username") or "").strip()
if not user or user in ignore_users:
continue
state = (e.get("state") or "playing").lower()
if state not in ("playing", "paused"):
continue
minutes = int(e.get("minutesAgo") or 0)
if state == "playing" and minutes > STALE_MINUTES and not e.get("positionMs"):
continue
entry = {
"title": e.get("title") or "",
"artist": e.get("displayArtist") or e.get("artist") or "",
"album": e.get("album") or "",
"durationS": int(e.get("duration") or 0),
"positionMs": int(e.get("positionMs") or 0),
"state": state,
"player": e.get("playerName") or "",
"coverArt": e.get("coverArt") or "",
"minutesAgo": minutes,
}
if not entry["title"]:
continue
prev = out.get(user)
if prev is None or minutes < prev["minutesAgo"]:
out[user] = entry
return out
def username_from_admin_user(user, email_domain):
"""Synapse admin v2 user JSON -> SSO username, or None."""
for ext in user.get("external_ids") or []:
provider = ext.get("auth_provider") or ""
ext_id = ext.get("external_id") or ""
# One legacy link stores the plain username instead of Authelia's opaque ID.
if (provider.startswith("http") and re.fullmatch(r"[a-z0-9._-]+", ext_id)
and not UUID_RE.fullmatch(ext_id)):
return ext_id
suffix = "@" + email_domain.lower()
for tp in user.get("threepids") or []:
addr = (tp.get("address") or "").strip().lower()
if tp.get("medium") == "email" and addr.endswith(suffix):
local = addr[: -len(suffix)]
if re.fullmatch(r"[a-z0-9._-]+", local):
return local
return None
def public_entry(entry, now_ms):
"""What Lotus Chat gets for one listener."""
out = {k: entry[k] for k in ("title", "artist", "album", "durationS", "positionMs", "state", "player")}
out["updatedAt"] = now_ms
out["cover"] = f"{PREFIX}/cover/{urllib.parse.quote(entry['coverArt'], safe='')}" if entry.get("coverArt") else None
return out
def parse_ids(raw, server_name):
ids = []
for part in (raw or "").split(","):
part = part.strip()
if part.startswith("@") and part.endswith(":" + server_name) and part not in ids:
ids.append(part)
return ids[:MAX_IDS]
# --- stateful pieces ---------------------------------------------------------
class OptIns:
"""{mxid: True} persisted atomically."""
def __init__(self, path):
self.path = path
self.lock = threading.Lock()
self.data = {}
try:
with open(path) as f:
loaded = json.load(f)
self.data = {k: True for k, v in loaded.items() if v is True}
except FileNotFoundError:
pass
except (OSError, ValueError) as exc:
log.error("could not read %s: %s", path, exc)
def get(self, mxid):
with self.lock:
return self.data.get(mxid, False)
def all(self):
with self.lock:
return list(self.data)
def set(self, mxid, share):
with self.lock:
if share:
self.data[mxid] = True
else:
self.data.pop(mxid, None)
snapshot = dict(self.data)
d = os.path.dirname(self.path) or "."
fd, tmp = tempfile.mkstemp(dir=d, prefix=".state-")
try:
with os.fdopen(fd, "w") as f:
json.dump(snapshot, f, indent=1, sort_keys=True)
os.replace(tmp, self.path)
except BaseException:
try:
os.unlink(tmp)
except OSError:
pass
raise
class TTLCache:
def __init__(self, max_items=2000):
self.lock = threading.Lock()
self.items = {}
self.max_items = max_items
def get(self, key, now=None):
now = time.monotonic() if now is None else now
with self.lock:
hit = self.items.get(key)
if hit and hit[1] > now:
return hit[0], True
self.items.pop(key, None)
return None, False
def put(self, key, value, ttl, now=None):
now = time.monotonic() if now is None else now
with self.lock:
if len(self.items) >= self.max_items:
for k in [k for k, (_, exp) in self.items.items() if exp <= now] or list(self.items)[:100]:
self.items.pop(k, None)
self.items[key] = (value, now + ttl)
class Service:
def __init__(self, cfg, http_get=None):
self.cfg = cfg
self.http_get = http_get or _http_get
self.optins = OptIns(os.path.join(cfg["state_dir"], "state.json"))
self.tokens = TTLCache()
self.users = TTLCache()
self.lock = threading.Lock()
self.playing = {}
self.overrides = {}
self.overrides_mtime = None
self.load_overrides()
# Navidrome
def subsonic_url(self, method, **params):
salt = secrets.token_hex(8)
token = hashlib.md5((self.cfg["nd_password"] + salt).encode()).hexdigest() # noqa: S324 (Subsonic protocol)
q = {"u": self.cfg["nd_user"], "t": token, "s": salt, "v": SUBSONIC_VERSION,
"c": SUBSONIC_CLIENT, "f": "json", **params}
return f"{self.cfg['nd_url']}/rest/{method}?{urllib.parse.urlencode(q)}"
def poll_once(self):
status, body, _ = self.http_get(self.subsonic_url("getNowPlaying"), {}, 10)
if status != 200:
raise ValueError(f"getNowPlaying HTTP {status}")
parsed = parse_now_playing(json.loads(body), self.cfg["ignore_users"])
with self.lock:
self.playing = parsed
return len(parsed)
def poll_forever(self):
failures = 0
while True:
try:
n = self.poll_once()
if failures:
log.info("navidrome reachable again (%d sessions)", n)
failures = 0
except Exception as exc: # keep polling whatever happens
failures += 1
if failures in (1, 10) or failures % 100 == 0:
log.warning("poll failed (%d in a row): %s", failures, exc)
if failures >= 3:
with self.lock:
self.playing = {}
time.sleep(self.cfg["poll_s"])
# identity
def verify_token(self, token):
if not token or len(token) > 512:
return None
cached, hit = self.tokens.get(token)
if hit:
return cached
url = (f"{self.cfg['synapse']}/_matrix/federation/v1/openid/userinfo?"
+ urllib.parse.urlencode({"access_token": token}))
try:
status, body, _ = self.http_get(url, {}, 5)
except Exception as exc:
log.warning("openid check failed: %s", exc)
return None
mxid = None
if status == 200:
sub = (json.loads(body) or {}).get("sub") or ""
if sub.startswith("@") and sub.endswith(":" + self.cfg["server_name"]):
mxid = sub
# Tokens live about an hour; a rejected one is remembered briefly.
self.tokens.put(token, mxid, 600 if mxid else 30)
return mxid
def load_overrides(self):
path = self.cfg["users_file"]
try:
mtime = os.path.getmtime(path)
except OSError:
self.overrides = {}
return
if mtime == self.overrides_mtime:
return
try:
with open(path) as f:
data = json.load(f)
self.overrides = {k: v for k, v in data.items() if isinstance(v, str) and v}
self.overrides_mtime = mtime
except (OSError, ValueError) as exc:
log.error("could not read %s: %s", path, exc)
def resolve(self, mxid):
self.load_overrides()
if mxid in self.overrides:
return self.overrides[mxid]
cached, hit = self.users.get(mxid)
if hit:
return cached
name = None
url = f"{self.cfg['synapse']}/_synapse/admin/v2/users/{urllib.parse.quote(mxid)}"
try:
status, body, _ = self.http_get(url, {"Authorization": f"Bearer {self.cfg['matrix_token']}"}, 5)
if status == 200:
name = username_from_admin_user(json.loads(body), self.cfg["email_domain"])
elif status != 404:
return None # don't cache transient failures
except Exception as exc:
log.warning("user lookup failed: %s", exc)
return None
self.users.put(mxid, name, 3600 if name else 300)
return name
# views
def entry_for(self, mxid):
if not self.optins.get(mxid):
return None
name = self.resolve(mxid)
if not name:
return None
with self.lock:
return self.playing.get(name)
def users_view(self, ids):
now_ms = int(time.time() * 1000)
out = {}
for mxid in ids:
entry = self.entry_for(mxid)
if entry:
out[mxid] = public_entry(entry, now_ms)
return out
def me_view(self, mxid):
name = self.resolve(mxid)
share = self.optins.get(mxid)
entry = None
if name:
with self.lock:
entry = self.playing.get(name)
return {
"share": share,
"linked": bool(name),
"playing": public_entry(entry, int(time.time() * 1000)) if entry else None,
}
def cover_allowed(self, cover_id):
with self.lock:
playing = dict(self.playing)
wanted = {name for name, e in playing.items() if e.get("coverArt") == cover_id}
if not wanted:
return False
return any(self.resolve(m) in wanted for m in self.optins.all())
# --- HTTP --------------------------------------------------------------------
def _http_get(url, headers, timeout):
req = urllib.request.Request(url, headers=headers)
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.status, resp.read(), resp.headers.get("Content-Type", "")
except urllib.error.HTTPError as exc:
return exc.code, exc.read(), exc.headers.get("Content-Type", "")
class Handler(BaseHTTPRequestHandler):
server_version = "lotus-now-playing"
service: Service = None # set in main()
def log_message(self, fmt, *args): # no request logs: paths carry user IDs
pass
def _cors(self):
self.send_header("Access-Control-Allow-Origin", "*")
self.send_header("Access-Control-Allow-Headers", "Authorization, Content-Type")
self.send_header("Access-Control-Allow-Methods", "GET, PUT, OPTIONS")
self.send_header("Access-Control-Max-Age", "600")
def _json(self, status, obj):
body = json.dumps(obj).encode()
self.send_response(status)
self._cors()
self.send_header("Content-Type", "application/json")
self.send_header("Cache-Control", "no-store")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def _caller(self):
auth = self.headers.get("Authorization") or ""
token = auth[7:].strip() if auth.lower().startswith("bearer ") else ""
return self.service.verify_token(token)
def do_OPTIONS(self):
self.send_response(204)
self._cors()
self.end_headers()
def do_GET(self):
url = urllib.parse.urlsplit(self.path)
if url.path == "/healthz":
return self._json(200, {"ok": True})
if url.path.startswith(PREFIX + "/cover/"):
return self._cover(urllib.parse.unquote(url.path[len(PREFIX) + 7:]), url.query)
mxid = self._caller()
if not mxid:
return self._json(401, {"errcode": "M_UNKNOWN_TOKEN", "error": "Matrix OpenID token required"})
if url.path == PREFIX + "/me":
return self._json(200, self.service.me_view(mxid))
if url.path == PREFIX + "/users":
raw = urllib.parse.parse_qs(url.query).get("ids", [""])[0]
ids = parse_ids(raw, self.service.cfg["server_name"])
return self._json(200, {"users": self.service.users_view(ids)})
return self._json(404, {"errcode": "M_NOT_FOUND"})
def do_PUT(self):
url = urllib.parse.urlsplit(self.path)
if url.path != PREFIX + "/me":
return self._json(404, {"errcode": "M_NOT_FOUND"})
mxid = self._caller()
if not mxid:
return self._json(401, {"errcode": "M_UNKNOWN_TOKEN", "error": "Matrix OpenID token required"})
try:
length = int(self.headers.get("Content-Length") or 0)
body = json.loads(self.rfile.read(min(length, 4096)) or b"{}")
share = body["share"]
if not isinstance(share, bool):
raise ValueError
except (ValueError, KeyError, TypeError):
return self._json(400, {"errcode": "M_BAD_JSON", "error": "expected {\"share\": true|false}"})
try:
self.service.optins.set(mxid, share)
except OSError as exc:
log.error("could not save opt-in: %s", exc)
return self._json(500, {"errcode": "M_UNKNOWN", "error": "could not save"})
log.info("sharing %s (now %d users)", "on" if share else "off", len(self.service.optins.all()))
return self._json(200, self.service.me_view(mxid))
def _cover(self, cover_id, query):
if not cover_id or len(cover_id) > 128 or not self.service.cover_allowed(cover_id):
return self._json(404, {"errcode": "M_NOT_FOUND"})
try:
size = int(urllib.parse.parse_qs(query).get("size", ["256"])[0])
except ValueError:
size = 256
size = min(COVER_SIZES, key=lambda s: abs(s - size))
try:
status, body, ctype = self.service.http_get(
self.service.subsonic_url("getCoverArt", id=cover_id, size=size), {}, 10)
except Exception:
status, body, ctype = 502, b"", ""
if status != 200 or not ctype.startswith("image/"):
return self._json(502, {"errcode": "M_UNKNOWN", "error": "cover unavailable"})
self.send_response(200)
self._cors()
self.send_header("Content-Type", ctype)
self.send_header("Cache-Control", "public, max-age=300")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def load_config(env=os.environ):
def need(name):
value = env.get(name, "")
if not value:
raise SystemExit(f"missing {name}")
return value
return {
"bind": env.get("BIND_HOST", "0.0.0.0"),
"port": int(env.get("BIND_PORT", "8072")),
"server_name": env.get("SERVER_NAME", "matrix.lotusguild.org"),
"email_domain": env.get("EMAIL_DOMAIN", "lotusguild.org"),
"synapse": env.get("SYNAPSE_API", "http://127.0.0.1:8008").rstrip("/"),
"matrix_token": need("MATRIX_TOKEN"),
"nd_url": need("NAVIDROME_URL").rstrip("/"),
"nd_user": need("NAVIDROME_USER"),
"nd_password": need("NAVIDROME_PASSWORD"),
"ignore_users": tuple(u for u in env.get("IGNORE_USERS", "droppedneedle").split(",") if u),
"state_dir": env.get("STATE_DIR", "/var/lib/lotus-now-playing"),
"users_file": env.get("USERS_FILE", "/etc/lotus-now-playing/users.json"),
"poll_s": max(5, int(env.get("POLL_SECONDS", "10"))),
}
def main():
logging.basicConfig(level=logging.INFO, format="%(levelname)s %(message)s")
cfg = load_config()
service = Service(cfg)
try:
log.info("navidrome ok, %d sessions", service.poll_once())
except Exception as exc:
log.warning("first poll failed: %s", exc)
threading.Thread(target=service.poll_forever, daemon=True).start()
Handler.service = service
server = ThreadingHTTPServer((cfg["bind"], cfg["port"]), Handler)
server.daemon_threads = True
log.info("listening on %s:%d (%d users sharing)", cfg["bind"], cfg["port"], len(service.optins.all()))
server.serve_forever()
if __name__ == "__main__":
main()
+173
View File
@@ -0,0 +1,173 @@
"""Tests for lotus_now_playing.py. Run: python3 -m unittest now-playing/test_lotus_now_playing.py"""
import json
import os
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import lotus_now_playing as np # noqa: E402
def nd(entries, status="ok"):
return {"subsonic-response": {"status": status, "nowPlaying": {"entry": entries}}}
def entry(user, title="Song", minutes=0, state="playing", cover="al-1", **kw):
e = {"username": user, "title": title, "artist": "Band", "album": "LP", "duration": 200,
"minutesAgo": minutes, "state": state, "coverArt": cover, "playerName": "DN", "positionMs": 1000}
e.update(kw)
return e
class ParseTests(unittest.TestCase):
def test_basic_and_ignored_users(self):
out = np.parse_now_playing(nd([entry("jared"), entry("droppedneedle")]), ("droppedneedle",))
self.assertEqual(list(out), ["jared"])
self.assertEqual(out["jared"]["artist"], "Band")
self.assertEqual(out["jared"]["durationS"], 200)
def test_prefers_display_artist(self):
out = np.parse_now_playing(nd([entry("a", displayArtist="Band feat. X")]))
self.assertEqual(out["a"]["artist"], "Band feat. X")
def test_newest_session_wins(self):
out = np.parse_now_playing(nd([entry("a", "Old", minutes=5), entry("a", "New", minutes=0)]))
self.assertEqual(out["a"]["title"], "New")
def test_drops_stopped_untitled_and_stale(self):
out = np.parse_now_playing(nd([
entry("a", state="stopped"),
entry("b", title=""),
entry("c", minutes=90, positionMs=0),
entry("d", state="paused"),
]))
self.assertEqual(list(out), ["d"])
def test_single_entry_object_and_empty(self):
self.assertEqual(list(np.parse_now_playing(nd(entry("a")))), ["a"])
self.assertEqual(np.parse_now_playing({"subsonic-response": {"status": "ok"}}), {})
def test_error_raises(self):
with self.assertRaises(ValueError):
np.parse_now_playing({"subsonic-response": {"status": "failed", "error": {"message": "nope"}}})
class MappingTests(unittest.TestCase):
def test_email_on_domain(self):
user = {"threepids": [{"medium": "email", "address": "Tyler@LotusGuild.org"}]}
self.assertEqual(np.username_from_admin_user(user, "lotusguild.org"), "tyler")
def test_other_domain_ignored(self):
user = {"threepids": [{"medium": "email", "address": "x@gmail.com"}]}
self.assertIsNone(np.username_from_admin_user(user, "lotusguild.org"))
def test_legacy_external_id(self):
user = {"external_ids": [{"auth_provider": "https://auth.lotusguild.org", "external_id": "cole"}]}
self.assertEqual(np.username_from_admin_user(user, "lotusguild.org"), "cole")
def test_opaque_uuid_is_not_a_username(self):
user = {"external_ids": [
{"auth_provider": "oidc-authelia", "external_id": "b546660f-371c-45de-812c-bbae0e615148"},
{"auth_provider": "https://auth.lotusguild.org", "external_id": "b546660f-371c-45de-812c-bbae0e615148"},
]}
self.assertIsNone(np.username_from_admin_user(user, "lotusguild.org"))
def test_parse_ids(self):
ids = np.parse_ids("@a:hs, @b:other,@a:hs,junk,@c:hs", "hs")
self.assertEqual(ids, ["@a:hs", "@c:hs"])
self.assertEqual(len(np.parse_ids(",".join(f"@u{i}:hs" for i in range(80)), "hs")), np.MAX_IDS)
class FakeHttp:
def __init__(self):
self.calls = []
self.now_playing = nd([entry("jared", cover="al-9"), entry("jeff", cover="al-7")])
self.admin = {
"@jared:hs": {"threepids": [{"medium": "email", "address": "jared@lotusguild.org"}]},
"@nomail:hs": {"threepids": []},
}
self.openid = {"good": "@jared:hs", "foreign": "@x:elsewhere"}
def __call__(self, url, headers, timeout):
self.calls.append(url)
if "/rest/getNowPlaying" in url:
return 200, json.dumps(self.now_playing).encode(), "application/json"
if "/openid/userinfo" in url:
tok = url.split("access_token=")[1]
if tok in self.openid:
return 200, json.dumps({"sub": self.openid[tok]}).encode(), "application/json"
return 401, b"{}", "application/json"
if "/_synapse/admin/v2/users/" in url:
mxid = url.rsplit("/", 1)[1].replace("%40", "@").replace("%3A", ":")
if mxid in self.admin:
return 200, json.dumps(self.admin[mxid]).encode(), "application/json"
return 404, b"{}", "application/json"
raise AssertionError(url)
class ServiceTests(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
users = os.path.join(self.tmp.name, "users.json")
with open(users, "w") as f:
json.dump({"@bigmanjeff:hs": "jeff"}, f)
self.http = FakeHttp()
self.svc = np.Service({
"server_name": "hs", "email_domain": "lotusguild.org", "synapse": "http://syn",
"matrix_token": "admin", "nd_url": "http://nd", "nd_user": "svc", "nd_password": "pw",
"ignore_users": ("droppedneedle",), "state_dir": self.tmp.name, "users_file": users, "poll_s": 10,
}, http_get=self.http)
self.svc.poll_once()
def tearDown(self):
self.tmp.cleanup()
def test_token_check_and_cache(self):
self.assertEqual(self.svc.verify_token("good"), "@jared:hs")
self.assertEqual(self.svc.verify_token("good"), "@jared:hs")
self.assertEqual(sum("/openid/" in c for c in self.http.calls), 1)
self.assertIsNone(self.svc.verify_token("bad"))
self.assertIsNone(self.svc.verify_token("foreign"))
self.assertIsNone(self.svc.verify_token(""))
def test_opt_in_required(self):
self.assertEqual(self.svc.users_view(["@jared:hs", "@bigmanjeff:hs"]), {})
self.svc.optins.set("@jared:hs", True)
view = self.svc.users_view(["@jared:hs", "@bigmanjeff:hs"])
self.assertEqual(list(view), ["@jared:hs"])
self.assertEqual(view["@jared:hs"]["cover"], "/now-playing/v1/cover/al-9")
def test_override_mapping(self):
self.svc.optins.set("@bigmanjeff:hs", True)
self.assertIn("@bigmanjeff:hs", self.svc.users_view(["@bigmanjeff:hs"]))
def test_opt_in_persists_and_opt_out(self):
self.svc.optins.set("@jared:hs", True)
reloaded = np.OptIns(os.path.join(self.tmp.name, "state.json"))
self.assertTrue(reloaded.get("@jared:hs"))
self.svc.optins.set("@jared:hs", False)
self.assertEqual(self.svc.users_view(["@jared:hs"]), {})
def test_me_view(self):
me = self.svc.me_view("@jared:hs")
self.assertEqual((me["share"], me["linked"]), (False, True))
self.assertEqual(me["playing"]["title"], "Song")
self.assertEqual(self.svc.me_view("@nomail:hs"), {"share": False, "linked": False, "playing": None})
def test_cover_allow_list(self):
self.assertFalse(self.svc.cover_allowed("al-9")) # nobody opted in yet
self.svc.optins.set("@jared:hs", True)
self.assertTrue(self.svc.cover_allowed("al-9"))
self.assertFalse(self.svc.cover_allowed("al-7")) # jeff hasn't opted in
self.assertFalse(self.svc.cover_allowed("al-unknown"))
def test_subsonic_url_never_contains_password(self):
url = self.svc.subsonic_url("getNowPlaying")
self.assertNotIn("pw", url.split("?")[1].split("&t=")[0])
self.assertIn("u=svc", url)
self.assertNotIn("p=", url)
if __name__ == "__main__":
unittest.main()
+31
View File
@@ -0,0 +1,31 @@
[Unit]
Description=Discord presence to Matrix ghosts (Lotus)
After=network-online.target matrix-synapse.service mautrix-discord.service
Wants=network-online.target
[Service]
Type=simple
# The bridge's user: it reads the bot token from the bridge's own config + database.
User=mautrix-discord
Group=mautrix-discord
EnvironmentFile=/etc/lotus-presence/env
Environment=BRIDGE_CONFIG=/opt/mautrix-discord/config.yaml
Environment=SERVER_NAME=matrix.lotusguild.org
Environment=DISCORD_GUILD_ID=1565159371355197570
ExecStart=/usr/bin/python3 /opt/lotus-presence/lotus_presence.py
Restart=always
RestartSec=30
StandardOutput=journal
StandardError=journal
SyslogIdentifier=lotus-presence
MemoryMax=128M
CPUQuota=20%
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
+21
View File
@@ -0,0 +1,21 @@
[Unit]
Description=Lotus Minecraft player count as the bridge bot's Matrix status
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/python3 -I /opt/lotus-mc-status/lotus_mc_status.py
Environment=LOTUS_MC_HOST=10.10.10.78
Environment=LOTUS_MC_PORT=25565
LoadCredential=bridge-config:/var/lib/pelican/volumes/5476addf-384e-4f49-baa6-303e05719e9f/config/minecraftmatrixbridge.toml
DynamicUser=yes
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
Restart=always
RestartSec=30
[Install]
WantedBy=multi-user.target
+38
View File
@@ -0,0 +1,38 @@
[Unit]
Description=Now playing (Navidrome) for Lotus Chat "Listening to" cards
After=network-online.target matrix-synapse.service
Wants=network-online.target
[Service]
Type=simple
User=lotus-now-playing
Group=lotus-now-playing
# MATRIX_TOKEN (Synapse admin API, user lookups only), shared with voice-limit-guard.
EnvironmentFile=/etc/matrix-deploy.env
# NAVIDROME_USER / NAVIDROME_PASSWORD for the read-only lotus-nowplaying account.
EnvironmentFile=/etc/lotus-now-playing/env
Environment=NAVIDROME_URL=http://10.10.10.28:4533
Environment=SERVER_NAME=matrix.lotusguild.org
Environment=BIND_HOST=0.0.0.0
Environment=BIND_PORT=8072
Environment=STATE_DIR=/var/lib/lotus-now-playing
Environment=USERS_FILE=/etc/lotus-now-playing/users.json
StateDirectory=lotus-now-playing
StateDirectoryMode=0750
ExecStart=/usr/bin/python3 /opt/lotus-now-playing/lotus_now_playing.py
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=lotus-now-playing
MemoryMax=96M
CPUQuota=20%
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
+17
View File
@@ -0,0 +1,17 @@
[Unit]
Description=Compress Synapse state groups (synapse_auto_compressor)
Documentation=https://github.com/matrix-org/rust-synapse-compress-state
After=postgresql.service
[Service]
Type=oneshot
User=postgres
# 100 chunks of 500 state groups per night. Progress is stored in the
# state_compressor_* tables, so each run continues where the last one stopped.
# Every state group is rewritten in its own transaction, after the compressed
# state is checked against the original; a chunk that would not shrink is skipped.
ExecStart=/usr/local/bin/synapse_auto_compressor -p postgresql:///synapse?host=/var/run/postgresql&user=postgres -c 500 -n 100
Environment=RUST_LOG=info
Nice=10
IOSchedulingClass=idle
TimeoutStartSec=2h
+10
View File
@@ -0,0 +1,10 @@
[Unit]
Description=Nightly Synapse state compression (after the 03:00 vzdump backup)
[Timer]
OnCalendar=*-*-* 04:30
RandomizedDelaySec=10min
Persistent=true
[Install]
WantedBy=timers.target