Add Wordle, welcome system, integrations, and update roadmap

- Add Wordle game engine with daily puzzles, hard mode, stats, and share - Add welcome module (react-to-join onboarding, Space join DMs) - Add Ollama LLM integration (!ask), Minecraft RCON whitelist (!minecraft) - Add !trivia, !champion, !agent, !health commands - Add DM routing for Wordle (games in DMs, share to public room) - Update README: reflect Phase 4 completion, hookshot webhook setup, infrastructure migration (LXC 151/109 to large1), Spam and Stuff room, all 12 webhook connections with UUIDs and transform notes Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 10:29:36 -05:00
parent 5723ac3581
commit dff2f0e2b1
11 changed files with 4324 additions and 167 deletions
--- a/README.md
+++ b/README.md
@@ -1,183 +1,241 @@
 # Lotus Matrix Bot & Server Roadmap

-Matrix bot and server improvements for the Lotus Guild homeserver (`matrix.lotusguild.org`).
+Matrix bot and server infrastructure for the Lotus Guild homeserver (`matrix.lotusguild.org`).

 **Repo**: https://code.lotusguild.org/LotusGuild/matrixBot

-## Status: Phase 1 Complete (Database + Voice/Video)
+## Status: Phase 4 — Webhooks & Integrations

 ---

-## Priority Order (suggested)
-1. ~~PostgreSQL migration (SQLite will bottleneck everything else)~~
-2. ~~TURN server (makes voice/video actually reliable)~~
-3. Custom Element Web (chat.lotusguild.org with branding)
-4. Discord bridge (lets people transition gradually)
-5. Custom emoji packs (makes it feel like home)
-6. Room structure + space setup
-7. Moderation bot
-8. Matrix bot
-9. Everything else
+## Priority Order
+1. ~~PostgreSQL migration~~
+2. ~~TURN server~~
+3. ~~Room structure + space setup~~
+4. ~~Matrix bot (core + commands)~~
+5. ~~LiveKit / Element Call~~
+6. ~~SSO / OIDC (Authelia)~~
+7. ~~Webhook integrations (hookshot)~~
+8. Discord bridge (lets people transition gradually)
+9. Custom Element Web (chat.lotusguild.org with branding)
+10. Custom emoji packs
+11. Moderation bot
+12. Everything else

 ---

 ## Infrastructure

 | Service | Host | IP | LXC | Notes |
-|---------|------|-----|-----|-------|
-| Synapse | storage-01 | 10.10.10.29 | 151 | Homeserver + coturn |
-| PostgreSQL 17 | storage-01 | 10.10.10.44 | 109 | Synapse database backend |
-| NPM | large1 | 10.10.10.27 | 139 | Reverse proxy for matrix.lotusguild.org |
+|---------|------|----|-----|-------|
+| Synapse | large1 | 10.10.10.29 | 151 | Homeserver + coturn + LiveKit + hookshot |
+| PostgreSQL 17 | large1 | 10.10.10.2 | 109 | Synapse database backend |
+| NPM | compute-storage-01 | 10.10.10.27 | 139 | Reverse proxy + landing page |
+| Authelia | compute-storage-01 | 10.10.10.36 | 167 | SSO/OIDC provider |
+| LLDAP | large1 | 10.10.10.39 | 147 | LDAP user directory |

-**Key paths on Synapse LXC (10.10.10.29):**
+**Key paths on Synapse LXC:**
 - Synapse config: `/etc/matrix-synapse/homeserver.yaml`
- Synapse venv: `/opt/venvs/matrix-synapse/`
 - coturn config: `/etc/turnserver.conf`
- Synapse admin UI: `/var/www/synapse-admin/` (nginx on :8080)
- SQLite backup: `/var/lib/matrix-synapse/homeserver.db`
+- LiveKit config: `/etc/livekit/config.yaml`
+- LiveKit service: `livekit-server.service`
+- Hookshot: `/opt/hookshot/`, service: `matrix-hookshot.service`
+- Hookshot config: `/opt/hookshot/config.yml`
+- Hookshot registration: `/etc/matrix-synapse/hookshot-registration.yaml`
+- Landing page: `/var/www/matrix-landing/index.html` (on NPM LXC 139)
+- Bot: `/opt/matrixbot/`, service: `matrixbot.service`

-**Port forwarding (router -> 10.10.10.29):**
- TCP+UDP 3478 (TURN signaling)
- UDP 49152-65535 (media relay)
+**Port forwarding (router → 10.10.10.29):**
+- TCP+UDP 3478 (TURN/STUN signaling)
+- TCP+UDP 5349 (TURNS/TLS)
+- TCP 7881 (LiveKit ICE TCP fallback)
+- TCP+UDP 49152-65535 (TURN relay + LiveKit WebRTC media)

 ---

-## Server - Quality of Life
- [x] Migrate from SQLite to PostgreSQL (critical for performance at any real scale)
- [x] Set up TURN/STUN server (coturn) for reliable voice/video calls behind NAT
- [x] Enable URL previews in Synapse
- [x] Increase upload size limit for media/GIFs (200MB)
- [x] Enable message search (full-text search with PostgreSQL backend)
- [x] Configure media retention policy (remote: 1yr, local: 3yr)
- [x] Set up sliding sync (native in Synapse, no proxy needed)
- [ ] Enable push notifications gateway for mobile clients
+## Rooms (all v12)

-## Server - Hardening
- [x] Rate limiting configuration in Synapse
- [ ] Federation allow/deny lists (decide if you want open federation or Lotus-only)
- [ ] E2EE by default for private rooms
- [ ] Regular Synapse version updates
- [ ] Monitoring with Prometheus + Grafana (you already have both at 10.10.10.x)
- [ ] Synapse worker mode if performance becomes an issue
+| Room | Room ID | Join Rule |
+|------|---------|-----------|
+| The Lotus Guild (Space) | `!-1ZBnAH-JiCOV8MGSKN77zDGTuI3pgSdy8Unu_DrDyc` | public |
+| General | `!wfokQ1-pE896scu_AOcCBA2s3L4qFo-PTBAFTd0WMI0` | public |
+| Commands | `!ou56mVZQ8ZB7AhDYPmBV5_BR28WMZ4x5zwZkPCqjq1s` | restricted (Space members) |
+| Memes | `!GK6v5cLEEnowIooQJv5jECfISUjADjt8aKhWv9VbG5U` | public |
+| Management | `!mEvR5fe3jMmzwd-FwNygD72OY_yu8H3UP_N-57oK7MI` | invite |
+| Cool Kids | `!R7DT3QZHG9P8QQvX6zsZYxjkKgmUucxDz_n31qNrC94` | invite |
+| Welcome | `!Y-wvNosuytqBOWampH9k-ta7bYXW7okqwBQ7PuRVBWE` | public |
+| Spam and Stuff | `!GttT4QYd1wlGlkHU3qTmq_P3gbyYKKeSSN6R7TPcJHg` | invite, **no E2EE** (hookshot) |

-## Server - Admin & Moderation
- [ ] Set up Mjolnir or Draupnir (moderation bot for ban lists, spam protection)
- [ ] Configure proper power levels per room (mimic Discord role hierarchy)
- [ ] Invite-only registration flow (already have token-based registration)
- [ ] Set up room ACLs for federation control (block known-bad servers)
- [x] Synapse admin API dashboard (synapse-admin v0.11.1 at http://10.10.10.29:8080)
- [ ] Automated backups of Synapse database and media
+**Power level roles (Cinny tags):**
+- 100: Owner (jared)
+- 50: The Nerdy Council (enhuynh, lonely)
+- 48: Panel of Geeks
+- 35: Cool Kids
+- 0: Member

-## Bridging (Transition Period)
- [ ] Set up mautrix-discord bridge so messages flow between Discord and Matrix
- [ ] Bridge key channels (general, gaming, memes, etc.)
- [ ] Bridge voice channels if possible (experimental, may not be worth it)
- [ ] Puppet bridging so Discord users appear as Matrix users and vice versa
+---

-## Room Structure
- [ ] Create room directory matching Discord channel layout
- [ ] Set up a Space for "Lotus Guild" as the top-level container
- [ ] Sub-spaces for categories (General, Gaming, Media, Admin, etc.)
- [ ] Welcome room with pinned onboarding instructions
+## Webhook Integrations (matrix-hookshot 7.3.2)
+
+Generic webhooks bridged into **Spam and Stuff** via [matrix-hookshot](https://github.com/matrix-org/matrix-hookshot).
+Each service gets its own virtual user (`@hookshot_<service>`) with a unique avatar.
+Webhook URL format: `https://matrix.lotusguild.org/webhook/<uuid>`
+
+| Service | Webhook UUID | Notes |
+|---------|-------------|-------|
+| Grafana | `df4a1302-2d62-4a01-b858-fb56f4d3781a` | Unified alerting contact point |
+| Proxmox | `9b3eafe5-7689-4011-addd-c466e524661d` | Notification system (8.1+) |
+| Sonarr | `aeffc311-0686-42cb-9eeb-6757140c072e` | All event types |
+| Radarr | `34913454-c1ac-4cda-82ea-924d4a9e60eb` | All event types |
+| Readarr | `e57ab4f3-56e6-4dc4-8b30-2f4fd4bbeb0b` | All event types |
+| Lidarr | `66ac6fdd-69f6-4f47-bb00-b7f6d84d7c1c` | All event types |
+| Uptime Kuma | `1a02e890-bb25-42f1-99fe-bba6a19f1811` | Status change notifications |
+| Seerr | `555185af-90a1-42ff-aed5-c344e11955cf` | Request/approval events |
+| Owncast | `9993e911-c68b-4271-a178-c2d65ca88499` | STREAM_STARTED / STREAM_STOPPED |
+| Bazarr | `470fb267-3436-4dd3-a70c-e6e8db1721be` | Subtitle events |
+| Huntarr | `78af735b-7802-4e6c-987b-db22f8636ceb` | Hunt/search events |
+| Tinker-Tickets | `6e306faf-8eea-4ba5-83ef-bf8f421f929e` | Custom code needed |
+
+**Hookshot notes:**
+- Spam and Stuff is intentionally **unencrypted** — hookshot bridges cannot join E2EE rooms
+- Webhook tokens stored in Synapse PostgreSQL `room_account_data` for `@hookshot`
+- JS transformation functions use hookshot v2 API: set `result = { version: "v2", plain, html, msgtype }`
+- The `result` variable must be assigned without `var`/`let`/`const` (needs implicit global scope in the QuickJS IIFE sandbox)
+- NPM proxies `https://matrix.lotusguild.org/webhook/*` → `http://10.10.10.29:9003`
+- Virtual user avatars: set via appservice token (`as_token` in hookshot-registration.yaml) impersonating each user
+
+---
+
+## Server Checklist
+
+### Quality of Life
+- [x] Migrate from SQLite to PostgreSQL
+- [x] TURN/STUN server (coturn) for reliable voice/video
+- [x] URL previews
+- [x] Upload size limit 200MB
+- [x] Full-text message search (PostgreSQL backend)
+- [x] Media retention policy (remote: 1yr, local: 3yr)
+- [x] Sliding sync (native Synapse)
+- [x] LiveKit for Element Call video rooms
+- [x] Default room version v12, all rooms upgraded
+- [x] Landing page with client recommendations (Cinny, Commet, Element, Element X mobile)
+- [ ] Push notifications gateway (Sygnal) for mobile clients
+
+### Auth & SSO
+- [x] Token-based registration
+- [x] SSO/OIDC via Authelia
+- [x] `allow_existing_users: true` for linking accounts to SSO
+- [x] Password auth alongside SSO
+
+### Webhooks & Integrations
+- [x] matrix-hookshot 7.3.2 installed and running
+- [x] Generic webhook bridge for 12 services
+- [x] Per-service JS transformation functions (formatted messages with emoji)
+- [x] Per-service virtual user avatars
+- [x] NPM reverse proxy for `/webhook` path
+- [ ] Bazarr native Matrix integration (hookshot connection exists as fallback)
+- [ ] Huntarr webhook format investigation
+- [ ] Tinker Tickets custom code
+
+### Bridging
+- [ ] mautrix-discord bridge
+- [ ] Bridge key channels (general, gaming, memes)
+- [ ] Puppet bridging
+
+### Room Structure
+- [x] The Lotus Guild space
+- [x] All core rooms with correct power levels and join rules
+- [x] Welcome room with react-to-join onboarding
+- [x] Spam and Stuff room for service notifications (hookshot)
+- [x] Custom room avatars
 - [ ] Read-only announcements room
- [ ] Audit/logging room (admin-only, for the bot)
- [ ] Off-topic / memes room
- [ ] Game-specific rooms (Minecraft, Valorant, League, Hytale, etc.)
- [ ] Bot commands room (keep bot spam contained)
- [ ] Voice/video call room

-## Custom Emoji & Stickers
- [ ] Export all custom emojis from Discord server
- [ ] Create Matrix emoji packs (per-room or space-wide)
- [ ] Set up sticker picker widget in Element
- [ ] Import/create Lotus Guild sticker pack
- [ ] Look into maunium/stickerpicker for a self-hosted sticker server
+### Hardening
+- [x] Rate limiting
+- [x] E2EE on all rooms (except Spam and Stuff — intentional for hookshot)
+- [ ] Federation allow/deny lists
+- [ ] Regular Synapse updates
+- [ ] Automated database + media backups

-## Element/Client Customization
- [ ] Custom Element Web instance (self-hosted on chat.lotusguild.org)
- [ ] Custom theme with #980000 branding (Element supports custom CSS themes)
- [ ] Custom welcome/home page in Element Web
- [ ] Set default room list to show Lotus Guild space on first login
- [ ] Configure .well-known to point clients to custom Element Web instance
- [ ] Custom app name and branding in Element Web config.json
-
-## Widgets & Integrations
- [ ] Dimension integration manager (self-hosted, replaces Scalar)
- [ ] Jitsi widget for group voice/video calls (self-hosted)
- [ ] Etherpad widget for collaborative notes
- [ ] RSS bot for game news feeds (Minecraft updates, Valorant patches, etc.)
- [ ] GitHub/Gitea notifications bot (push events to a dev room)
-
-## Fun Stuff
- [ ] Custom room avatars with Lotus Guild branding
- [ ] Animated room banners
- [ ] Welcome bot message for new members joining the space
- [ ] Daily adjective posting (port from Discord bot)
- [ ] Game night scheduling bot/widget
- [ ] Karma/points system via bot reactions
- [ ] Custom Matrix "profile badges" via bot (similar to Discord roles showing on profile)
+### Admin
+- [x] Synapse admin API dashboard (synapse-admin at http://10.10.10.29:8080)
+- [x] Power levels per room
+- [ ] Mjolnir/Draupnir moderation bot

 ---

-## Bot - Core Setup
- [ ] Project scaffolding (`bot.py`, config, `.env`, requirements)
- [ ] matrix-nio async client with E2EE support
- [ ] Device verification / trust storage
- [ ] Logging (rotating file + stdout, matching Discord bot pattern)
- [ ] Config validation (homeserver URL, access token, room IDs)
- [ ] Graceful shutdown and reconnection handling
+## Bot Checklist

-## Bot - Command Porting (from Discord bot)
- [ ] `!help` - List all available commands
- [ ] `!ping` - Bot latency check
- [ ] `!8ball <question>` - Magic 8-ball
- [ ] `!fortune` - Fortune cookie message
- [ ] `!flip` - Coin flip
- [ ] `!roll <NdS>` - Dice roller
- [ ] `!random <min> <max>` - Random number generator
- [ ] `!rps <choice>` - Rock Paper Scissors
- [ ] `!poll <question>` - Poll (using reactions)
- [ ] `!trivia` - Trivia game (using reactions for answers)
- [ ] `!champion` - Random LoL champion picker
+### Core
+- [x] matrix-nio async client with E2EE
+- [x] Device trust (auto-trust all devices)
+- [x] Graceful shutdown (SIGTERM/SIGINT)
+- [x] Initial sync token (ignores old messages on startup)
+- [x] Auto-accept room invites
+- [x] Deployed as systemd service (`matrixbot.service`) on LXC 151

-## Bot - Integrations
- [ ] `!minecraft <username>` - RCON whitelist (Mojang API validation + RCON)
- [ ] `!hytale <username>` - Hytale whitelist request (audit log to admin room)
- [ ] `!ask <question>` - Ollama LLM integration (lotusllm model)
+### Commands
+- [x] `!help` — list commands
+- [x] `!ping` — latency check
+- [x] `!8ball <question>` — magic 8-ball
+- [x] `!fortune` — fortune cookie
+- [x] `!flip` — coin flip
+- [x] `!roll <NdS>` — dice roller
+- [x] `!random <min> <max>` — random number
+- [x] `!rps <choice>` — rock paper scissors
+- [x] `!poll <question>` — poll with reactions
+- [x] `!trivia` — trivia game (reactions, 30s reveal)
+- [x] `!champion [lane]` — random LoL champion
+- [x] `!agent [role]` — random Valorant agent
+- [x] `!wordle` — full Wordle game (daily, hard mode, stats, share)
+- [x] `!minecraft <username>` — RCON whitelist add
+- [x] `!ask <question>` — Ollama LLM (lotusllm, 2min cooldown)
+- [x] `!health` — bot uptime + service status

-## Bot - Admin Commands
- [ ] `!clear <count>` - Redact messages (requires power level)
- [ ] `!health` - Bot stats (uptime, command counts, service status)
- [ ] Power level checks (Matrix equivalent of Discord role checks)
+### Welcome System
+- [x] Auto-post welcome message in Welcome room on startup
+- [x] React-to-join: react with ✅ → bot invites to General, Commands, Memes
+- [x] Welcome event ID persisted to `welcome_state.json`
+- [x] Watches Space joins and DMs new members

-## Bot - Audit Logging
- [ ] Member join/leave events
- [ ] Message edits and redactions
- [ ] Room state changes
- [ ] Batched audit log posting to admin room
-
-## Bot - Deployment
- [ ] Systemd service (`matrixbot.service`)
- [ ] Auto-deploy from Gitea webhook (matching Discord bot pattern)
- [ ] Deployment script (`/usr/local/bin/matrix_bot_deploy.sh`)
- [ ] Determine host LXC (new container or colocate with Synapse on 10.10.10.29)
-
-## Bot - Not Porting (Discord-specific)
- Reaction roles (no Matrix equivalent)
- Status cycling (Matrix presence is simpler)
- Guild-specific event handlers (channel create/delete, boost, etc.)
+### Wordle
+- [x] Daily puzzles with two-pass letter evaluation
+- [x] Hard mode with constraint validation
+- [x] Stats persistence (`wordle_stats.json`)
+- [x] Cinny-compatible rendering (inline `<span>` tiles)
+- [x] DM-based gameplay, `!wordle share` posts result to public room
+- [x] Virtual keyboard display

 ---

 ## Tech Stack
- **Language**: Python 3
- **Library**: matrix-nio (with E2EE)
- **Homeserver**: matrix.lotusguild.org (Synapse on 10.10.10.29)
- **Database**: PostgreSQL 17 on 10.10.10.44
- **TURN**: coturn on 10.10.10.29 (colocated with Synapse)
- **Dependencies**: matrix-nio[e2ee], aiohttp, python-dotenv, mcrcon

-## Production Server
- **Host**: TBD
- **Bot Directory**: TBD
- **Service**: `matrixbot.service`
+| Component | Technology |
+|-----------|-----------|
+| Bot language | Python 3 |
+| Bot library | matrix-nio (E2EE) |
+| Homeserver | Synapse 1.147+ |
+| Database | PostgreSQL 17 |
+| TURN | coturn |
+| Video calls | LiveKit + lk-jwt-service |
+| SSO | Authelia (OIDC) + LLDAP |
+| Webhook bridge | matrix-hookshot 7.3.2 |
+| Reverse proxy | Nginx Proxy Manager |
+| Bot dependencies | matrix-nio[e2ee], aiohttp, python-dotenv, mcrcon |
+
+## Bot Files
+
+```
+matrixBot/
+├── bot.py              # Entry point, client setup, event loop
+├── callbacks.py        # Message + reaction event handlers
+├── commands.py         # All command implementations
+├── config.py           # Environment config + validation
+├── utils.py            # send_text, send_html, send_reaction, get_or_create_dm
+├── welcome.py          # Welcome message + react-to-join logic
+├── wordle.py           # Full Wordle game engine
+├── wordlist_answers.py # Wordle answer word list
+├── wordlist_valid.py   # Wordle valid guess word list
+├── .env.example        # Environment variable template
+└── requirements.txt    # Python dependencies
+```