Files
cinny/LOTUS_TESTING.md
T

71 lines
13 KiB
Markdown
Raw Normal View History

# Lotus Chat — Testing
> **Manual QA checklists no longer live here.** Every shipped-but-unverified behaviour is a Gitea issue labelled **`qa`** — cinny milestone [Manual QA backlog](https://code.lotusguild.org/LotusGuild/cinny/milestones), cinny-desktop milestone [Desktop QA backlog](https://code.lotusguild.org/LotusGuild/cinny-desktop/milestones). Each issue carries the full steps + expected results; tick items as they pass, comment on FAIL (what you saw vs expected, browser/OS, web vs desktop, theme, console errors), close when green. Migrated from this file on 2026-09-17 (full text in git history).
This file keeps what a contributor needs to run and extend the **automated** coverage, plus the ops tip at the bottom.
## Environment notes
- Test the **web** build (`chat.lotusguild.org`, LXC 106) first; re-run call + poll items on the **desktop (Tauri)** build too, since CSP and the EC iframe behave differently there.
- Several call checks need a **second participant** (marked **👥 2 people** in the issues); a couple need a third room/call in parallel (**👥👥**).
---
## Local dev environment — drive the real UI against a throwaway homeserver
```
scripts/dev-homeserver.sh start # Synapse in .dev-homeserver/ (venv, SQLite), open registration, no rate limits, :8008
python3 scripts/dev-seed.py 400 # alice + bob, "Busy Room": 400 messages, an image every 10th
npm start # Vite on :5173
```
Log in at `http://127.0.0.1:5173/login/http%3A%2F%2Flocalhost%3A8008/` as `alice` / `password123` (bob is the second participant; both can also be driven over the client API with their tokens). Playwright is installed (`npm run test:e2e:install`), so a scripted reproduction is `node` + `chromium.launch()` against `:5173` — this is how Gitea #163 was reproduced and its fix verified in both plain and encrypted rooms. `scripts/dev-homeserver.sh reset` wipes the database; `stop` shuts it down.
## Automated coverage map — what the unit tests already pin (2026-07)
**Read this before working a `qa` issue.** Much of the _logic_ the manual checks were written to catch is now locked by deterministic unit tests (`npm test`, 920+ cases, green in CI). Unit tests do **not** prove visual rendering, real-call behavior, the desktop build, E2EE, or cross-device sync — those still need a human. But where a decision is pure logic, you can **trust the test and spend your manual time on the human-only part**. For each row below, the middle column is "don't bother re-deriving this by hand"; the right column is "this is what your manual pass is actually for."
| QA item | Logic **pinned by a unit test** (trust it) | What still needs **you** (manual) |
| :----------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------- |
| **A1** ringtone previews | `callSounds.test.ts` — each style's synthesized melody (chime/soft/retro), click-free gain ramps, context unlock/reuse, unknown = no-op | that it's actually **audible** + the WebAudio first-gesture caveat |
| **A2** ringtone persist/fallback | `settings.test.ts` — unknown `ringtoneId` → default, malformed JSON → defaults, merge-over-defaults (this **is** A2 step 3) | the dropdown shows the persisted value after reload (trivial glance) |
| **B2/B3** poll voting | `poll.test.ts` (18) — vote tally, latest-per-sender, multi-select, cleared/re-vote, winners, results-visible, single-vs-multi validation | **visual** only: borders, radio-vs-checkbox, progress-bar fill, on each theme |
| **O2/P4-1** thread notifications | `threadNotifications.test.ts` (32) — the **entire** notify decision (participating default, All/Mentions/Mute, @mention+highlight override, room-mute trumps), mode-map + muted-badge hygiene | live **2-person** delivery + sound + **cross-device** account-data sync |
| **O3** math / LaTeX | `mathParse.test.ts` (14) — inline `$…$`, block `$$…$$`, **currency guard** (`$5 and $10`), escaped/unbalanced stay text, adjacency rules | KaTeX **renders** visually + lazy-chunk load; code-block-literal is the markdown pipeline |
| **O4/P4-8** encrypted search cache | `searchCache.test.ts` — the pure helpers `mergeSearchResults` (merge/dedupe/sort) + `computeCoverage` (window widening) + resilient-when-IDB-absent. **The IDB round-trip test is `skip`ped under `npm test`** (node has no IndexedDB), so it runs only in a browser-like env, not CI | the actual IndexedDB persist-across-**reload**, Clear button, **logout wipe** (integration — and the round-trip itself) |
| **M1** `has:image/file/video` | `useMessageSearch.test.ts``filterGroupsByMsgType` union filter, drops empty groups, ignores non-string msgtype | the chips render + compose with room/sender/date filters |
| **M4** pinned-only filter | `useMessageSearch.test.ts``filterGroupsByPinned` keeps pinned, drops empty | chip renders; needs a room with actual pins |
| **M2** recent searches | `recentSearches.test.ts` (6) — prepend, dedupe+move-to-front, trim, ignore-empty, cap-at-10 | chips render/click-re-run; persistence across refresh |
| **Retention** (disappearing msgs) | `retention.test.ts``isExpired` window math (strict boundary), disabled = never, preset monotonicity | the timeline **hide** + self-**redact** integration; Synapse-side purge |
| **O5/N97a** session hardening | `sessions.test.ts` (22) — blob migration, legacy-key coercion, dual-write blob↔legacy sync, corrupt/partial-blob fallback, token-refresh, AND the `subscribeSessionChanges` storage-event logic (fires on session/null, ignores unrelated keys) | the real **cross-tab** logout _behavior_ end-to-end (two live tabs) |
| **Q1/Q2** embeds (URL→player) | `videoEmbed.test.ts` (26) — every provider's URL→`{provider, kind, embedUrl, height}` parse (incl. Mixcloud/Deezer, TikTok, reserved-path guards) | the click-to-play **facade**, no-network-until-Play, the **CSP** (esp. desktop), visuals |
| **Seasonal theme resolution** (part of F2) | `seasonSchedule.test.ts``resolveSeasonTheme` (off→none, auto→active season, pinned→that) + `getActiveSeason` priority/boundary days. **NB: this pins _which_ theme shows for a date, NOT F2's background↔seasonal mutual exclusion** — that write-side logic is untested | all of **F2**: the picker actually clearing the _other_ setting live, and the overlay suppression when a background is set |
Everything else in the guide (calls, screen readers, desktop/Tauri, chat backgrounds, animated visuals, PWA install, real E2EE) is genuinely manual — no unit test substitutes for it. Items already verified live were dropped when the checklists moved to issues.
---
## Playwright smoke test (Gitea #90) — `npm run test:e2e`
Browser-level smoke tests under `e2e/` (config: `playwright.config.ts`). They boot the **built** `dist/` through `vite preview` on port 4173, so run `npm run build` first (one-time: `npm run test:e2e:install` downloads the pinned Chromium). Two tiers:
| Tier | File | When it runs | What it proves |
| :------------------------ | :-------------------------- | :----------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Boot** (always) | `e2e/boot.spec.ts` | every CI run (`e2e` job in `.gitea/workflows/ci.yml`) and locally | login page renders with `#root` populated and **no** `pageerror` / unexpected `console.error` (allowlist in `e2e/helpers.ts`: the README's avatar-thumbnail 404, the login page's `POST /register` 401 probe, offline discovery), `sw.js` is served and registers, bundled Element Call mounts in a frame with every `/public/element-call/` asset returning 200 |
| **E2EE composer** (gated) | `e2e/e2ee-composer.spec.ts` | only when `E2E_HOMESERVER`, `E2E_USER`, `E2E_PASSWORD` are all set | password login → `/home/create/` with the End-to-End Encryption switch on (asserts `createRoom` carries `m.room.encryption`) → text message renders → attach a generated JPEG with "Compress image before uploading" ticked, image renders → every `PUT …/rooms/*/send/*` was `m.room.encrypted` with `ciphertext` and no plaintext `body` / `url` / `file` / `mxc://` |
**CI secrets** (Gitea → repo → Settings → Actions → Secrets; the `e2e` job forwards them via `env:`; until they exist the E2EE tier reports `skipped`, the boot tier still runs):
- `E2E_HOMESERVER` — server name as typed on the login page (e.g. `matrix.example.org`). Must offer `m.login.password`; a next-gen-auth (MAS/OIDC-issuer) server shows only the OIDC button and the tier will fail at the username field.
- `E2E_USER` / `E2E_PASSWORD` — a **throwaway** account: each run logs in as a new device and creates a new `e2e-smoke-<timestamp>` room. Prune devices/rooms occasionally.
The `e2e` job is `continue-on-error: true` for now because `playwright install --with-deps` needs `apt` on the runner image — promote it to a hard gate once it is green on the runner. Locally: `npm run test:e2e` (boot tier only), or `E2E_HOMESERVER=… E2E_USER=… E2E_PASSWORD=… npm run test:e2e` for both; on failure look in `test-results/` (screenshot + trace) and `playwright-report/`.
---
---
### Green CI but the fix isn't live?
`curl -s https://chat.lotusguild.org/index.html | grep -o 'assets/index-[^"]*\.js'` gives the deployed entry chunk; grep it for a string unique to your change (`curl -s https://chat.lotusguild.org/<that path> | grep -c <string>`). If it's 0 after ~15 min, the deploy trigger was lost — push again (any commit) to re-fire the `lotus-deploy` webhook. The deploy log lives at `/var/log/lotus-deploy.log` on LXC 106.