LotusGuild/cinny
3
0
Fork 0
Code Issues 27 Pull Requests 5 Actions Packages Projects Releases Wiki Activity

[research] Search operators: from:, has:, in:, before:/after: — audit what exists first #106

New Issue
Closed
opened 2026-09-17 01:39:34 -04:00 by jared · 2 comments
jared commented 2026-09-17 01:39:34 -04:00
Owner
Copy Link
Copy Source

The local encrypted search has a date range; we believe it lacks from:@user, has:image|file|link|video, in:#room, before:/after: operators, but this needs verifying before building.

Research first:

  1. Inventory the current search surface: src/app/features/message-search, the encrypted search cache (src/app/utils/searchCache.ts) and the server-side /search path — which filters each already supports (sender, room, date, content type) and how they are exposed in the UI.
  2. Check upstream Cinny 4.12.x for filter UI we already inherit.
  3. Propose: operators typed in the search box (Discord-style, with autocomplete chips) AND equivalent filter buttons, both mapping to the same query object; which operators can be answered from the local index vs. need the server.
    Deliverable: findings + a plan in this issue; implementation as a follow-up.
The local encrypted search has a date range; we believe it lacks `from:@user`, `has:image|file|link|video`, `in:#room`, `before:`/`after:` operators, but this needs verifying before building. **Research first:** 1. Inventory the current search surface: `src/app/features/message-search`, the encrypted search cache (`src/app/utils/searchCache.ts`) and the server-side `/search` path — which filters each already supports (sender, room, date, content type) and how they are exposed in the UI. 2. Check upstream Cinny 4.12.x for filter UI we already inherit. 3. Propose: operators typed in the search box (Discord-style, with autocomplete chips) AND equivalent filter buttons, both mapping to the same query object; which operators can be answered from the local index vs. need the server. Deliverable: findings + a plan in this issue; implementation as a follow-up.
search-raid.png
757 KiB
search-raid.png
jared added this to the Features 2026-Q4 milestone 2026-09-17 01:39:34 -04:00
jared added the enhancementpriority: mediumarea: messaging labels 2026-09-17 01:39:34 -04:00
jared self-assigned this 2026-09-17 01:39:34 -04:00
jared commented 2026-09-20 15:08:10 -04:00
Author
Owner
Copy Link
Copy Source

Inventory (2026-09-20)

What exists today

filter typed operator UI control server /search local encrypted index
sender from:@user — parsed in SearchInput.tsx:28-31 (FROM_REGEX), with member autocomplete; sender-only mode when there's no body text (MessageSearch.tsx:269-272) chips in SearchFilters.tsx:480-523 filter.senders (useMessageSearch.ts:172) SearchCacheRow.sender (searchCache.ts:37) — yes
room none room picker + "all rooms" (global URL param, MessageSearch.tsx:60/230) filter.rooms rows are per room — yes
date none DateRangeButton (SearchFilters.tsx:536-649) not a spec filter field — enforced client-side after the fact (filterGroupsByDateRange, useMessageSearch.ts:85; comment at :169-171) row.ts — yes, cheap
has link none containsUrl toggle (SearchFilters.tsx:675, URL param containsUrl) filter.contains_url (spec field, :175) no field; would need body URL sniffing
has image / file / video none none /search has no msgtype filter; server FTS indexes body (filenames), so has:image can't be expressed index has no msgtype (SearchCacheRow = roomId, eventId, ts, sender, body, formattedBody, pollText)
ordering none recent / rank toggle order_by n/a

Upstream Cinny 4.12 inheritance: the room/global picker, sender chips, order toggle. from: typing, the date range and containsUrl are Lotus additions. So the belief in the issue is half right: from: exists (typed + chips); in:, before:/after:, has: do not exist as operators, and has:image|file|video has no backing on either side.

Proposal (one query object, two entry points)

parseSearchQuery(text) → { term, senders[], rooms[], fromTs?, toTs?, has: Set<'link'|'image'|'video'|'file'> } (pure, unit-tested) fed by both the typed box (Discord-style chips rendered for recognised operators, autocomplete for from: members and in: room names, before:/after: accepting YYYY-MM-DD and 7d/2w) and the existing filter buttons, which just write into the same object; the URL params stay the serialisation.

Where each is answered:

  • from:, in:, before:/after: — server (senders, rooms, client-side date trim as now) and local index (already has sender/room/ts). Cheap: ~150 lines, mostly the parser + chip rendering.
  • has:link — server contains_url; local: URL regex over body at query time (fine at index sizes we have).
  • has:image|video|file — needs a msgtype column in SearchCacheRow (bump the index schema version; rows re-fill on the next room open) for the local path; server side is impossible with /search as specified, so in unencrypted rooms it would silently degrade to a local-only answer with a "cached messages only" note (the UI already has that vocabulary, MessageSearch.tsx:617/715). Recommend shipping has:link first and has:image|video|file as local-only with that caveat.

Order of work if approved: parser + chips (1), in: + before:/after: wiring (2), has:link (3), msgtype column + has:image|video|file local-only (4). Decision needed on whether a local-only operator with a caveat is acceptable or whether has: should wait for server-side support (it won't come — the spec has no such filter).

## Inventory (2026-09-20) ### What exists today | filter | typed operator | UI control | server `/search` | local encrypted index | |---|---|---|---|---| | sender | **`from:@user`** — parsed in `SearchInput.tsx:28-31` (`FROM_REGEX`), with member autocomplete; sender-only mode when there's no body text (`MessageSearch.tsx:269-272`) | chips in `SearchFilters.tsx:480-523` | `filter.senders` (`useMessageSearch.ts:172`) | `SearchCacheRow.sender` (`searchCache.ts:37`) — yes | | room | none | room picker + "all rooms" (`global` URL param, `MessageSearch.tsx:60/230`) | `filter.rooms` | rows are per room — yes | | date | none | `DateRangeButton` (`SearchFilters.tsx:536-649`) | **not a spec filter field** — enforced client-side after the fact (`filterGroupsByDateRange`, `useMessageSearch.ts:85`; comment at :169-171) | `row.ts` — yes, cheap | | has link | none | `containsUrl` toggle (`SearchFilters.tsx:675`, URL param `containsUrl`) | `filter.contains_url` (spec field, :175) | no field; would need `body` URL sniffing | | has image / file / video | none | none | `/search` has no msgtype filter; server FTS indexes `body` (filenames), so `has:image` can't be expressed | index has **no msgtype** (`SearchCacheRow` = roomId, eventId, ts, sender, body, formattedBody, pollText) | | ordering | none | recent / rank toggle | `order_by` | n/a | Upstream Cinny 4.12 inheritance: the room/global picker, sender chips, order toggle. `from:` typing, the date range and `containsUrl` are Lotus additions. So the belief in the issue is half right: `from:` exists (typed + chips); `in:`, `before:`/`after:`, `has:` do not exist as operators, and `has:image|file|video` has no backing on either side. ### Proposal (one query object, two entry points) `parseSearchQuery(text) → { term, senders[], rooms[], fromTs?, toTs?, has: Set<'link'|'image'|'video'|'file'> }` (pure, unit-tested) fed by both the typed box (Discord-style chips rendered for recognised operators, autocomplete for `from:` members and `in:` room names, `before:`/`after:` accepting `YYYY-MM-DD` and `7d`/`2w`) and the existing filter buttons, which just write into the same object; the URL params stay the serialisation. Where each is answered: - `from:`, `in:`, `before:`/`after:` — server (`senders`, `rooms`, client-side date trim as now) **and** local index (already has sender/room/ts). Cheap: ~150 lines, mostly the parser + chip rendering. - `has:link` — server `contains_url`; local: URL regex over `body` at query time (fine at index sizes we have). - `has:image|video|file` — needs a `msgtype` column in `SearchCacheRow` (bump the index schema version; rows re-fill on the next room open) for the local path; server side is **impossible** with `/search` as specified, so in unencrypted rooms it would silently degrade to a local-only answer with a "cached messages only" note (the UI already has that vocabulary, `MessageSearch.tsx:617/715`). Recommend shipping `has:link` first and `has:image|video|file` as local-only with that caveat. Order of work if approved: parser + chips (1), `in:` + `before:`/`after:` wiring (2), `has:link` (3), msgtype column + `has:image|video|file` local-only (4). Decision needed on whether a local-only operator with a caveat is acceptable or whether `has:` should wait for server-side support (it won't come — the spec has no such filter).
jared added the needs-human-review label 2026-09-20 15:08:11 -04:00
jared commented 2026-09-25 12:54:27 -04:00
Author
Owner
Copy Link
Copy Source

Built (75951f90)

typed sets
from:@user / from:bob sender chip. A bare name now resolves to a known @bob:server, this homeserver first. Before, it sent @bob and matched nobody unless you picked from the autocomplete.
in:general, in:"voice lounge" the room filter, matched by name, alias, alias localpart, room id, or a unique partial name. A room outside the page's own scope switches the search to all rooms.
after:2026-09-01, before:7d the date range. Relative units are h, d, w, m (30 d) and y (365 d).
has:link Contains link (server contains_url)
has:image / video / file the existing Images / Video / Files filters. These already filter the results client-side, so no new index was needed and there's no extra caveat.
is:pinned the existing Pinned filter
  • Parsing (searchOperators.ts, pure, 10 unit tests):
    • Keys are case-insensitive and quoted values are supported.
    • Operators are only recognised at the start of a word, so a URL or a time isn't mistaken for one.
    • Recognised operators are removed from the box and appear as filters in the bar.
  • Anything not understood (has:gif, after:someday) stays in the searched text, with a notice listing the valid operators.
  • Other notices: "No room matches in:x"; operators with no words and no from: give "Add a word to search for, or a from:@user.", and the text stays in the box.
  • Empty state: the hint now lists the operators.

Verified in Chromium on a local Synapse:

  • raid in:"Thread Lab" from:bob sets rooms to Thread Lab and senders to @bob:localhost, and finds Bob's message (screenshot).
  • The date, unknown-room, operators-only and has:gif cases give the expected filters and notices.

search

## Built (`75951f90`) | typed | sets | |---|---| | `from:@user` / `from:bob` | sender chip. **A bare name now resolves to a known `@bob:server`, this homeserver first.** Before, it sent `@bob` and matched nobody unless you picked from the autocomplete. | | `in:general`, `in:"voice lounge"` | the room filter, matched by name, alias, alias localpart, room id, or a unique partial name. A room outside the page's own scope switches the search to all rooms. | | `after:2026-09-01`, `before:7d` | the date range. Relative units are h, d, w, m (30 d) and y (365 d). | | `has:link` | Contains link (server `contains_url`) | | `has:image` / `video` / `file` | the existing Images / Video / Files filters. These already filter the results client-side, so no new index was needed and there's no extra caveat. | | `is:pinned` | the existing Pinned filter | - **Parsing** (`searchOperators.ts`, pure, 10 unit tests): - Keys are case-insensitive and quoted values are supported. - Operators are only recognised at the start of a word, so a URL or a time isn't mistaken for one. - Recognised operators are removed from the box and appear as filters in the bar. - **Anything not understood** (`has:gif`, `after:someday`) stays in the searched text, with a notice listing the valid operators. - **Other notices:** "No room matches in:x"; operators with no words and no `from:` give "Add a word to search for, or a from:@user.", and the text stays in the box. - **Empty state:** the hint now lists the operators. **Verified** in Chromium on a local Synapse: - `raid in:"Thread Lab" from:bob` sets rooms to Thread Lab and senders to `@bob:localhost`, and finds Bob's message (screenshot). - The date, unknown-room, operators-only and `has:gif` cases give the expected filters and notices. ![search](https://code.lotusguild.org/attachments/bbf5c87b-69d8-40e7-aa3c-15ab3f42b6f5)
jared closed this issue 2026-09-25 12:54:28 -04:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
a11y
Accessibility: keyboard, screen reader, contrast, motion
 area: appearance
Client area: appearance
 area: auth-session
Client area: auth-session
 area: build-ci
Client area: build-ci
 area: calls
Client area: calls
 area: desktop
Client area: desktop
 area: media
Client area: media
 area: messaging
Client area: messaging
 area: mobile
Client area: mobile
 area: moderation
Client area: moderation
 area: navigation
Client area: navigation
 area: notifications
Client area: notifications
 area: settings
Client area: settings
 area: threads
Client area: threads
 bug
Something is not working
 dependencies
Third-party package versions and advisories
 docs
README / LOTUS_* docs wrong or missing
 duplicate
This issue or pull request already exists
 enhancement
New feature
 help wanted
Need some help
 invalid
Something is wrong
 needs-human-review
performance
Re-render storms, leaks, heavy work on hot paths
 planning
priority: critical
Data loss, security hole, or crash on a main path
 priority: high
Broken feature or serious usability problem
 priority: low
Minor issue or polish
 priority: medium
Wrong behaviour in an edge case or notable degradation
 qa
Manual QA: shipped, needs a human in a real environment
 question
More information is needed
 research
security
XSS, unsafe URLs, data leaks, auth/session
 tech-debt
Code health, dead code, fragile patterns
 ux
Usability or visual inconsistency
 wontfix
This won't be fixed
No labels area: messaging enhancement needs-human-review priority: medium
Milestone
No items
No Milestone Features 2026-Q4
Projects
Clear projects
No projects
Assignees
Clear assignees
cole (Cole Keller) jared (Jared Vititoe) nathan (Nathan Vititoe)
No Assignees jared
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: LotusGuild/cinny#106