[research] Offline outbox — what happens today when a send fails or the client is offline #112
Open
opened 2026-09-17 13:27:07 -04:00 by jared
·
2 comments
No Branch/Tag Specified
lotus
update-packages
sw-fix
read-me-update
image-path-changes
dm-calls
fix-2469
renovate/element-hq-element-call-embedded-0.x
renovate/npm-i18next-http-backend-vulnerability
renovate/npm-vite-vulnerability
dev
docs-update
more-theme
fix-257
imporve-thread-reply
revert-2402-improve-menu-congestion
mxidColor-toggle
update-sw-main-msg
v4.11.1
v4.10.5
v4.10.4
v4.10.3
v4.10.2
v4.10.1
v4.10.0
v4.9.1
v4.9.0
v4.8.1
v4.8.0
v4.7.1
v4.7.0
v4.6.0
v4.5.1
v4.5.0
v4.4.0
v4.3.2
v4.3.0
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.0
v4.0.3
v4.0.0
v3.2.0
v3.1.0
v3.0.0
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v2.0.0
v1.8.2
v1.8.1
v1.8.0
v1.7.0
v1.6.1
v1.6.0
v1.5.1
v1.5.0
v1.4.0
v1.3.2
v1.3.1
v1.3.0
v1.2.1
v1.2.0
v1.1.0
v1.0.0
Labels
Clear labels
a11y
area: appearance
area: auth-session
area: build-ci
area: calls
area: desktop
area: media
area: messaging
area: mobile
area: moderation
area: navigation
area: notifications
area: settings
area: threads
bug
dependencies
docs
duplicate
enhancement
help wanted
invalid
needs-human-review
performance
planning
priority: critical
priority: high
priority: low
priority: medium
qa
question
research
security
tech-debt
ux
wontfix
Accessibility: keyboard, screen reader, contrast, motion
Client area: appearance
Client area: auth-session
Client area: build-ci
Client area: calls
Client area: desktop
Client area: media
Client area: messaging
Client area: mobile
Client area: moderation
Client area: navigation
Client area: notifications
Client area: settings
Client area: threads
Something is not working
Third-party package versions and advisories
README / LOTUS_* docs wrong or missing
This issue or pull request already exists
New feature
Need some help
Something is wrong
Re-render storms, leaks, heavy work on hot paths
Data loss, security hole, or crash on a main path
Broken feature or serious usability problem
Minor issue or polish
Wrong behaviour in an edge case or notable degradation
Manual QA: shipped, needs a human in a real environment
More information is needed
XSS, unsafe URLs, data leaks, auth/session
Code health, dead code, fragile patterns
Usability or visual inconsistency
This won't be fixed
Milestone
No items
No Milestone
Features 2026-Q4
Projects
Clear projects
No projects
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: LotusGuild/cinny#112
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Before deciding anything: establish exactly what the client does now.
Questions to answer (with file:line evidence)
EventStatus.QUEUED/SENDING/NOT_SENT) and retry automatically on reconnect, or does it fail fast? What does Lotus render for each status (Message.tsx,ThreadTimeline.tsxreferenceNOT_SENT)?pendingEventOrdering, IndexedDB store)? Are drafts and pending events separate?tryDeleteMxcContent)?SyncStatusshow, and does it block sending?Deliverable
A short state-of-play write-up in this issue with a recommendation: nothing / small UX polish (visible "queued — will send when online", retry-all) / real outbox work. Decision is Jared's.
Research: send-failure / offline behavior in Lotus (matrix-js-sdk 41.7.0)
I read
node_modules/matrix-js-sdk(src) and Lotus'ssrc/appdirectly. Summary per question, with file:line evidence.1. Sending while offline — queue+retry or fail fast?
Fail fast, no automatic retry.
MatrixScheduler(node_modules/matrix-js-sdk/src/scheduler.ts:44-302, withRETRY_BACKOFF_RATELIMITat line 51 andQUEUE_MESSAGESat line 61) implements exactly the queue/backoff/retry machinery the issue describes — but it is only wired up if ascheduleris passed intocreateClient()(client.ts:1374-1389,client.ts:2944-2955). Lotus'sinitMatrix.ts:59-74callscreateClient({...})without ascheduleroption, sothis.schedulerisundefinedand the scheduler code (including all retry/backoff) is dead code in this app.Without a scheduler,
encryptAndSendEvent(client.ts:2894-2984) goes straight tosendEventHttpRequest(line 2957-2965). Any failure (including a network error while offline) is caught atclient.ts:2968-2982, which setsevent.errorand callsupdatePendingEventStatus(room, event, EventStatus.NOT_SENT)— immediately, on the first failure, with no retry and no reconnect-triggered resend anywhere in the SDK or in Lotus's own code (confirmed by grep:resendEventis only ever called from the "Retry Send" context-menu handler,src/app/features/room/message/Message.tsx:1397).Lotus does render all the statuses:
src/app/features/room/message/Message.tsx:99-150(DeliveryStatus) mapsSENDING/ENCRYPTING→ "Sending..." spinner,QUEUED→ "Queued" (never reachable here since there's no scheduler),NOT_SENT/CANCELLED→ red cross "Failed to send", anything else → "Sent" check.src/app/features/room/thread/ThreadTimeline.tsx:930-948shows a "Failed to send" caption under thread replies withNOT_SENT/CANCELLEDstatus.RoomTimeline.tsxhas no bespoke pending/failed handling — pending/failed own-messages flow through the sameMessage.tsxrender path and get the sameDeliveryStatusicon (main timeline usesPendingEventOrdering.Chronological, so pending events live directly in the live timeline, not a side list — see Q3).2. Manual resend / cancel — is it discoverable?
Yes, but it's easy to miss.
Message.tsx:1386-1431: whenmEvent.statusisNOT_SENTorCANCELLED, the message's hover "..." context menu grows an extra group with "Retry Send" (callsmx.resendEvent(mEvent, room), line 1397) and "Cancel Message" (callsmx.cancelPendingEvent(mEvent), line 1416). There is no standalone "failed to send, tap to retry" affordance and no global "you have N failed messages" indicator — the only visible cue is the small red cross icon next to the message (DeliveryStatus,Message.tsx:111-114); a user has to notice the icon, hover the message, open the menu, and find the item at the bottom.3. Does the pending-event store survive reload? Are drafts separate from pending events?
No, pending/failed events do not survive a reload, and yes, drafts are a completely separate mechanism.
Roomonly populates/persistspendingEventList(and thus calls the store'ssetPendingEvents) when constructed withpendingEventOrdering: Detached(node_modules/matrix-js-sdk/src/models/room.ts:493,512-524,savePendingEvents()guarded byif (this.pendingEventList)atroom.ts~2849-2865). Restoring persisted pending events fromclient.store.getPendingEvents()also only happens in that sameDetachedbranch (room.ts:512-522).pendingEventOrderingwhen creating the client (src/client/initMatrix.ts:59-74), andsync.ts:1965-1969in the SDK propagatesopts.pendingEventOrdering(undefined here) straight intonew Room(...), so Lotus rooms default toPendingEventOrdering.Chronological(room.ts:493). Net effect:pendingEventListis never allocated,IndexedDBStore.setPendingEvents/getPendingEvents(node_modules/matrix-js-sdk/src/store/indexeddb.ts:353-368) are never invoked, and aNOT_SENT/still-SENDINGlocal echo is pure in-memory state — a page reload silently drops it. The user's typed text is gone from the timeline as if never typed (no persisted "failed message" to retry after reload).src/app/utils/draft.ts:1-19definesDRAFT_MSG_KEY_PREFIX = 'draft-msg-';RoomInput.tsx:437,477reads/writeslocalStorage['draft-msg-<roomId>'](JSON{userId, nodes}) as the user types, independent of send state;src/app/hooks/useHydrateMsgDrafts.tsrehydrates these into atoms on boot for the room-nav "has draft" dot. Drafts are composer text that was never submitted; pending events are messages that were submitted and are in flight or failed. The two never interact.4. Media uploads mid-flight when network drops — retried or orphaned? mxc cleanup?
Uploads have their own retry loop, independent of the (unused) SDK scheduler:
src/app/utils/matrix.ts:193-274(uploadContent) retries up toUPLOAD_MAX_RETRY_COUNT = 3(line 191) with capped exponential backoff or the server'sRetry-After(lines 257-263), but only forisRetryableUploadErrorcases — no HTTP status (network/transport failure), 408, 429, or 5xx (matrix.ts:174-189); a user-drivenAbortErrorfrommx.cancelUpload()is explicitly excluded from retry (line 179). After exhausting retries it surfacesUploadStatus.Error, andUploadCardRenderer.tsx:332-341renders a manual "Retry Upload" button that callsstartUploadagain.mxc cleanup:
tryDeleteMxcContent(src/app/utils/matrix.ts:532-544) is a best-effortDELETE /_matrix/client/v1/media/...call (Synapse 1.97+ media-owner delete), invoked fromUploadCardRenderer.tsx:311-318(removeUpload, only if the upload had already reachedSuccessbefore the user removes it) andRoomInput.tsx:591(after image compression replaces an already-uploaded original). If an upload fails outright (never reachedSuccess), there is no mxc to delete — nothing was orphaned server-side. So: uploads auto-retry a few times, then require a manual retry click; content is only explicitly deleted when an already-successful upload is discarded, not when a failed one is abandoned (there's nothing to clean up in that case).5. What does SyncStatus show, and does it block sending?
src/app/pages/client/SyncStatus.tsx:33-92renders a thin banner keyed offSyncStatefromuseSyncState: "Connecting..." forPrepared/Syncing/Catchup(line 34-53), "Connection Lost! Reconnecting..." forReconnecting(line 56-71), "Connection Lost!" forError(line 74-89). It is purely informational and never touches the composer —RoomInput.tsxhas no reference toSyncState/useSyncStateand never disables its send button or input based on connectivity (confirmed by grep). A user can type and hit send while fully offline; the message goes toSENDINGthen immediatelyNOT_SENTper Q1.6. Encrypted rooms — is a retried pending event re-encrypted correctly?
Yes, and correctly so: it is not re-encrypted — it is resent as-is.
resendEvent()(node_modules/matrix-js-sdk/src/client.ts:2501-2507) sets status back toSENDINGand callsencryptAndSendEventagain, which callsencryptEventIfNeeded→shouldEncryptEventForRoom(client.ts:3006-3013):if (event.isEncrypted()) return false;with the comment "this happens if the encryption step succeeded, but the send step failed on the first attempt." So a message whose encryption already completed keeps its original ciphertext (from whatever Megolm session was active at encrypt time) and only the HTTP send is retried — there's no double-encryption, no session-mismatch risk, and no plaintext re-exposure. This is core SDK behavior (not anything Lotus added/broke).Recommendation
Small UX polish, not a real outbox rewrite. The dangerous gap here isn't missing retry logic — matrix-js-sdk's scheduler intentionally isn't the right tool (rooms would serialize all messages FIFO per-room, which nobody wants) — it's that failed/in-flight sends are silently lost on reload (Q3) with only a small red icon (Q1/Q2) as the only sign anything went wrong, and the composer gives zero feedback that you're offline (Q5) beyond a thin top banner that's easy to miss while scrolled into a room. I'd suggest: (a) surface a lightweight "N messages failed to send — Retry all / Dismiss" affordance instead of requiring per-message menu digging, and (b) when the client is offline (
SyncState.Error/Reconnecting), show a small inline hint inRoomInputitself (not just the top-of-timeline banner) so users don't type into a void. Persisting pending events across reload (switching topendingEventOrdering: Detachedand wiringIndexedDBStore's existingsetPendingEvents/getPendingEvents) would fix the reload-loses-your-message case for real, but it's a bigger, riskier change (touches every timeline/pending-event code path) that I'd scope separately. Estimated effort: ~0.5-1 day for the UX polish (failed-message banner + composer offline hint, reusing existingEventStatus/SyncStateplumbing); ~3-5 days for theDetachedpending-event persistence work plus regression testing across normal/thread/encrypted timelines.State of play (2026-09-20, code audit with file:line)
1. Sending while offline.
mx.sendMessage(...)(RoomInput.tsx:317/356/365/613) goes through the SDK'sMatrixSchedulerwith the defaultRETRY_BACKOFF_RATELIMIT→calculateRetryBackoff(err, attempts, /*retryConnectionError*/ false)(matrix-js-sdk/lib/scheduler.js:42,http-api/utils.js:155-161). AConnectionError(offline / DNS / TCP) returns −1 = give up immediately; only 429 and 5xx get up to 4 exponential retries. So an offline send fails fast: local echo →SENDING→NOT_SENTwithin one request timeout. Nothing retries on reconnect. Lotus renders each status inMessage.tsx:108-130:QUEUED"Queued",SENDING/ENCRYPTING"Sending...",NOT_SENT/CANCELLEDred ✕ "Failed to send" (andThreadTimeline.tsx:961mirrors it).2. Manual affordance. Only in the ⋮ message menu, and only for
NOT_SENT/CANCELLEDevents: "Retry Send" →mx.resendEventand "Cancel Message" →mx.cancelPendingEvent(Message.tsx:1277-1310). Not discoverable: the ✕ icon itself is not a button, there is no "retry all", and on touch the menu is the long-press sheet.3. Persistence across reload.
pendingEventOrderingis not set (initMatrix.ts:94-101) → SDK defaultChronological(models/room.js:334). Pending events are only persisted to the store (store.setPendingEvents,room.js:2450-2461) inDetachedmode; in Chronological mode a reload drops every unsent message — it vanishes from the timeline with no trace. Drafts are separate (roomIdToMsgDraftAtomFamily+draft-msg-*localStorage, #37/#41) and do survive; a failed send is not put back into the draft.4. Media uploads.
uploadContent(utils/matrix.ts:193-270) retries transient failures up to 3× with Retry-After/backoff, and the upload card's cancel callstryDeleteMxcContent(UploadCardRenderer.tsx:315,utils/matrix.ts:553). Once the upload succeeded and the event send fails, the mxc stays on the server (orphaned until media retention) — same as Element.5. SyncStatus.
SyncStatus.tsx:49/67/85shows "Connecting…", "Connection Lost! Reconnecting…", "Connection Lost!". It does not block sending — the composer stays live, so an offline user sends into a fail-fast queue.6. Encrypted rooms on retry.
resendEventre-runs the send pipeline from the pending event's status; forENCRYPTING/NOT_SENTthe SDK re-encrypts with the current outbound session (client.jsencryptAndSendEventpath —NOT_SENTis set atclient.js:1983after encryption fails or the request fails). No known Lotus-side bug here; the risk is only the KE-1 key-cluster issues (#201).Recommendation
Middle option — small UX work, no real outbox:
pendingEventOrdering: Detachedso unsent messages survive a reload (they then render at the bottom, "Failed to send", with the existing Retry/Cancel). One line + a check that the timeline handles the detached pending list (room.getPendingEvents()), which upstream Cinny supports.ClientEvent.Sync→PREPARED/SYNCINGafterRECONNECTING/ERROR, walk each room'sNOT_SENTpending events andresendEventthem once, oldest first (aretryFailedSends()util, unit-tested). Show "Queued — will send when online" instead of the red ✕ while the client is offline (navigator.onLine === falseor sync stateRECONNECTING), red ✕ only once online and still failing.That's ~200 lines and covers the "I typed on the train" case without building a durable outbox. A real outbox (send while the app is closed, ordering guarantees across devices) needs the SW + push plumbing from #202 and isn't worth it yet. Decision is Jared's.