Incident 2026-09-29: the homeserver's host ran out of memory and stalled for
~2 minutes. The /sync that finally went out carried events whose `age` was
computed ~30 s before it arrived, so every client showed "Your computer's
clock is 30 seconds ahead of the server" while the real problem was the
server (all host clocks were within 0.25 s the whole evening).
The skew estimate was the median of the last 5 samples, and a sample is
local skew + delivery delay, so one late /sync with a handful of events
tripped it.
- Estimate = the LOWEST sample of the last 5 minutes: delay only ever adds,
so the fastest-delivered event is the truest.
- "Behind" (which a delay can't cause) is reported as soon as there are 3
samples, like before. "Ahead" must hold across samples received at least
a minute apart, so a single late burst never trips it.
- Samples are aged on the monotonic clock, and a change of the local clock
(someone fixing it) resets the measurement, so the warning clears at once.
- Only events stamped by our own homeserver are sampled: a federated event's
origin_server_ts is the other server's clock.
- Wording: "This device's clock is … Voice calls and encrypted messages can
fail until it's corrected." / call bar "Device clock … : calls may fail"
(was "will fail").
Unit tests: the incident (late burst after normal traffic, and a fresh
client whose first samples are all late), mixed slow/fast deliveries,
ahead only after a minute, behind at once, hysteresis, clock fixed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA
Incident 2026-09-17: a wrong Windows clock broke calls and media keys while
the server answered 200 to everything, with no hint in the UI.
Measurement needs no extra requests and no CORS-exposed headers: every live
event carries origin_server_ts and unsigned.age (our server's now − ts at
response time), so localTimestamp − origin_server_ts is the skew. Only
RoomEvent.Timeline live events count (cache replays have stale age and are
already flagged liveEvent=false by the SDK); the initial network sync
qualifies, so a wrong clock is flagged within seconds of startup. Median of
the last 5 samples, ≥3 needed; warn at |skew| > 30 s, clear below 15 s.
UI: a banner in the sync-status slot — "Your computer's clock is 14 minutes
ahead of the server. Encrypted messages and voice calls will fail until it is
fixed." with a per-OS How-to-fix hint and Dismiss for 24 h — plus the same
line in the call status bar while in a call. Never auto-corrects anything.
Unit-tested (median, hysteresis, stale-age rejection, wording); verified
headless with Playwright's clock skewed +14 min and −3 h (banner, hint,
in-call line, dismiss) and in sync (nothing shown).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPmy3tPq869XDW4njjVaKA