Compare commits

...
1 Commits
Author SHA1 Message Date
jaredandClaude Opus 5.5 c2ff072eb2 Restart modem button: silence Gandalf and Uptime Kuma, restart, end both
Lint / Python (flake8) (push) Failing after 15s
Lint / JS (eslint) (push) Successful in 1m43s
Lint / Deploy (push) Skipped
Test / Python Tests (pytest) (push) Failing after 21s
Security / Python Security (bandit) (push) Successful in 2m3s
Lint / Python (flake8) (pull_request) Failing after 15s
Lint / JS (eslint) (pull_request) Failing after 15s
Lint / Notify on failure (pull_request) Skipped
Lint / Deploy (pull_request) Skipped
Security / Python Security (bandit) (pull_request) Successful in 2m17s
Lint / Notify on failure (push) Successful in 3s
Test / Python Tests (pytest) (pull_request) Successful in 2m58s
Admins get a Restart modem button on the WAN / Fiber panel (needs
modem.access_code). After a confirmation it:

1. switches an Uptime Kuma maintenance window on (new kuma.py: logs in
   over Kuma 2's Socket.IO API and resumes/pauses one pre-made manual
   window, uptime_kuma.maintenance_id); if Kuma fails it stops and asks
   whether to go on without it,
2. starts Gandalf maintenance (global suppression, 20 minutes at most),
3. posts the gateway's Restart Device form (ModemAdmin.restart), undoing
   1 and 2 if that fails.

The monitor waits for the modem to answer with its WAN up and an uptime
showing it restarted, then switches Kuma off, ends the suppression and
records how long it took; after 30 minutes it ends both regardless. The
panel shows the restart in progress and the result for an hour.

New dependency: python-socketio[client].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 19:27:25 -05:00
13 changed files with 488 additions and 10 deletions

No files matched your search

+13 -1
View File
@@ -532,6 +532,16 @@ for read-only checks, stored in `monitor_state.modem_checks`:
- **The gateway's own speed tests** (AT&T runs them from the modem), read with the NAT check; the latest
down and up results are shown on the panel as a reference for what the line itself delivers.
**Restart modem** (admins, WAN / Fiber panel, needs `modem.access_code`): after a confirmation it
1. switches the `uptime_kuma` maintenance window on (if that fails it stops and asks whether to go on
without it),
2. starts Gandalf maintenance (a global suppression for up to 20 minutes, shown in the banner),
3. posts the gateway's Restart Device form. If that fails, steps 1 and 2 are undone.
The monitor then waits for the gateway to answer with its WAN up and an uptime showing it restarted
(at least 3 minutes later), switches the Kuma window off, ends the suppression, and records how long it
took (`monitor_state.modem_restart`). After 30 minutes it ends both regardless. One restart at a time.
The login posts md5(access code + nonce), as the gateway's own page does, so the code itself is never sent.
The session is kept between checks and renewed when it expires. A refused code is reported in Needs
attention and not tried again for 30 minutes, and any other failure pauses the checks for 10 minutes, so
@@ -680,6 +690,7 @@ Shared by both processes. Located in the working directory (`/var/www/html/prod/
| `modem.failure_threshold` | Consecutive failures before ticketing a modem alert (default: same as `failure_threshold`). Raise it if gateway reboots or automatic firmware updates cut tickets. |
| `modem.rx_drift_db` | Rx drop below the 7-day median that raises `fiber_rx_drift` (default: 3.0) |
| `modem.rx_drift_min_samples` | Readings required before drift alerts start (default: 288) |
| `uptime_kuma.url`, `.username`, `.password`, `.maintenance_id` | Optional. Lets the Restart modem button switch an Uptime Kuma maintenance window on and off (Kuma 2.x, over its Socket.IO login). Create the window once in Kuma with the *manual* strategy and the monitors a modem restart takes down, pause it, and put its id here. The account must not use two-factor login. |
| `modem.access_code` | Optional. The gateway's Device Access Code (printed on its label). With it, Gandalf logs in to read NAT session counts and to ping from the gateway when the internet is slow (see WAN modem notes). Keep it only in `config.json`, which is git-ignored and mode 600. |
---
@@ -865,7 +876,8 @@ curl http://10.10.10.48:9090/api/v1/query?query=up | jq '.data.result[] | select
gandalf/
├── app.py # Flask web app (routes, auth, API endpoints)
├── monitor.py # Background daemon (Prometheus, UniFi, Pulse, alert logic)
├── modem.py # AT&T gateway client: status pages, plus optional logged-in NAT and ping checks
├── kuma.py # Switches one Uptime Kuma maintenance window on/off (Restart modem)
├── modem.py # AT&T gateway client: status pages, plus optional logged-in checks and restart
├── devices.py # Device inventory for search and device pages
├── topology.py # The topology graph: devices, cables, depth, last-seen links
├── cutsheet.py # Cutsheet validation, path tracing, plan-vs-actual checks, CSV
+63
View File
@@ -27,6 +27,8 @@ import cutsheet
import db
import devices
import diagnose
import kuma
import modem as modem_api
import topology
from monitor import PulseClient, UnifiClient
@@ -557,10 +559,71 @@ def api_modem():
data['wan_history'] = _state_json('wan_history')
data['wan_quality'] = _state_json('wan_quality') or None
data['modem_checks'] = _state_json('modem_checks') or None
data['modem_restart'] = _state_json('modem_restart') or None
data['can_restart'] = bool((_config().get('modem') or {}).get('access_code'))
data['configured'] = True
return jsonify(data)
# How long the silence lasts if the monitor never sees the modem come back to end it sooner
MODEM_RESTART_SILENCE_MIN = 20
@app.route('/api/modem/restart', methods=['POST'])
@require_auth
@require_admin
def api_modem_restart():
"""Restart the AT&T gateway, with alerts silenced first.
Body: {"confirm": true} and, after a Kuma failure the admin chose to accept,
"without_kuma": true. Order: Uptime Kuma maintenance on, Gandalf maintenance on, then
the restart; the first two are undone if the restart can't be sent. The monitor ends
both once the modem is back (monitor._finish_modem_restart).
"""
body = _json_body()
if body.get('confirm') is not True:
return jsonify({'error': 'confirm must be true'}), 400
cfg = _config()
modem_cfg = cfg.get('modem') or {}
if not modem_cfg.get('url') or not modem_cfg.get('access_code'):
return jsonify({'error': 'Set modem.access_code in config.json first'}), 400
pending = _state_json('modem_restart')
if pending and not pending.get('finished_at') and time.time() - pending.get('at', 0) < 1800:
return jsonify({'error': 'A restart is already in progress'}), 409
user = _get_user()['username']
kuma_cfg = cfg.get('uptime_kuma') or {}
kuma_on = False
if kuma.configured(kuma_cfg):
try:
kuma.set_maintenance(kuma_cfg, True)
kuma_on = True
except kuma.KumaError as e:
logger.warning(f'Uptime Kuma maintenance could not be started: {e}')
if not body.get('without_kuma'):
return jsonify({'error': f'Uptime Kuma: {e}', 'kuma_failed': True}), 502
sup_id = db.create_suppression('all', '', '', 'Modem restart from Gandalf', user,
MODEM_RESTART_SILENCE_MIN)
try:
result = modem_api.ModemAdmin(modem_cfg).restart()
except (modem_api.ModemUnreachable, modem_api.ModemDataInvalid, modem_api.ModemLoginFailed) as e:
logger.error(f'Modem restart failed: {e}')
db.deactivate_suppression(sup_id)
if kuma_on:
try:
kuma.set_maintenance(kuma_cfg, False)
except kuma.KumaError as ke:
logger.error(f'Uptime Kuma maintenance could not be ended: {ke}')
return jsonify({'error': f'The modem did not restart: {e}'}), 502
db.set_state('modem_restart', {'at': time.time(), 'by': user, 'result': result,
'suppression_id': sup_id, 'kuma': kuma_on, 'finished_at': None})
logger.warning(f'Modem restart requested by {user} ({result}); Kuma maintenance '
f'{"on" if kuma_on else "not set"}, suppression #{sup_id}')
return jsonify({'ok': True, 'result': result, 'kuma': kuma_on, 'suppression_id': sup_id})
@app.route('/api/devices')
@require_auth
def api_devices():
+56
View File
@@ -0,0 +1,56 @@
"""Switch one Uptime Kuma maintenance window on and off (Kuma 2.x, over its Socket.IO API).
Kuma has no REST API for maintenance, so this logs in over the same socket its web UI
uses. It only ever resumes or pauses the one window named in config, which is created
once in Kuma ("manual" strategy, covering the monitors that a modem restart takes down):
"uptime_kuma": {"url": "http://10.10.10.25:3001", "username": "...", "password": "...",
"maintenance_id": 7}
Nothing else in Kuma is read or changed.
"""
import logging
from typing import Callable, Optional
import socketio
logger = logging.getLogger('gandalf.kuma')
class KumaError(Exception):
"""Kuma couldn't be reached, refused the login, or refused the change."""
def configured(cfg: Optional[dict]) -> bool:
cfg = cfg or {}
return all(cfg.get(k) for k in ('url', 'username', 'password', 'maintenance_id'))
def set_maintenance(cfg: dict, active: bool, client_factory: Optional[Callable] = None) -> None:
"""Resume (active=True) or pause the configured maintenance window. Raises KumaError."""
timeout = cfg.get('timeout', 10)
factory = client_factory or (lambda: socketio.Client(
reconnection=False, ssl_verify=cfg.get('verify_ssl', True)))
sio = factory()
try:
sio.connect(cfg['url'], transports=['websocket'], wait_timeout=timeout)
res = sio.call('login', {'username': cfg['username'], 'password': cfg['password'], 'token': ''},
timeout=timeout) or {}
if not res.get('ok'):
if res.get('tokenRequired'):
raise KumaError('the Kuma account has two-factor login, which Gandalf cannot answer')
raise KumaError(f'login refused: {res.get("msg") or "no reason given"}')
event = 'resumeMaintenance' if active else 'pauseMaintenance'
res = sio.call(event, int(cfg['maintenance_id']), timeout=timeout) or {}
if not res.get('ok'):
raise KumaError(f'{event} refused: {res.get("msg") or "no reason given"}')
logger.info(f'Uptime Kuma maintenance {cfg["maintenance_id"]} {"on" if active else "off"}')
except KumaError:
raise
except Exception as e: # socketio raises its own errors and plain timeouts
raise KumaError(f'{type(e).__name__}: {e}') from e
finally:
try:
sio.disconnect()
except Exception: # nosec B110 - nothing left to clean up
pass
+24 -5
View File
@@ -1,13 +1,14 @@
"""Read-only client for AT&T BGW-series fiber gateways (tested on BGW320-505).
"""Client for AT&T BGW-series fiber gateways (tested on BGW320-505).
The gateway has no API, but its status pages are served without a login, so
this scrapes them over HTTP. Only GET requests to the fixed status pages below
are ever made: nothing here can change gateway settings.
ModemClient scrapes them over HTTP with GET requests only: it cannot change
anything on the gateway.
With an access code configured, ModemAdmin (at the end) also logs in to read the
NAT session counts, the firewall log and speed test history, and to run the
Troubleshoot page's ping from the gateway.
Those are a login, page reads and a ping test: still no setting is changed.
Troubleshoot page's ping from the gateway. None of that changes a setting. Its
restart() is the one call that does something to the gateway, and it runs only
when an admin presses Restart modem.
Units come from the gateway's own help text, checked against live readings:
Rx/Tx power tenths of a dBm (-169 -> -16.9 dBm)
@@ -392,6 +393,7 @@ _NAT_PAGE = '/cgi-bin/nattable.ha'
_DIAG_PAGE = '/cgi-bin/diag.ha'
_LOGS_PAGE = '/cgi-bin/logs.ha'
_SPEED_PAGE = '/cgi-bin/speed.ha'
_RESTART_PAGE = '/cgi-bin/restart.ha'
# Firewall log row: number, ISO time (gateway's local clock), source, destination, protocol, reason
_FW_ROW_RE = re.compile(r'<tr[^>]*>\s*<td[^>]*>\s*\d+\s*</td>\s*<td>\s*([^<]+?)\s*</td>\s*<td>\s*([^<]*?)\s*</td>'
r'\s*<td>\s*([^<]*?)\s*</td>\s*<td>\s*([^<]*?)\s*</td>\s*<td>\s*([^<]*?)\s*</td>\s*</tr>', re.I)
@@ -536,6 +538,23 @@ class ModemAdmin:
def speed_tests(self) -> List[dict]:
return parse_speed_tests(self._page(_SPEED_PAGE))
def restart(self) -> str:
"""Restart the gateway from its Restart Device page: the one change this client makes,
and only when an admin asks. Returns 'accepted', or 'unconfirmed' when the gateway
dropped the connection before answering (it may already be going down)."""
nonce = _NONCE_RE.search(self._page(_RESTART_PAGE))
if not nonce:
raise ModemDataInvalid('Restart page has no nonce')
try:
resp = self.session.post(f'{self.base_url}{_RESTART_PAGE}', allow_redirects=False,
timeout=max(self.timeout, 30),
data={'nonce': nonce.group(1), 'Restart': 'Restart'})
except (requests.ConnectionError, requests.Timeout):
return 'unconfirmed'
if resp.status_code not in (200, 302) or is_login_page(resp.text):
raise ModemDataInvalid(f'Restart was refused (HTTP {resp.status_code})')
return 'accepted'
def ping(self, target: str, wait_s: float = 30) -> dict:
"""Ping *target* from the gateway (its Troubleshoot page; 5 packets) and return the result."""
nonce = _NONCE_RE.search(self._page(_DIAG_PAGE))
+51
View File
@@ -24,6 +24,7 @@ from urllib3.exceptions import InsecureRequestWarning
import db
import cutsheet
import devices
import kuma
import modem as modem_api
import ticket_text
@@ -1973,6 +1974,50 @@ class NetworkMonitor:
if quality is not None:
db.set_state('wan_quality', quality)
_RESTART_SETTLE_S = 180 # the modem answering this soon may not have restarted yet
_RESTART_GIVE_UP_S = 1800
def _finish_modem_restart(self) -> None:
"""After the Restart modem button: once the modem answers again with its WAN up (or
after 30 minutes regardless), end the Uptime Kuma window and the Gandalf silence the
button started, and note how long it took."""
raw = db.get_state('modem_restart')
try:
state = json.loads(raw) if raw else None
except ValueError:
state = None
if not state or state.get('finished_at'):
return
now = time.time()
age = now - state.get('at', now)
raw = db.get_state('modem_stats')
try:
stats = json.loads(raw) if raw else {}
except ValueError:
stats = {}
uptime = stats.get('uptime_s')
rebooted = uptime is not None and uptime < age
back = (age >= self._RESTART_SETTLE_S and stats.get('reachable')
and not stats.get('wan_problems') and (rebooted or uptime is None))
if not back and age < self._RESTART_GIVE_UP_S:
return
if state.get('kuma') and not state.get('kuma_ended'):
try:
kuma.set_maintenance(self.cfg.get('uptime_kuma') or {}, False)
state['kuma_ended'] = True
except kuma.KumaError as e:
logger.warning(f'Ending the Uptime Kuma window after the modem restart: {e}')
if age < self._RESTART_GIVE_UP_S:
return # try again next cycle
state['kuma_error'] = str(e)
if state.get('suppression_id'):
db.deactivate_suppression(state['suppression_id'])
state.update(finished_at=now, back=bool(back), rebooted=rebooted,
back_after_s=round(age) if back else None)
db.set_state('modem_restart', state)
logger.info(f'Modem restart finished: {"back" if back else "gave up waiting"} after {round(age)} s'
f'{", rebooted" if rebooted else ""}')
_NAT_EVERY_S = 1800 # NAT sessions and the gateway's speed test history
_FIREWALL_EVERY_S = 300 # the firewall log only holds about ten minutes
_FIREWALL_HOURS = 24
@@ -2247,6 +2292,12 @@ class NetworkMonitor:
except Exception as e:
logger.error(f'WAN quality collection failed: {e}', exc_info=True)
# An admin's modem restart: end the silence once the modem is back
try:
self._finish_modem_restart()
except Exception as e:
logger.error(f'Finishing the modem restart failed: {e}', exc_info=True)
# Logged-in modem checks (NAT sessions; a ping from the modem when the internet is slow)
try:
self._collect_modem_checks()
+1
View File
@@ -4,3 +4,4 @@ ldap3>=2.9
pymysql>=1.1.0
requests>=2.31.0
urllib3>=2.0.0
python-socketio[client]>=5.8
+55 -2
View File
@@ -470,6 +470,47 @@ function _modemQuality(q) {
</div>`;
}
// A restart from the Restart modem button: in progress, or finished in the last hour
function _modemRestartNote(r) {
if (!r || !r.at) return '';
const who = r.by ? ` by ${r.by}` : '';
if (!r.finished_at) {
return `<div class="modem-restart-note">Restart requested${lt.escHtml(who)} ${lt.escHtml(_modemWhen(r.at))}. ` +
`Alerts are silenced${r.kuma ? ' here and in Uptime Kuma' : ''} until the modem is back.</div>`;
}
if (Date.now() / 1000 - r.finished_at > 3600) return '';
const how = r.back ? `back after ${_modemDuration(r.back_after_s)}` : 'did not come back within 30 minutes';
return `<div class="modem-restart-note">Restarted${lt.escHtml(who)} ${lt.escHtml(_modemWhen(r.at))}: ` +
`${lt.escHtml(how)}. Alerts are on again.</div>`;
}
async function restartModem(btn) {
const msg = 'Restart the AT&T modem?\n\nThe internet will be down for about 3 to 5 minutes. ' +
'Gandalf alerts and the Uptime Kuma maintenance window are switched on first, and switched off ' +
'again when the modem is back.';
if (!window.confirm(msg)) return;
btn.disabled = true;
try {
let res;
try {
res = await lt.api.post('/api/modem/restart', { confirm: true });
} catch (err) {
if (!String(err.message).startsWith('Uptime Kuma:')) throw err;
if (!window.confirm(`${err.message}\n\nRestart anyway? Uptime Kuma will report the outage.`)) {
btn.disabled = false;
return;
}
res = await lt.api.post('/api/modem/restart', { confirm: true, without_kuma: true });
}
showToast(res.result === 'accepted' ? 'Modem restarting' : 'Restart sent; the modem dropped the connection',
'success');
window.location.reload();
} catch (err) {
btn.disabled = false;
showToast(err.message || 'Could not restart the modem', 'error');
}
}
function updateModem(data) {
const section = document.getElementById('modem-section');
const body = document.getElementById('modem-body');
@@ -477,6 +518,18 @@ function updateModem(data) {
section.hidden = !data.configured;
if (!data.configured) return;
const restartBtn = document.getElementById('modem-restart-btn');
if (restartBtn) {
const pending = data.modem_restart && !data.modem_restart.finished_at;
restartBtn.hidden = !data.can_restart;
restartBtn.disabled = Boolean(pending);
if (!restartBtn.dataset.bound) {
restartBtn.dataset.bound = '1';
restartBtn.addEventListener('click', () => restartModem(restartBtn));
}
}
const restartNote = _modemRestartNote(data.modem_restart);
// On the dashboard, link to the modem's page. On that page, it would link to itself.
const detailsLink = document.getElementById('modem-device-link');
if (detailsLink && typeof DEVICE_NAME === 'undefined' && data.name) {
@@ -487,7 +540,7 @@ function updateModem(data) {
const title = document.getElementById('modem-title');
if (!data.reachable) {
body.innerHTML = `
body.innerHTML = `${restartNote}
<div class="modem-error">Modem status unavailable: ${lt.escHtml(data.error || 'unknown error')}
<span class="modem-stat-note">Last attempt ${lt.escHtml(data.updated || '-')}</span></div>
${_modemQuality(data.wan_quality)}
@@ -508,7 +561,7 @@ function updateModem(data) {
const errs = (counters.rx_errors || 0) + (counters.tx_errors || 0);
const drops = (counters.rx_drops || 0) + (counters.tx_drops || 0);
body.innerHTML = `
body.innerHTML = `${restartNote}
${problems.length ? `<div class="modem-error">WAN down: ${lt.escHtml(problems.join('; '))}</div>` : ''}
<div class="modem-stats">
${_modemStat('WAN', problems.length ? 'DOWN' : 'UP', problems.length ? 'lt-text-red' : 'lt-text-green',
+5
View File
@@ -1849,3 +1849,8 @@ body > .lt-footer { flex-shrink: 0; }
#events-table td { max-width: none; }
#events-table td.desc-cell { white-space: normal; text-align: right; }
}
/* ── WAN / Fiber: restart button and progress note ── */
.modem-actions { display: inline-flex; gap: 6px; }
.modem-restart-note { margin: 8px 12px 0; padding: 6px 10px; font-size: .8em;
border: 1px solid var(--accent-amber); color: var(--text-primary); }
+7 -1
View File
@@ -1,7 +1,13 @@
<section class="g-section" id="modem-section" hidden>
<div class="g-section-header">
<h2 class="g-section-title">WAN / Fiber</h2>
<a class="g-section-actions lt-btn lt-btn-ghost lt-btn-sm" id="modem-device-link" hidden>↗ Details</a>
<span class="g-section-actions modem-actions">
{% if user.groups and 'admin' in user.groups %}
<button type="button" class="lt-btn lt-btn-ghost lt-btn-sm" id="modem-restart-btn" hidden
title="Silence alerts (Gandalf and Uptime Kuma), then restart the modem">⟳ Restart modem</button>
{% endif %}
<a class="lt-btn lt-btn-ghost lt-btn-sm" id="modem-device-link" hidden>↗ Details</a>
</span>
</div>
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
+63 -1
View File
@@ -4,7 +4,7 @@ import time
from datetime import datetime, timedelta, timezone
import os
import sys
from unittest.mock import patch
from unittest.mock import MagicMock, patch
import pytest
@@ -875,3 +875,65 @@ class TestCalmDashboard:
with patch.object(gandalf.db, 'get_recent_resolved', return_value=[ev]):
html = client.get('/', headers=AUTH).get_data(as_text=True)
assert '<details class="g-section resolved-fold">' in html
class TestModemRestart:
CFG = dict(CONFIG, modem={'url': 'http://192.0.2.254', 'access_code': 'secret'},
uptime_kuma={'url': 'http://kuma.test', 'username': 'u', 'password': 'p', 'maintenance_id': 7})
def _post(self, client, body=None, headers=AUTH, cfg=None, kuma_error=None, restart=None):
admin = MagicMock()
if isinstance(restart, Exception):
admin.restart.side_effect = restart
else:
admin.restart.return_value = restart or 'accepted'
with patch.object(gandalf, '_cfg', cfg or self.CFG), \
patch.object(gandalf.kuma, 'set_maintenance', side_effect=kuma_error) as km, \
patch.object(gandalf.modem_api, 'ModemAdmin', return_value=admin):
gandalf.db.create_suppression.return_value = 42
resp = client.post('/api/modem/restart', json=body if body is not None else {'confirm': True},
headers=headers)
return resp, km, admin
def test_admin_only(self, client):
resp, _, admin = self._post(client, headers=USER)
assert resp.status_code == 403 and not admin.restart.called
def test_needs_confirm(self, client):
resp, km, admin = self._post(client, body={})
assert resp.status_code == 400 and not km.called and not admin.restart.called
def test_needs_the_access_code(self, client):
resp, _, admin = self._post(client, cfg=dict(CONFIG, modem={'url': 'http://192.0.2.254'}))
assert resp.status_code == 400 and 'access_code' in resp.get_json()['error']
def test_silences_then_restarts(self, client):
resp, km, admin = self._post(client)
assert resp.status_code == 200
assert resp.get_json() == {'ok': True, 'result': 'accepted', 'kuma': True, 'suppression_id': 42}
km.assert_called_once_with(self.CFG['uptime_kuma'], True)
args = gandalf.db.create_suppression.call_args.args
assert args[:3] == ('all', '', '') and args[4] == 'jared' and args[5] == gandalf.MODEM_RESTART_SILENCE_MIN
admin.restart.assert_called_once_with()
key, state = gandalf.db.set_state.call_args.args
assert key == 'modem_restart' and state['suppression_id'] == 42 and state['kuma'] is True
def test_kuma_failure_stops_unless_accepted(self, client):
resp, _, admin = self._post(client, kuma_error=gandalf.kuma.KumaError('login refused'))
assert resp.status_code == 502 and resp.get_json()['kuma_failed'] and not admin.restart.called
resp, _, admin = self._post(client, body={'confirm': True, 'without_kuma': True},
kuma_error=gandalf.kuma.KumaError('login refused'))
assert resp.status_code == 200 and resp.get_json()['kuma'] is False and admin.restart.called
def test_failed_restart_undoes_the_silence(self, client):
resp, km, _ = self._post(client, restart=gandalf.modem_api.ModemUnreachable('timeout'))
assert resp.status_code == 502
gandalf.db.deactivate_suppression.assert_called_once_with(42)
assert km.call_args_list[-1].args == (self.CFG['uptime_kuma'], False)
def test_one_at_a_time(self, client):
pending = json.dumps({'at': time.time() - 60, 'finished_at': None})
state = {'modem_restart': pending}
with patch.object(gandalf.db, 'get_state', side_effect=lambda k, d=None: state.get(k, d)):
resp, _, admin = self._post(client)
assert resp.status_code == 409 and not admin.restart.called
+69
View File
@@ -0,0 +1,69 @@
"""kuma.py: switching the Uptime Kuma maintenance window, against a fake Socket.IO client."""
import os
import sys
import pytest
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
import kuma # noqa: E402
CFG = {'url': 'http://kuma.test:3001', 'username': 'gandalf', 'password': 'pw', 'maintenance_id': 7}
class FakeSocket:
def __init__(self, answers=None, connect_error=None):
self.answers = answers or {}
self.connect_error = connect_error
self.calls, self.disconnected = [], False
def connect(self, url, **kw):
if self.connect_error:
raise self.connect_error
self.url = url
def call(self, event, data=None, timeout=None):
self.calls.append((event, data))
return self.answers.get(event, {'ok': True})
def disconnect(self):
self.disconnected = True
def run(active=True, **kw):
sock = FakeSocket(**kw)
kuma.set_maintenance(CFG, active, client_factory=lambda: sock)
return sock
class TestKuma:
def test_configured_needs_every_field(self):
assert kuma.configured(CFG)
assert not kuma.configured(dict(CFG, password=''))
assert not kuma.configured(None)
def test_resume_logs_in_then_switches_the_window_on(self):
sock = run(True)
assert sock.calls == [('login', {'username': 'gandalf', 'password': 'pw', 'token': ''}),
('resumeMaintenance', 7)]
assert sock.disconnected and sock.url == CFG['url']
def test_pause(self):
assert run(False).calls[-1] == ('pauseMaintenance', 7)
def test_refused_login(self):
with pytest.raises(kuma.KumaError, match='login refused: Incorrect username or password'):
run(answers={'login': {'ok': False, 'msg': 'Incorrect username or password.'}})
def test_two_factor_account(self):
with pytest.raises(kuma.KumaError, match='two-factor'):
run(answers={'login': {'ok': False, 'tokenRequired': True}})
def test_refused_change(self):
with pytest.raises(kuma.KumaError, match='resumeMaintenance refused'):
run(answers={'resumeMaintenance': {'ok': False, 'msg': 'not found'}})
def test_unreachable(self):
sock = FakeSocket(connect_error=ConnectionError('refused'))
with pytest.raises(kuma.KumaError, match='ConnectionError'):
kuma.set_maintenance(CFG, True, client_factory=lambda: sock)
assert sock.disconnected
+30
View File
@@ -389,6 +389,8 @@ class FakeGateway:
return MagicMock(status_code=302 if ok else 200, text='' if ok else LOGIN_FORM)
if not self.logged_in:
return MagicMock(status_code=200, text=LOGIN_FORM)
if path == '/cgi-bin/restart.ha' and method == 'GET':
return MagicMock(status_code=200, text=DIAG_FORM.replace('diag.ha', 'restart.ha'))
if path == '/cgi-bin/nattable.ha':
return MagicMock(status_code=200, text=NAT_PAGE)
if path == '/cgi-bin/diag.ha' and method == 'POST':
@@ -514,3 +516,31 @@ class TestModemExtras:
def test_speed_tests_missing(self):
assert modem.parse_speed_tests('<html></html>') == []
class TestModemRestart:
def _admin(self):
gw = FakeGateway()
a = modem.ModemAdmin({'url': 'http://192.0.2.254', 'access_code': 'secret'}, sleep=lambda s: None)
a.session.request = MagicMock(side_effect=gw)
return a, gw
def test_posts_the_restart_form(self):
a, gw = self._admin()
a.session.post = MagicMock(return_value=MagicMock(status_code=200, text='Restarting...'))
assert a.restart() == 'accepted'
data = a.session.post.call_args.kwargs['data']
assert data == {'nonce': 'ffee00112233445566778899aabbccdd', 'Restart': 'Restart'}
assert a.session.post.call_args.args[0].endswith('/cgi-bin/restart.ha')
assert ('POST', '/cgi-bin/login.ha') in gw.calls # logged in to read the form
def test_connection_dropped_mid_restart(self):
a, _ = self._admin()
a.session.post = MagicMock(side_effect=requests.ConnectionError('reset'))
assert a.restart() == 'unconfirmed'
def test_refused(self):
a, _ = self._admin()
a.session.post = MagicMock(return_value=MagicMock(status_code=200, text=LOGIN_FORM))
with pytest.raises(modem.ModemDataInvalid, match='refused'):
a.restart()
+51
View File
@@ -1154,3 +1154,54 @@ class TestModemExtrasCollection:
saved = [c.args[1] for c in db.set_state.call_args_list if c.args[0] == 'modem_checks'][-1]
assert saved['firewall']['hours'] == {'2026-11-19T18': {'flood': 1, 'invalid': 0, 'other': 0}}
assert 'speed' in saved and admin.firewall_log.called
class TestFinishModemRestart:
NOW = 1_000_000.0
def _run(self, mon, db, restart, stats, kuma_error=None):
state = {'modem_restart': json.dumps(restart), 'modem_stats': json.dumps(stats)}
db.get_state.side_effect = lambda k, d=None: state.get(k, d)
db.set_state.reset_mock()
mon.cfg = {'uptime_kuma': {'url': 'http://kuma.test'}}
with patch.object(monitor.time, 'time', return_value=self.NOW), \
patch.object(monitor.kuma, 'set_maintenance', side_effect=kuma_error) as km:
mon._finish_modem_restart()
saved = [c.args[1] for c in db.set_state.call_args_list if c.args[0] == 'modem_restart']
return (saved[-1] if saved else None), km
def _restart(self, ago, **kw):
return dict({'at': self.NOW - ago, 'by': 'jared', 'suppression_id': 9, 'kuma': True,
'finished_at': None}, **kw)
def test_back_ends_kuma_and_the_silence(self, mon, db):
out, km = self._run(mon, db, self._restart(300), {'reachable': True, 'wan_problems': [], 'uptime_s': 200})
km.assert_called_once_with({'url': 'http://kuma.test'}, False)
db.deactivate_suppression.assert_called_once_with(9)
assert out['back'] and out['rebooted'] and out['back_after_s'] == 300 and out['finished_at'] == self.NOW
def test_waits_while_the_modem_is_down(self, mon, db):
out, km = self._run(mon, db, self._restart(300), {'reachable': False})
assert out is None and not km.called and not db.deactivate_suppression.called
def test_too_soon_to_tell(self, mon, db):
out, _ = self._run(mon, db, self._restart(60), {'reachable': True, 'wan_problems': [], 'uptime_s': 30})
assert out is None
def test_answering_without_having_rebooted_is_not_back(self, mon, db):
out, _ = self._run(mon, db, self._restart(300), {'reachable': True, 'wan_problems': [], 'uptime_s': 99999})
assert out is None
def test_gives_up_after_half_an_hour(self, mon, db):
out, km = self._run(mon, db, self._restart(1900), {'reachable': False})
assert out['back'] is False and out['back_after_s'] is None and km.called
db.deactivate_suppression.assert_called_once_with(9)
def test_kuma_failure_retries_next_cycle(self, mon, db):
out, _ = self._run(mon, db, self._restart(300), {'reachable': True, 'wan_problems': [], 'uptime_s': 200},
kuma_error=monitor.kuma.KumaError('down'))
assert out is None and not db.deactivate_suppression.called
def test_finished_restart_is_left_alone(self, mon, db):
out, km = self._run(mon, db, self._restart(300, finished_at=1.0), {'reachable': True})
assert out is None and not km.called