Monitor: keep checking interfaces whose link-down alert is still open
Lint / JS (eslint) (push) Successful in 8s
Lint / Python (flake8) (push) Successful in 49s
Lint / Notify on failure (push) Skipped
Lint / Deploy (push) Skipped
Security / Python Security (bandit) (push) Successful in 53s
Test / Python Tests (pytest) (push) Successful in 1m3s
Lint / Python (flake8) (pull_request) Successful in 55s
Lint / JS (eslint) (pull_request) Successful in 6s
Lint / Notify on failure (pull_request) Skipped
Lint / Deploy (pull_request) Skipped
Security / Python Security (bandit) (pull_request) Successful in 1m22s
Test / Python Tests (pytest) (pull_request) Successful in 1m27s

After the interface baseline is reset, a port that is already down reads as
"first seen down", which neither alerts nor resolves. An alert opened before
the reset was then stuck open without updates: micro1's enp3s0 had sat at
one failed check since March with no ticket. An interface with an open
interface_down alert is now treated as previously up, so the alert keeps
counting, tickets and resolves as normal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-11 01:02:53 -05:00
co-authored by Claude Opus 5.5
parent 7e75ba8b3c
commit d07787ee38
2 changed files with 25 additions and 0 deletions
+6
View File
@@ -1195,6 +1195,10 @@ class NetworkMonitor:
def _process_interfaces(self, states: Dict[str, Dict[str, bool]], suppressions: list) -> None:
baseline = db.get_baseline()
new_baseline = {k: dict(v) for k, v in baseline.items()}
# An interface with an open link-down alert was up before it went down, whatever the
# baseline says now: after a baseline reset it reads as "first seen down", which would
# leave the alert open forever without updating, ticketing or resolving it.
open_down = set(db.get_open_event_targets('interface_down'))
# Only count hosts with genuine regressions (UP→DOWN) toward cluster threshold
hosts_with_regression: List[str] = []
@@ -1207,6 +1211,8 @@ class NetworkMonitor:
for iface, is_up in ifaces.items():
prev = baseline.get(host, {}).get(iface) # 'up', 'initial_down', or None
if (host, iface) in open_down:
prev = 'up'
if is_up:
new_baseline[host][iface] = 'up'