Add separate unifi_failure_threshold to avoid false tickets on reboots
Lint / Python (flake8) (push) Failing after 1m4s
Lint / JS (eslint) (push) Successful in 5s
Lint / Deploy (push) Skipped
Security / Python Security (bandit) (push) Failing after 37s
Test / Python Tests (pytest) (push) Failing after 33s
Lint / Notify on failure (push) Successful in 2s

UniFi switches/APs take several minutes to rejoin after a firmware-update
reboot (observed AP taking 12 min on the scheduled 3am update), longer than
the 2-consecutive-check (~4 min) threshold shared with host link-down
detection. Give UniFi device offline detection its own, more forgiving
threshold so momentary maintenance reboots don't cut tickets.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UB2D82eYNzD1bnMHhFPjRr
This commit is contained in:
2026-09-04 20:59:43 -04:00
co-authored by Claude Sonnet 5
parent 28fd5aad14
commit 42cf754796
2 changed files with 10 additions and 3 deletions
+3 -1
View File
@@ -148,7 +148,7 @@ returned by the UniFi API for that device.
| Condition | Priority |
|---|---|
| UniFi device offline (≥2 consecutive checks) | P2 High |
| UniFi device offline (≥`unifi_failure_threshold` consecutive checks) | P2 High |
| Proxmox host NIC link-down regression (≥2 consecutive checks) | P2 High |
| Host unreachable via ping (≥2 consecutive checks) | P2 High |
| ≥3 hosts simultaneously reporting interface failures | P1 Critical |
@@ -218,6 +218,7 @@ Shared by both processes. Located in the working directory (`/var/www/html/prod/
"monitor": {
"poll_interval": 120,
"failure_threshold": 2,
"unifi_failure_threshold": 5,
"cluster_threshold": 3,
"ping_hosts": [
{ "name": "pbs", "ip": "10.10.10.3" }
@@ -243,6 +244,7 @@ Shared by both processes. Located in the working directory (`/var/www/html/prod/
| `hosts` | Maps Prometheus instance labels → display hostnames |
| `monitor.poll_interval` | Seconds between full check cycles (default: 120) |
| `monitor.failure_threshold` | Consecutive failures before creating ticket (default: 2) |
| `monitor.unifi_failure_threshold` | Consecutive failures before ticketing a UniFi device as offline (default: same as `failure_threshold`). Set higher than `failure_threshold` — UniFi switches/APs can take several minutes to rejoin after a firmware-update reboot, so a low threshold cuts tickets for momentary maintenance reboots instead of real failures. |
| `monitor.cluster_threshold` | Hosts with failures to trigger cluster-wide P1 (default: 3) |
| `monitor.ping_hosts` | Hosts checked only by ping (no node_exporter) |