Add separate unifi_failure_threshold to avoid false tickets on reboots

UniFi switches/APs take several minutes to rejoin after a firmware-update
reboot (observed AP taking 12 min on the scheduled 3am update), longer than
the 2-consecutive-check (~4 min) threshold shared with host link-down
detection. Give UniFi device offline detection its own, more forgiving
threshold so momentary maintenance reboots don't cut tickets.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UB2D82eYNzD1bnMHhFPjRr

README.md

@@ -148,7 +148,7 @@ returned by the UniFi API for that device.
 
 | Condition | Priority |
 |---|---|
-| UniFi device offline (≥2 consecutive checks) | P2 High |
+| UniFi device offline (≥`unifi_failure_threshold` consecutive checks) | P2 High |
 | Proxmox host NIC link-down regression (≥2 consecutive checks) | P2 High |
 | Host unreachable via ping (≥2 consecutive checks) | P2 High |
 | ≥3 hosts simultaneously reporting interface failures | P1 Critical |
@@ -218,6 +218,7 @@ Shared by both processes. Located in the working directory (`/var/www/html/prod/
   "monitor": {
     "poll_interval": 120,
     "failure_threshold": 2,
+    "unifi_failure_threshold": 5,
     "cluster_threshold": 3,
     "ping_hosts": [
       { "name": "pbs", "ip": "10.10.10.3" }
@@ -243,6 +244,7 @@ Shared by both processes. Located in the working directory (`/var/www/html/prod/
 | `hosts` | Maps Prometheus instance labels → display hostnames |
 | `monitor.poll_interval` | Seconds between full check cycles (default: 120) |
 | `monitor.failure_threshold` | Consecutive failures before creating ticket (default: 2) |
+| `monitor.unifi_failure_threshold` | Consecutive failures before ticketing a UniFi device as offline (default: same as `failure_threshold`). Set higher than `failure_threshold` — UniFi switches/APs can take several minutes to rejoin after a firmware-update reboot, so a low threshold cuts tickets for momentary maintenance reboots instead of real failures. |
 | `monitor.cluster_threshold` | Hosts with failures to trigger cluster-wide P1 (default: 3) |
 | `monitor.ping_hosts` | Hosts checked only by ping (no node_exporter) |
 

monitor.py

@@ -677,6 +677,11 @@ class NetworkMonitor:
         mon = self.cfg.get('monitor', {})
         self.poll_interval = mon.get('poll_interval', 120)
         self.fail_thresh = mon.get('failure_threshold', 2)
+        # UniFi gear (switches/APs) can take several minutes to rejoin after a
+        # firmware-update reboot, much longer than a Proxmox host link flap.
+        # Use a separate, more forgiving threshold so those momentary reboots
+        # don't cut tickets, while still catching genuine device failures.
+        self.unifi_fail_thresh = mon.get('unifi_failure_threshold', self.fail_thresh)
         self.cluster_thresh = mon.get('cluster_threshold', 3)
         self.cluster_name = mon.get('cluster_name', CLUSTER_NAME)
 
@@ -809,7 +814,7 @@ class NetworkMonitor:
                     name, d.get('type', ''),
                     f'UniFi {name} ({d.get("ip","")}) offline ({_now_utc()})',
                 )
-                if not sup and consec >= self.fail_thresh and not ticket_id:
+                if not sup and consec >= self.unifi_fail_thresh and not ticket_id:
                     self._ticket_unifi(event_id, d)
             else:
                 db.resolve_event('unifi_device_offline', name, d.get('type', ''))
@@ -931,7 +936,7 @@ class NetworkMonitor:
     def run(self) -> None:
         logger.info(
             f'Gandalf monitor started – poll_interval={self.poll_interval}s '
-            f'fail_thresh={self.fail_thresh}'
+            f'fail_thresh={self.fail_thresh} unifi_fail_thresh={self.unifi_fail_thresh}'
         )
         while True:
             try: