Compare commits

..
Author SHA1 Message Date
jaredandClaude Sonnet 4.6 65536d58c5 Fix misleading docstring on _purge_old_jobs_loop
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
The comment claimed the function "runs daily event purge" — that
housekeeping is done by monitor.py's main loop, not here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-14 11:06:28 -04:00
jaredandClaude Sonnet 4.6 0e852ebd11 Fix ping-only hosts polled twice per cycle with inconsistent parameters
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
_collect_snapshot called pulse.ping(count=1) independently from
_process_ping_hosts which called pulse.ping(count=3). This doubled
network load and could show a host as 'up' in the dashboard while
simultaneously firing an 'unreachable' alert, or vice versa.

Now ping_states is computed once in run() using the alert-quality
parameters (count=3) and shared by both snapshot and alert processing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 23:13:43 -04:00
jaredandClaude Sonnet 4.6 8aaaa89f03 Replace deprecated datetime.utcnow() with datetime.now(timezone.utc)
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
datetime.utcnow() is deprecated in Python 3.12 and removed in 3.13.
Replace all four call sites with timezone-aware equivalents so the
codebase is ready for Python 3.12+.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 15:34:41 -04:00
jaredandClaude Sonnet 4.6 7467236ffd Add ESLint config enforcing no-undef and eqeqeq
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Without a config file, ESLint was running with no-undef disabled, meaning
undefined variable references in static/app.js were silently ignored.
Add .eslintrc.json with no-undef: error and eqeqeq: error so CI actually
catches JS bugs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 15:33:26 -04:00
jaredandClaude Sonnet 4.6 dc16a583a5 Fix inspector auto-refresh ignoring 'Off' setting on page load
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Same ?? / || issue as the previous fix in index.html and links.html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 13:20:42 -04:00
jaredandClaude Sonnet 4.6 12ef20901a Fix auto-refresh ignoring 'Off' setting on page load
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Using || 30 / || 60 as a fallback treats refreshInterval=0 (Off) as
falsy and replaces it with the default, causing auto-refresh to start
even when the user saved 'Off'. Replace with nullish coalescing (??)
so only null/undefined triggers the default.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 13:19:44 -04:00
jaredandClaude Sonnet 4.6 0693ad56f9 Fix monitor loop double-sleep on error; add grep -F regression test
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
On exception the monitor slept 30s inside the except block then fell
through to time.sleep(poll_interval), giving a 150s recovery gap instead
of 30s. Adding continue after the error sleep fixes this.

Also adds a regression test asserting dmesg filtering uses grep -F --
so a future refactor cannot silently reintroduce the regex wildcard bug.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 13:16:43 -04:00
jaredandClaude Sonnet 4.6 c455c5d4b9 Fix suppression annotation for interface_down not checking host-level rules
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
monitor.py checks both 'interface' and 'host' suppressions for interface_down
events, but _annotate_suppressions only checked 'interface'. A host-level
suppression would silently suppress tickets but not mark the table row as
suppressed in the UI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 13:14:46 -04:00
jaredandClaude Sonnet 4.6 13e6839cdf Show suppressed badge on host cards during global maintenance windows
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Global suppressions (target_type='all') have an empty target_name, so
the selectattr filter never matched them, leaving no visual indicator
when a global maintenance window was active. Pre-compute has_global_sup
before the host loop and OR it into the badge condition.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 13:12:25 -04:00
jaredandClaude Sonnet 4.6 e7c83eff2b Use grep -F in dmesg filter to prevent interface name treated as regex
grep {iface} treats dots and other special chars as regex metacharacters.
Switch to grep -F -- {iface} for fixed-string matching and to prevent
a leading dash in the interface name from being parsed as a grep flag.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 11:12:02 -04:00
jaredandClaude Sonnet 4.6 5c2a92eaee Remove unused is_new parameter from ticket helper methods
After fixing the is_new guard bug, is_new is no longer used inside
_ticket_interface, _ticket_unifi, or _ticket_unreachable. Drop it from
their signatures and call sites.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-13 11:10:32 -04:00
jaredandClaude Sonnet 4.6 f942e96c24 Fix ticket_id never stored when fail_thresh>1; guard sessionStorage JSON.parse
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
monitor.py: _ticket_interface/_ticket_unifi/_ticket_unreachable all used
`if tid and is_new` to guard db.set_ticket_id(). Since is_new is True only
on the first upsert (consec=1) but tickets are created at consec>=fail_thresh
(default 2), is_new is always False when the ticket is created, so the
ticket link never appeared in the UI. Changed to `if tid:`.

links.html: JSON.parse(sessionStorage.getItem(...)) in togglePanel and
restoreCollapseState had no try-catch. Corrupt/stale session storage would
throw an uncaught SyntaxError. Also wrapped all sessionStorage.setItem
calls in try-catch to defend against storage-full / private-browsing errors.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:45:20 -04:00
jaredandClaude Sonnet 4.6 8173b9dc7f Cancel in-flight diagnostic poll when user selects a new port
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Previously switching ports while a diagnostic was running left the
setInterval timer active, causing the result to be written into the
old (now detached) DOM elements and never shown to the user.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:26:53 -04:00
jaredandClaude Sonnet 4.6 2896692d93 Fix empty-object false negative in links page no-data check
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
The check `!data.hosts && !data.unifi_switches` never caught empty
objects `{}`, which are truthy. Replace with Object.keys length checks
so the friendly "no data yet" banner renders when both collections
are empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:21:50 -04:00
jaredandClaude Sonnet 4.6 c2492bbfe8 Use shlex.quote for remote_cmd in build_ssh_command
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Matches the pattern already used in monitor.py's _ssh_batch(); prevents
quoting breakage if shlex.quote(iface) emits single-quoted tokens inside
the remote command string.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:17:11 -04:00
jaredandClaude Sonnet 4.6 33b214eac0 fix: escape ticket_id text content in dynamic events table
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
ticket_id was already escaped in the href attribute but the visible
text (#<id>) used the raw value in an innerHTML template literal.
Apply lt.escHtml() for defense-in-depth against a compromised ticket API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 23:02:09 -04:00
jaredandClaude Sonnet 4.6 0331c2bd00 fix: add aria-required to s-reason field in suppressions form
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
The reason input had `required` for browser validation but was missing
`aria-required="true"`, so screen readers did not announce it as required.
Matches the fix already applied to the equivalent field in base.html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 15:11:05 -04:00
jaredandClaude Sonnet 4.6 2710eaa118 fix: SSH shell quoting bug breaks ethtool collection; ticket_id KeyError
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
monitor.py _ssh_batch(): the remote command was wrapped in double-quotes
(f'root@{ip} "{shell_cmd}"') but shell_cmd itself contains double-quoted
echo sentinels ("___IFACE:eth0___"). When Pulse's shell parses the full
ssh invocation, the nested double-quotes cause mis-parsing — the remote
command is split incorrectly, silently breaking all ethtool/SFP DOM
collection. Fix: use shlex.quote(shell_cmd) so the entire remote command
is single-quoted, leaving inner double-quotes untouched.

TicketClient.create(): data['ticket_id'] raises KeyError if the Tinker
Tickets API returns success=true without a ticket_id field (malformed
response). Use data.get('ticket_id') with an explicit warning log.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 13:41:09 -04:00
jaredandClaude Sonnet 4.6 9c011cfce1 fix: UTC timezone suffix missing from all isoformat() timestamp outputs
db.py returned all datetime columns (first_seen, last_seen, resolved_at,
created_at, expires_at) as bare ISO strings like "2026-03-14T14:14:21"
with no timezone marker. Per the ECMAScript spec, new Date() on a
datetime string without timezone treats it as LOCAL time, not UTC.
This made lt.time.ago() and stale-detection wrong for any user whose
browser is not in UTC — event ages and stale warnings would be off by
the client's UTC offset.

monitor.py had the same issue on the network_snapshot 'updated' field.

Fix: append 'Z' to all isoformat() calls (UTC datetimes confirmed by
MySQL server timezone and _now_utc() pattern used throughout codebase).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 13:28:49 -04:00
jaredandClaude Sonnet 4.6 182cc8c451 fix: inspector.html stale/updated timestamp broken date parsing
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Same bug as was just fixed in links.html: data.updated is stored as
"YYYY-MM-DD HH:MM:SS UTC" by monitor.py, so appending 'Z' produced
"…UTCZ" — an invalid date. The stale-data warning and Updated timestamp
in Inspector were silently showing "Invalid Date" and the stale overlay
never fired. Fixed to use _toIso() (already global via app.js).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 13:25:17 -04:00
jaredandClaude Sonnet 4.6 0d4f209c79 fix: admin-only suppression enforcement, links.html broken date parsing
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Security: add require_admin decorator; apply to POST/DELETE /api/suppressions
and /suppressions page. Previously any user in allowed_groups could create or
delete suppressions even though the nav restricts the UI to admins.

Bug: links.html "Updated:" timestamp and stale-warning both produced
Invalid Date because the raw "YYYY-MM-DD HH:MM:SS UTC" string was appended
with 'Z' instead of being normalised through _toIso(). Fix both call sites to
use _toIso(), and remove the now-redundant local _toIso redefinition.

Style: use `with open(sentinel, 'w'): pass` consistently (was open().close()
at avatar JPEG validation path).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 13:03:37 -04:00
jaredandClaude Sonnet 4.6 b3495caac8 fix: aria-required sync, aria-label pills, deduplicate setDuration logic
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- updateSuppressForm() now sets required + aria-required on sup-name/sup-detail
  when target type changes; sup-reason gets static aria-required="true"
- onTypeChange() in suppressions page syncs aria-required on s-name
- s-name in suppressions.html gets initial required/aria-required (default type=host)
- Duration pills in both modal and suppressions page now have descriptive
  aria-label ("30 minutes", "1 hour", etc.) alongside the group aria-label
- setDuration() in app.js accepts optional {expiresId,pillSel,hintId} opts so
  logic lives in one place; suppressions.html setDur() delegates to it
- Post-create form reset uses setDur() instead of manually patching DOM

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:58:32 -04:00
jaredandClaude Sonnet 4.6 ac7f406175 fix: inspector stale data warning, remove dead supported_modes code
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- inspector.html: show orange '⚠ Stale: HH:MM' with tooltip when link_stats data is >15 min old (previously just showed the time with no visual warning)
- style.css: add .g-stale-warn helper class (orange, bold) for the stale indicator
- diagnose.py: remove supported_modes accumulation from parse_ethtool() — field was collected but never consumed by analyze() or displayed anywhere

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:05:08 -04:00
jaredandClaude Sonnet 4.6 124ca10bb0 fix: diagnostic toggle hint, link_stats schema, pagination UX, rate-limit feedback
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- inspector.html: collapsible section hint text now toggles between [expand]/[collapse] when clicked
- inspector.html: timeout and connection-loss during diagnostic poll now show a Retry button instead of a dead end
- inspector.html: 429 rate-limit response shows a clear human-readable message instead of generic error
- app.py: empty link_stats fallback now includes unifi_switches:{} for schema consistency with real data shape
- index.html: pagination overflow notice now says "export all as JSON" (opens in new tab) instead of misleadingly linking to raw API as navigation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 12:01:56 -04:00
jaredandClaude Sonnet 4.6 18066ef062 fix: escape poe_class in inspector panel for consistency
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
d.poe_mode was already wrapped in escHtml(); apply same to d.poe_class.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 11:56:11 -04:00
jaredandClaude Sonnet 4.6 160dfdfd79 fix: LLDP port label bug, suppression SQL dead code, avatar path hardening
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- inspector.html: fix LLDP neighbor label in port blocks — port.lldp_table never exists; data is at port.lldp (dict with system_name/chassis_id); both port block renderers corrected
- db.py: remove dead 'target_detail IS NULL' branch in suppression check — target_detail is always stored as '' not NULL; query simplified to target_detail=''
- app.py: resolve cache_dir/cache_file/sentinel to absolute paths; guard against path escape before use
- app.py: wrap sentinel os.path.getmtime() in try/except OSError to handle TOCTOU deletion race

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 09:31:25 -04:00
jaredandClaude Sonnet 4.6 e0452193ee fix: flake8 E701 in avatar handler; update SSH test to match accept-new
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- app.py: split 'with open(sentinel): pass' onto two lines (flake8 E701)
- tests/test_diagnose.py: rename test and assert StrictHostKeyChecking=accept-new (not =no which was fixed earlier)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 09:23:06 -04:00
jaredandClaude Sonnet 4.6 41c350815a fix: cache_ttl config validation; ticket_web_url via tojson in base.html
- app.py: wrap int(cache_ttl) in try/except so a misconfigured non-integer value falls back to 3600 instead of raising ValueError
- base.html: use Jinja2 tojson filter for ticket_web_url to ensure proper JS string escaping regardless of URL contents

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 09:05:53 -04:00
jaredandClaude Sonnet 4.6 72651e5d33 fix: LDAP conn leak, health timing info, security headers, link_stats size guard
- app.py: move conn.unbind() into finally block in api_avatar() so connection is always closed even if conn.search() throws
- app.py: remove elapsed-time strings from /health response (unauthenticated endpoint no longer leaks monitor timing)
- app.py: add after_request hook setting X-Content-Type-Options, X-Frame-Options, Referrer-Policy on all responses
- app.py: add 10 MB size guard on link_stats before JSON parse; log actual exception on parse failure
- app.py: wrap suppressions_page network_snapshot parse in try/except (same protection as index page)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 08:50:51 -04:00
jaredandClaude Sonnet 4.6 a28533946e fix: LDAP empty-password guard, expires_minutes bounds, snapshot JSON safety, rate dict cleanup
- app.py: fail loudly if LDAP bind_pw is not configured rather than attempting anonymous bind
- app.py: validate expires_minutes is 1–43200 (max 30 days) before storing suppression
- app.py: wrap network_snapshot JSON parse in try/except so a corrupt DB value returns degraded page instead of 500
- app.py: prune _diag_rate entries inactive for >1h to prevent unbounded growth

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 08:47:43 -04:00
jaredandClaude Sonnet 4.6 d3616badb2 fix: LLDP input validation, mgmt_ip early validation, poll timer cleanup, monitor backoff
- app.py: validate server_name from LLDP with fullmatch before use in logs/lookups (prevents log injection)
- app.py: validate each mgmt_ip candidate before assigning host_ip (avoids assigning non-IP string that then fails later check)
- app.py: log actual exception in link_stats JSON parse error
- inspector.html: clear _diagPollTimer in closePanel() so timer doesn't orphan when panel is closed mid-poll
- monitor.py: sleep 30s after a monitor loop exception before resuming normal poll interval

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 08:45:28 -04:00
jaredandClaude Sonnet 4.6 705d06ca5b fix: diagnostic rate limiting, lock-held ownership check, iface name length cap
- app.py: add per-user diagnostic rate limit (5/min) enforced atomically under _diag_lock
- app.py: move diagnostic job ownership check inside _diag_lock to close TOCTOU window; snapshot result before releasing lock
- monitor.py: cap interface name regex to 15 chars (Linux IFNAMSIZ limit)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 08:42:50 -04:00
jaredandClaude Sonnet 4.6 1daa9ab475 security: harden exception exposure, SSL config, and Pulse response parsing
- app.py: replace raw str(e) in diagnostic _run() with generic client message; log internally only
- app.py: /health endpoint no longer leaks exception strings to unauthenticated callers; errors logged server-side
- monitor.py: UniFi SSL verification now defaults True, configurable via config.json unifi.verify_ssl; urllib3 warning suppression scoped to verify=False only (removed global disable)
- monitor.py: Pulse execution_id extracted with .get() + explicit None check to avoid KeyError on malformed response
- monitor.py: interface name regex drops '@' (not a valid kernel interface char) to match app.py and fix inconsistency

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 08:40:25 -04:00
jaredandClaude Sonnet 4.6 17d273d792 arch+security: route all server contact through Pulse, harden SSH
Architecture:
- Remove direct subprocess ping from Gandalf; add PulseClient.ping()
  which runs the ping via the Pulse worker instead
- Remove standalone ping() function and subprocess import from monitor.py
- Add self.pulse alias to NetworkMonitor for convenience
- Both _process_ping_hosts() and snapshot builder now use self.pulse.ping()

Security:
- Change StrictHostKeyChecking=no → accept-new in both SSH command
  builders (monitor.py _ssh_batch, diagnose.py build_ssh_command).
  The Pulse worker's known_hosts is now authoritative; host keys are
  recorded on first connection and verified on all subsequent ones.
  MITM attacks after initial key exchange are now detectable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:58:16 -04:00
jaredandClaude Sonnet 4.6 088ba789c9 security+a11y: job ownership check, aria-live chips, aria-hidden topo
security:
- Fix bare open(sentinel, 'w').close() file descriptor leak; use
  context manager instead
- Store requesting username in _diag_jobs at creation time; return 403
  from api_diagnose_poll if the polling user does not match the job owner

accessibility:
- Add aria-live="polite" aria-atomic="true" to .status-chips container
  so screen readers announce critical/warning count changes on refresh
- Add aria-controls="events-table-wrap" to critical and warning stat
  cards so assistive tech knows these buttons control the events table
- Add aria-hidden sync to topology setCollapsed() — hidden topology
  content is now removed from the accessibility tree when collapsed,
  preventing keyboard focus from entering invisible elements

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:53:17 -04:00
jaredandClaude Sonnet 4.6 60c7c27ac7 refactor: const for _inspInterval in inspector.html
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Last remaining var declaration; matches the pattern in index.html and
links.html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:45:42 -04:00
jaredandClaude Sonnet 4.6 b0817e2582 refactor: replace inline onclick with data-action event delegation
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
The command palette button used an inline onclick handler while every
other interactive element in base.html uses data-action + event
delegation. Now consistent: data-action="open-cmdpalette" handled in
the global footer click listener.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:45:09 -04:00
jaredandClaude Sonnet 4.6 80587bbbc4 a11y: keyboard-accessible panel toggles, region landmarks in inspector
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Add role="button" tabindex="0" aria-expanded to .link-host-title
  in both static and JS-rendered panels (host panels + UniFi switches)
- Sync aria-expanded in togglePanel(), restoreCollapseState(),
  collapseAll(), and expandAll()
- Add keydown handler (Enter/Space) so panel headers are keyboard-operable
- Add role="region" aria-label to inspector main chassis area
- Add role="complementary" aria-label to inspector port detail panel
- Replace last inline date-parse in renderLinks() with _toIso() helper

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:44:23 -04:00
jaredandClaude Sonnet 4.6 18ab78c35a security: escape user input in 403 error response to prevent XSS
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
The require_auth decorator was interpolating user['username'] and the
allowed_groups list directly into HTML strings. An attacker with a
crafted username or control over group names could inject arbitrary HTML.

Use html.escape() on both values before insertion.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:41:31 -04:00
jaredandClaude Sonnet 4.6 68f69e1539 refactor: extract _annotate_suppressions helper, remove orphaned CSS
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Extract identical suppression-annotation loop from index() and
  api_status() into _annotate_suppressions() helper to eliminate DRY
  violation
- Improve stuck-job error message: 'thread crash' → 'no activity for
  5 minutes' (less alarming, more accurate)
- Remove orphaned .events-filter-bar CSS class (never referenced in
  any template or JS file)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:39:52 -04:00
jaredandClaude Sonnet 4.6 8e9a6b9b3f refactor: const/let modernisation and eliminate duplicate date-parse logic
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Replace all var declarations in base.html, index.html scripts with
  const/let (const for bindings that are never reassigned, let otherwise)
- Add _toIso() helper to links.html script block and replace the two
  inline .replace(' UTC','Z').replace(' ','T') patterns with it
- Replace var with const in links.html _linksInterval

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:37:32 -04:00
jaredandClaude Sonnet 4.6 872dbf3b48 a11y: aria-pressed for all pill groups, aria-label on search inputs and buttons
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Add role="group" + aria-label to duration-pills and sev-pills containers
- Add aria-pressed to severity filter, duration, and refresh-interval pills
- Keep aria-pressed in sync with JS (setDuration, applyRefreshPillUI, modal reset)
- Add aria-label to events-search, host-search, links-search inputs
- Add aria-label to host and UniFi device suppress buttons in templates
- Replace dynamic style color strings in links.html stat cards with TDS
  utility classes (lt-text-red/green/amber) via downCls/errCls variables

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:34:16 -04:00
jaredandClaude Sonnet 4.6 ff2c0bd90a Fix: inspector empty states and diagnostic button accessibility
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Replace .empty-state (removed class) with TDS lt-empty-state--sm in
  both error branches of renderInspector() and loadInspector()
- Diagnostic run button: add aria-label, apply lt-btn TDS classes for
  consistent styling instead of custom btn-diag-only styling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:21:27 -04:00
jaredandClaude Sonnet 4.6 5ee7154c45 CSS: migrate links.html static inline styles to classes
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- lt-divider--unifi / lt-divider-label--unifi: replace hardcoded margin
  and cyan label color on the UniFi switch section divider
- lt-text-amber / lt-text-cyan on stat card icons and values (matches
  same migration done in index.html)
- lt-stats-grid--mb: margin-bottom:16px on the summary stats grid
- g-page-sub-aside: replaces margin-left:8px on the updated timestamp
  span in links and inspector page subtitle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:19:32 -04:00
jaredandClaude Sonnet 4.6 d63f73e90f CSS: extract notification panel inline styles to classes
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- lt-notif-empty: replaces all hardcoded padding/font/color/align on
  the empty-state and loading/error text in the notification bell panel
- lt-notif-view-all: replaces width/text-align/display/font-size inline
  style on the 'View dashboard' footer link
- lt-notif-dot: moves border-radius:50%;margin-top from inline style
  (only background color remains inline, which is dynamic per-severity)
- Initial 'Loading…' text in the panel HTML uses lt-notif-empty

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:18:33 -04:00
jaredandClaude Sonnet 4.6 bc71cc829b CSS: replace remaining inline color/size styles with TDS utilities
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Stat card icons and values: style="color:var(--red)" etc replaced with
  lt-text-red, lt-text-amber, lt-text-cyan, lt-text-green (defined in
  base.css with both color and glow-shadow)
- Host search input: style="width:180px" extracted to .lt-search-input--sm
- base.html: suppress modal form groups use lt-form-group--last for last
  item (already committed); lt-divider--compact applied to settings divider

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:17:22 -04:00
jaredandClaude Sonnet 4.6 c3f0fcbefb CSS: remove remaining fixable inline styles across templates
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Suppress modal form groups: strip margin-bottom:12px (lt-form-group
  already has margin-bottom in TDS); use lt-form-group--last for the
  final group where zero margin is needed
- Keyboard shortcuts table: remove width:100% (lt-table is already full-
  width in base.css)
- Settings divider: replace style=margin override with .lt-divider--compact
- Topology bus section: move max-width:860px into .topo-bus-section rule

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:15:15 -04:00
jaredandClaude Sonnet 4.6 d8a4474814 CSS: convert all topology inline styles to modifier classes
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Replace style= attributes on topology wire/node elements with semantic
modifier classes:
- topo-vc-wire--wan, --10g, --mgmt (wire colour semantics in CSS)
- topo-v2-host--bus (bus-section node size constraint)
- topo-legend-item--offrack already done in prior commit

Zero inline styles remain in the topology section.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:13:24 -04:00
jaredandClaude Sonnet 4.6 9e0f8555cc Feature: show suppression status on active alert rows
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Active events now carry an is_suppressed boolean (added in api_status()
and the index() route via check_suppressed() against the pre-loaded
suppression list). The events table renders a muted '🔕 sup' badge next
to the severity and dims the entire row (.row-suppressed) so operators
can immediately see which firing alerts are silenced.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:11:15 -04:00
jaredandClaude Sonnet 4.6 f25a377968 Cleanup: strip redundant inline styles, add CSS classes
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Remove style="margin-top:4px" from .g-page-sub in all three secondary
  pages (the value is already defined in .g-page-sub rule in style.css)
- suppressions.html: replace inline style="padding:12px 14px" with TDS
  lt-section-body class
- index.html topology legend: replace inline dashed-border style with
  .topo-legend-item--offrack modifier class in style.css

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:09:25 -04:00
jaredandClaude Sonnet 4.6 7abb76afaa Redesign: alerts above fold, fix nav, globalise suppress modal
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Dashboard: reorder sections so Active Alerts appear immediately below
  stat cards rather than after 200+ lines of topology — primary content
  is now above the fold on all screen sizes
- Network Hosts section gains a collapse toggle (persisted to localStorage)
  so the topology diagram can be hidden when not needed
- Admin nav corrected: admin now gets a direct Suppressions link;
  non-admin no longer sees the page at all (it was always admin-only)
- Suppress modal moved from index.html into base.html so [S] keyboard
  shortcut works on every page, not just the dashboard; form listeners
  wired in app.js accordingly
- Settings modal KV grid: replaced lt-kv-row/lt-kv-label/lt-kv-value
  (light-mode only) with lt-kv-key/lt-kv-val (dark-mode defined)
- style.css: add lt-btn-secondary dark-mode definition, lt-cmd-hint-btn
  class for ⌘K button, topology collapse styles; remove dead .g-page-header,
  .g-page-title, .empty-state classes; strip redundant inline styles

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 23:07:53 -04:00
jaredandClaude Sonnet 4.6 6e3a579956 Cleanup: fix eslint warnings, button loading state, inspector footer hint
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- app.js: replace != with !== (eqeqeq rule) on resolved_24h null check
  and totalActive pagination check
- style.css: extend loading state to all .lt-btn.is-loading (not just
  refresh button), so suppressions form submit shows disabled feedback;
  remove dead .link-collapse-bar rule
- base.html: add inspector page to footer R=REFRESH hint; update
  keyboard shortcut table to include Inspector in refresh shortcut doc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-10 22:42:40 -04:00
jaredandClaude Sonnet 4.6 332f041e05 Dynamic resolved count, host search filter, lt-divider for UniFi section
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- db.py: add resolved_24h to get_status_summary() so each /api/status
  poll carries the fresh 24h resolved count
- app.js: wire stat-resolved-val to update from summary.resolved_24h
  so the Resolved 24h card stays accurate after auto-refresh
- index.html: add lt-toolbar/lt-search above host grid for quick
  client-side host filtering by name
- links.html: replace custom unifi-section-header div with lt-divider
- style.css: remove unused .unifi-section-header rules

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 18:36:57 -04:00
jaredandClaude Sonnet 4.6 66c2c6bc5f Fix B108: replace hardcoded /tmp with tempfile.gettempdir()
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Bandit flags hardcoded /tmp strings as CWE-377 (insecure temp file).
Use tempfile.gettempdir() for the avatar cache dir default so the
path resolves correctly on all platforms and passes the security scan.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 13:34:37 -04:00
jaredandClaude Sonnet 4.6 ddc258a453 TDS polish: lt-frame tables, links search toolbar, dead CSS cleanup
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- index.html: wrap UniFi devices table in lt-frame with section header
- links.html: add static lt-toolbar with lt-search filter and collapse
  controls above the dynamic container; remove collapse bar from
  renderLinks() since it's now static; add applyLinksSearch() to
  filter host/switch panels by name as user types
- suppressions.html: wrap Available Targets section in lt-frame
- style.css: remove unused .link-summary-panel and related rules
  (replaced by lt-stats-grid in previous commit)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 17:39:11 -04:00
jaredandClaude Sonnet 4.6 e6649a4c39 TDS polish: lt-frame tables, lt-stats-grid link summary, settings-aware refresh
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- links.html: replace custom link-summary-panel with lt-stats-grid/lt-stat-card
  showing total interfaces, ports down, errors, and PoE load
- suppressions.html: wrap active suppressions and history tables in lt-frame
  with lt-section-header labels
- inspector.html: wire auto-refresh to gandalfSettings (respects interval pill),
  fix updated timestamp to use locale time

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 17:15:48 -04:00
jaredandClaude Sonnet 4.6 f0759ed24c Upgrade page headers to lt-page-header/lt-page-title across all pages
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-01 01:09:30 -04:00
jaredandClaude Sonnet 4.6 9d15d4b802 Add stat cards, lt-frame alert queue, and timeline for resolved alerts
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Four lt-stat-card widgets (Critical, Warning, Hosts, Resolved 24h)
  below the status bar; Critical card pulses red when count > 0
- Clicking Critical or Warning card filters the events table by severity
- Events table wrapped in lt-frame with ASCII corner ornaments and
  lt-section-header; filter bar moved to lt-toolbar with lt-search icon
- Recently Resolved table replaced with lt-timeline component
- updateStatusBar() and updateHostGrid() keep stat card values live

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 22:19:50 -04:00
jaredandClaude Sonnet 4.6 6f9787b74a Add notification bell, settings modal, and context-sensitive footer
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Notification bell in header polls /api/status and shows active alerts
  with severity-colored dots; badge counts unread items via localStorage
- Settings modal ([ * ] CFG) controls auto-refresh interval (15s/30s/1m/5m/off)
  persisted to localStorage and wired into lt.autoRefresh on all pages
- Context-sensitive footer hints: Dashboard shows REFRESH + SUPPRESS,
  Link Debug shows REFRESH, all pages show CFG + HELP
- Added S key (quick suppress) and * key (settings) shortcuts
- ⌘K affordance button added to header-right
- R key now uses lt.autoRefresh.now() so it works on any page
- refreshAll() pushes fresh events to notification bell on each poll

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:33:02 -04:00
jaredandClaude Sonnet 4.6 032d0db81e Add LDAP avatar photos, UX polish, and TDS component upgrades
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Add /api/avatar endpoint querying lldap for user jpegPhoto; disk cache
  with sentinel pattern avoids repeat LDAP hits for users without photos
- Add ldap3 dependency and ldap config block to config.json
- Wire lt-avatar img overlay in base.html with capture-phase error
  fallback (lt-avatar-img-err) to reveal initials when image is absent
- Fix lt-avatar CSS shim: position:relative + absolute inset on img
  (local base.css was missing these; added to style.css)
- Replace all empty-state paragraphs with proper lt-empty-state markup
  (icon + title + body) across index, suppressions, inspector, app.js
- Add lt-spinner--cyan next to refresh button; shows during refreshAll()
- Replace inspector panel-section-title with lt-divider throughout
- Add data-tooltip attributes to SFP DOM metrics, TX/RX/Carrier/Duplex/
  Auto-neg/Error labels in links.html and inspector panel
- Add tooltips to events table column headers (Sev, First Seen, Failures)
- Fix links.html host panel timestamp (was reading sample.updated which
  is always undefined; now uses data.updated)
- Fix UniFi status text casing (Online→ONLINE to match server render)
- Remove dead topo-status-* class manipulation from updateTopology()
- Always render alert-count-badge; toggle display:none when count is 0
- Fix double UniFi get_devices() call in monitor.py run loop
- Fix chip-critical animation (was using green pulse-glow; now red)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 21:09:56 -04:00
jaredandClaude Sonnet 4.6 5fdb77d664 Integrate test code improvements using web_template components
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
lt-alert:
- Replace custom .stale-banner with lt-alert lt-alert--warning in app.js
  and links.html; remove stale-banner CSS, reuse lt-alert margin rule

lt-progress:
- Replace custom .traffic-bar-track/.traffic-bar-fill in links.html with
  lt-progress from base.css; TX uses default (orange), RX uses --cyan,
  both flip to --red when utilisation >85% (trafficBarClass helper)
- Keep traffic layout classes (.traffic-section/.traffic-row etc.) for structure

Suppression type badges:
- Map target_type to distinct badge colors: host→badge-warning (orange),
  interface→badge-info (cyan), unifi_device→badge-purple (new alias using
  --accent-purple from base.css), all→badge-critical (red)
- Applied in both server-rendered table (Jinja2 dict lookup) and
  renderActiveRows() JS

Topology animated down-wire:
- Add data-host attribute to .topo-v2-wire-10g/.topo-v2-wire-1g elements
- updateTopology() toggles .wire-down class on the 10G drop-wire when
  host.status === 'down'
- .wire-down CSS: animated repeating-linear-gradient dashed red line
  via wire-dash-anim @keyframes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 23:37:47 -04:00
jaredandClaude Sonnet 4.6 d05deb6ccd Remove all inline event handlers; replace with data-action delegation
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- inspector.html: onclick on port blocks, close button, run-diagnostic button,
  and diag-toggle sections all converted to data-action attributes; single
  delegated click listener handles all cases + Escape key closes panel
- links.html: onclick on panel title headers, Collapse All, Expand All
  converted to data-action with delegated listener
- suppressions.html: onsubmit/onchange wired via addEventListener at init
- index.html: onsubmit/onchange on suppress modal form wired at init

No behavioural changes — pure event-handling refactor for TDS compliance.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 17:53:48 -04:00
jaredandClaude Sonnet 4.6 3a9cc472e1 Audit quick wins: null guard, API error toasts, aria-labels on suppress buttons
- app.js: guard events array with || [] before .filter() to prevent crash on null
- app.js: show warning toast when /api/network or /api/status fail (was silent)
- app.js: add aria-label to all dynamically-generated suppress buttons
- index.html: add aria-label to server-rendered suppress buttons

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 17:50:00 -04:00
jaredandClaude Sonnet 4.6 54369bf6c3 fix: resolve bandit B324/B104 and flake8 E302/E303/E501 in app.py
Lint / Python (flake8) (push) Skipped
Lint / JS (eslint) (push) Skipped
- Add nosec B324 to md5 avatar-colour call (non-security deterministic hash)
- Extend nosec on host='0.0.0.0' to cover B104 alongside existing B201
- Fix E302 (missing blank line before template_filter decorator)
- Fix E303 (4 blank lines → 2 before _purge_old_jobs_loop)
- Add extend-exclude = node_modules to .flake8 so CI --exclude flag
  doesn't override config and third-party JS Python helpers stay ignored

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 20:51:41 -04:00
jaredandClaude Sonnet 4.6 a5fcd619f8 test: expand diagnose test coverage — parsers, dmesg, lldp, and analyze
Add 47 new tests covering parse_ethtool_driver, parse_nic_stats,
parse_ethtool_dom (SFP DOM), parse_ip_link, parse_dmesg, parse_lldpctl,
and the analyze() health-analysis method with all issue/warning/info
code paths (NO_CARRIER, HALF_DUPLEX, SPEED_MISMATCH, SFP thresholds,
CRC errors, carrier flapping, LLDP mismatch/missing).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 19:53:35 -04:00
jaredandClaude Sonnet 4.6 1767c12f1b Fix field name mismatches, add events filter, in-place suppression refresh
- links.html: fix all field name bugs (auto_negotiation→autoneg, full_duplex,
  tx/rx_errors/drops_per_sec→_rate, tx/rx_bytes_per_sec→_rate, poe_total_w/poe_max_w
  computed from ports, renderUnifiSwitches uses top-level updated timestamp)
- suppressions.html: in-place DOM refresh after create/remove (no page reload),
  datalist autocomplete for target names, form reset after submit
- inspector.html: ESC key closes detail panel via lt.keys.on
- index.html: events filter bar with search input + severity pills (All/Critical/Warning),
  MutationObserver re-applies filter after dynamic updates
- style.css: g-section-actions, events-filter-bar, sev-pills layout
- app.js/db.py/monitor.py: carry forward prior session fixes (Promise.allSettled,
  daemon_ok, stale connection handling, double Prometheus call, self.cfg fix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 23:35:02 -04:00
jaredandClaude Sonnet 4.6 6461afe12f Clean up suppressions.html: standardise pill attribute and remove inline onclick
- Rename data-dur → data-duration to match index.html/app.js convention
- Replace onclick="removeSuppression({{ s.id }})" with data-action="remove-sup" data-sup-id delegation
- Scope pill delegation to #create-suppression-form to avoid cross-page conflicts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 00:01:52 -04:00
jaredandClaude Sonnet 4.6 6473d932a7 Migrate inspector and links pages to TDS lt.* APIs
- Add escHtml alias (lt.escHtml) to both pages so existing template strings work without touching 40+ call sites
- Replace raw fetch() with lt.api.get/post in loadInspector, loadLinks, runDiagnostic, pollDiagnostic
- Replace setInterval(load*, 60000) with lt.autoRefresh.start() for intelligent polling
- Add lt.toast.error() to catch blocks for user-visible error feedback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 23:59:19 -04:00
jaredandClaude Sonnet 4.6 b31b5dd122 Add avatar color, initials structure, and admin nav dropdown
- app.py: avatar_color Jinja filter using deterministic hash → lt-avatar--orange/green/purple
- base.html: proper lt-avatar--sm with lt-avatar-initials span and color class; multi-word initials support
- base.html: admin users get lt-nav-dropdown for Suppressions; non-admins see flat link; mobile drawer hides Suppressions for non-admins

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 23:54:01 -04:00
jaredandClaude Sonnet 4.6 1e9ce08dca Integrate TDS v1.2 lt.* APIs throughout app
- app.js: replace raw fetch/escHtml/fmtRelTime with lt.api, lt.escHtml, lt.time.ago; modal open/close via lt.modal; add _toIso() for timestamp normalisation
- index.html: data-action="refresh", data-duration pills, lt.autoRefresh.start, remove local fmtRelTime
- suppressions.html: lt.api.post/delete, data-dur pill delegation
- base.html: user avatar with initials, admin badge, lt.keys.on('r') replaces manual keydown handler
- base.css: add dot-*, chip, row-state aliases so apps can use unprefixed class names

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 23:46:44 -04:00
jaredandClaude Sonnet 4.6 c0f8975dee Replace dangling symlinks with real base.css and base.js files
The symlinks pointed to /root/code/web_template/ which does not exist on
the production LXC (10.10.10.61). Flask returned 404 for both assets.
Replaced with actual file copies from web_template v1.2.

To update when web_template changes:
  cp /root/code/web_template/base.css static/base.css
  cp /root/code/web_template/base.js  static/base.js

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 22:11:44 -04:00
jaredandClaude Sonnet 4.6 51c18fee5c Align base.html and modal with tinker_tickets reference implementation
- Add lt-glitch + data-text to brand title (signature glitch effect)
- Add mobile nav drawer (lt-nav-drawer) and hamburger button (lt-menu-btn)
- Add VT323 font, theme toggle button (lt-theme-btn), footer with hints
- Add command palette overlay + lt.cmdPalette.init() with nav commands
- Add keyboard shortcuts help modal and skip link
- Move base.js to <head> so lt.* is available for inline scripts
- Fix suppress modal: lt-modal-backdrop → lt-modal-overlay (base.css class)
- Fix modal open/close: use .is-open / aria-hidden instead of inline style

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 21:42:04 -04:00
jaredandClaude Sonnet 4.6 b61501d81d Restructure app to use LotusGuild Terminal Design System v1.2
Replace custom phosphor-green terminal aesthetic with the lt-* component
system from base.css/base.js. All templates now inherit the LotusGuild
multi-accent Anduril palette via variable aliases in style.css, and use
lt-header, lt-nav, lt-card, lt-table, lt-btn, lt-modal, lt-badge etc.
Custom components (topology, inspector chassis, link debug, SFP panels)
are preserved with color values updated to base.css palette variables.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 21:01:20 -04:00
jaredandClaude Sonnet 4.6 3e36dc9505 ci: add notify-failure, deploy tagging, and coverage reporting
- lint.yml: add notify-failure Matrix alert job; add Tag deployed commit
  step to deploy job with deploy-YYYY.MM.DD-N tagging via Gitea API
- test.yml: add pytest-cov for coverage reporting
- .coveragerc: omit tests and site-packages from coverage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 15:16:02 -04:00
jaredandClaude Sonnet 4.6 0d4f792a46 Add CI badges and CI/CD section to README
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 12:53:47 -04:00
jaredandClaude Sonnet 4.6 9dc10422e3 Suppress bandit B201 false positive in dev runner
app.run(debug=True) is only reached via __main__ (dev mode).
Production runs gunicorn, never this block.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 12:40:10 -04:00
jaredandClaude Sonnet 4.6 903dc960c3 Add pytest test suite and security scanning
- Add 33 tests for DiagnosticsRunner static methods (build_ssh_command,
  parse_output, parse_sysfs_stats, parse_ethtool and variants)
- Add test.yml CI workflow running pytest on every push/PR
- Add security.yml CI workflow running bandit on every push/PR (weekly)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 12:22:57 -04:00
jaredandClaude Sonnet 4.6 6880abdff3 Add bandit security scanning workflow
Scans Python code weekly and on every push/PR for medium+ severity issues.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 11:04:39 -04:00
jaredandClaude Sonnet 4.6 a4fd74754c Add JS linting and deploy gating to CI pipeline
- Add js-lint job (ESLint 8) for static/ directory
- Add deploy job gated on both python-lint and js-lint passing
- Deploy triggers gandalf-deploy webhook on main branch only
- Add static/.eslintrc.json with browser env config
- Add node_modules/ to .gitignore

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 10:14:33 -04:00
jaredandClaude Sonnet 4.6 5c3db5e934 ci: install Python3 via apt before pip — runner is node:20-bullseye
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 00:04:42 -04:00
jaredandClaude Sonnet 4.6 3aff52f972 ci: add flake8 lint workflow; fix unused imports
Adds .gitea/workflows/lint.yml running flake8 with .flake8 config.
Removes unused imports (flask.redirect, flask.url_for, time, typing.Tuple).
Config ignores E221/E305 (intentional column alignment and function spacing).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 22:23:26 -04:00
jaredandClaude Sonnet 4.6 de5677fe4d Auto-reload on auth timeout (401 response)
Wrap window.fetch so any 401 triggers window.location.reload(),
sending the browser back through the Authelia proxy to the login page.
Covers all pages since app.js is loaded by base.html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 20:43:08 -04:00
jaredandClaude Sonnet 4.6 eb9fb3f88f Exclude LXC IPs from link stats collection
Add links_exclude_ips to monitor config; collect() skips any Prometheus
instance whose IP is in that list, preventing LXC containers from
appearing on the links/inspector pages as phantom hosts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 20:39:47 -04:00
jaredandClaude Sonnet 4.6 44d4a7d1eb Add pagination to event queries, input validation, daily event purge
- get_active_events() now takes limit/offset (default 200) to cap unbounded queries
- count_active_events() added to return total for pagination display
- /api/events supports ?limit=, ?offset=, ?status= query params (max 1000)
- /api/status includes total_active count alongside paginated events list
- index() route passes total_active to template for server-side truncation notice
- Show "Showing X of Y" notice in dashboard when events are truncated
- Suppression POST validates: reason ≤500 chars, target_name/detail ≤255 chars
- _purge_old_jobs_loop runs purge_old_resolved_events(90d) once per day

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 20:32:32 -04:00
jaredandClaude Sonnet 4.6 ee0ea8fd47 Fix host filtering: only show/monitor configured hosts; add PBS
- _collect_snapshot() and _process_interfaces() now skip any Prometheus
  instance not explicitly listed in config.json hosts[]. LXC app servers
  (postgresql, matrix, etc.) report node_exporter metrics but are not
  infrastructure hosts Gandalf should display or alert on.
- Add PBS (10.10.10.3) to config hosts[] with prometheus_instance;
  remove from ping_hosts (node_exporter already running on PBS, now
  added to Prometheus scrape config as job pbs-node).
- The _instance_map membership check is now consistent across snapshot,
  alerting, and ethtool SSH collection.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 17:17:40 -04:00
jaredandClaude Sonnet 4.6 95ec211037 Fix: only SSH into explicitly configured hosts for ethtool collection
LinkStatsCollector.collect() was SSHing into every host reporting
node_network_* metrics to Prometheus, including unrelated app servers
like postgresql and matrix. Add instance_map membership check so ethtool
collection via Pulse only runs on hosts defined in config.json.

Prometheus metrics (traffic rates, errors) are still collected for all
instances — only the SSH/ethtool step is gated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 18:35:21 -04:00
jaredandClaude Sonnet 4.6 253bc568d6 Improve Pulse execution reliability: retry logic, better logging, SSH hardening
monitor.py / diagnose.py PulseClient.run_command:
- Add automatic single retry on submit failure, explicit Pulse failure
  (status=failed/timed_out), and poll timeout — handles transient SSH
  or Pulse hiccups without dropping the whole collection cycle
- Log execution_id and full Pulse URL on every failure so failed runs
  can be found in the Pulse UI immediately
- Handle 'timed_out' and 'cancelled' Pulse statuses explicitly (previously
  only 'failed' was caught; others would spin until local deadline)
- Poll every 2s instead of 1s to reduce Pulse API chatter

SSH command options (_ssh_batch + diagnose.py):
- Add BatchMode=yes: aborts immediately instead of hanging on a
  password prompt if key auth fails
- Add ServerAliveInterval=10 ServerAliveCountMax=2: SSH detects a
  hung remote command within ~20s instead of sitting silent until the
  45s Pulse timeout expires

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 09:19:07 -04:00
jaredandClaude Sonnet 4.6 892d1cfe2d Fix topology chain order and inspector SFP port width
Topology:
- Correct series layout: UDM-Pro → USW-Agg → Pro 24 PoE (not a fork)
- Remove CSS fork divs, replace with straight vertical connectors
- Labels: WAN · 10G SFP+ (UDM→Agg), 10G trunk (Agg→PoE)
- Remove ISL from legend (no parallel switch pair)

Inspector:
- Fix USW-Agg port blocks appearing narrower than other switches
- SFP ports in rows now use same width (34px) as copper ports;
  all-SFP switches like USL8A no longer look undersized

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 22:42:38 -04:00
jaredandClaude Sonnet 4.6 834acfd393 Fix topology diagram: replace SVG fork with CSS, fix line alignment
- Remove SVG fork with preserveAspectRatio="none" (caused line width
  distortion and stretched 10G DAC label like a tube TV)
- Replace with pure CSS .topo-fork: stem + horizontal bar + left/right
  drops, all absolutely positioned at consistent 2px width
- Use .topo-sw-row with two 50% halves so switch centres land at
  exactly 25% and 75% — matching fork drop positions mathematically
- ISL rendered via ::before/::after on .topo-sw-row (switch boxes
  with solid bg cover the line at their edges, leaving only the gap)
- Add .topo-sw-drops: two vertical stubs from switch centres to bus rails
- All lines are now exactly 2px, no distortion, no misalignment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 22:35:02 -04:00
jaredandClaude Sonnet 4.6 148fc57cc2 Redesign topology diagram with dual-homed bus layout and improve inspector chassis
- Replace flat topology with tiered bus-bar layout: Internet → UDM-Pro → SVG fork → USW-Agg + Pro 24 PoE → dual-homed servers
- Show 10G VLAN90 (Ceph) bus from USW-Agg and 1G DHCP management bus from Pro 24 PoE per host
- Add per-host drop wires (solid 10G + dashed 1G) with correct rack positions
- Mark large1 as off-rack (dashed border), ZimaBoards as off-rack mon-01/mon-02
- Add topology legend, inter-switch 10G ISL indicator
- Add recently resolved events section (last 24h) to dashboard
- Add last_seen column and relative timestamps to events table
- Add stale data banner when monitoring data >15 min old
- Improve inspector chassis with port speed labels, LLDP neighbor info, mounting ears, chassis legend
- Add duplex/speed mismatch warnings and carrier changes to path debug panel
- Bump updateTopology() to handle both topo-v2-status-* and topo-status-* classes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 22:22:19 -04:00
jaredandClaude Sonnet 4.6 35d785db64 fix: topology — reflect VLAN90 Ceph network and DHCP management separation
10G SFP+ ports on USW-Agg are VLAN90 (10.10.90.x/24, static IPs, Ceph storage).
1G ports on Pro 24 PoE are DHCP management. Update topology to show this:
- USW-Agg sublabel shows VLAN90 · 10.10.90.x (cyan)
- Pro 24 PoE sublabel shows DHCP mgmt (cyan)
- Host sublabels changed from "10G+1G" to "VLAN90" for the 10G Agg connection
- 1G management band label updated to "← 1G DHCP mgmt (Pro 24 PoE) →"
- Add .topo-vlan-tag CSS for cyan VLAN annotation on switch nodes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 22:10:17 -04:00
jaredandClaude Sonnet 4.6 f3753092e0 fix: topology — all servers dual-homed 10G+1G, show mgmt band
All rack servers (and large1 on table) have both a 10G link to USW-Agg
and a 1G management link to Pro 24 PoE. Update topology:
- Move all 6 hosts into single row (including large1)
- Update sublabels to "10G+1G" for all nodes
- large1 dashed-border (off-rack) with "table · 10G+1G"
- Add dashed amber "1G mgmt (PoE)" horizontal band above hosts
  to represent the PoE switch management connections
- 10G primary fan-out lines still drop from Agg switch above
- large1 primary line rendered as dashed green (off-rack run)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 22:08:48 -04:00
jaredandClaude Sonnet 4.6 55989f4c50 feat: redesign network topology diagram with accurate rack layout
Replace linear Internet→UDM→Agg→PoE→all-hosts chain with accurate topology:
- USW-Aggregation and Pro 24 PoE switch shown side-by-side with horizontal
  10G SFP+ link between them (not in series)
- 5 compute/storage/monitor nodes fanned out under Agg Switch with 10G labels
  and rack unit positions (RU4–12, RU14–17) as sublabels
- large1 shown separately under PoE switch, dashed border = off-rack (table)
- Add device specs as subtitles on all nodes (Dream Machine Pro · RU24, etc.)
- Shorter display names: csg-01 / cs-01 instead of full hostnames
- Live status badges still updated by JS via data-host attributes
- New CSS: .topo-node-sub, .topo-switch-tier, .topo-h-link, .topo-host-tier,
  .topo-host-table (dashed), .topo-badge-unknown

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 22:06:03 -04:00
jaredandClaude Sonnet 4.6 bc1d3fb33b Remove aesthetic_diff.md reference from README — convergence complete
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 21:50:02 -04:00
jaredandClaude Sonnet 4.6 3840d2ccb7 feat: link health summary, recently resolved panel, event duration
- dashboard: pass recent_resolved (last 24h, limit 10) to index template;
  render "Recently Resolved" section showing type, target, resolved time,
  and calculated duration (first_seen → resolved_at)
- dashboard: event-age spans now also update via setInterval; duration
  shown for resolved events (e.g. "2h 15m")
- links page: link health summary panel shows server iface count,
  error/flap counts, switch port up/down, PoE total draw/capacity bar;
  only shows problematic stats if non-zero; shows "All OK ✔" when clean
- style.css: new classes for summary panel, resolved row/badge

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 21:48:40 -04:00
jaredandClaude Sonnet 4.6 df7b7e36e5 feat: UI improvements — event ages, error badges, PoE bars, mismatch detection
- events table: add Last Seen column; show relative times ("3h ago") with
  absolute timestamp on hover; update updateEventsTable() in app.js to match
- links.html: add error/drop/flap alert badges to interface and port card headers
- links.html: PoE power bar (draw/max ratio with colour-coded fill) and poe_mode
- links.html: stale data warning banner when link_stats are >2 minutes old
- links.html: improved error handler shows HTTP status instead of generic message
- links.html: fix collapse state persisted to localStorage (was sessionStorage,
  lost on browser restart); fix collapseAll/expandAll to also persist state
- inspector.html: duplex mismatch and speed mismatch warnings in path debug panel
- inspector.html: carrier changes added to server column of path debug
- style.css: new classes — .link-alert-badge, .poe-bar-*, .path-mismatch-alert,
  .error-state; fix .stale-banner to use CSS variables

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 21:46:11 -04:00
jaredandClaude Sonnet 4.6 fb6848b7e0 Apply LotusGuild design system convergence (aesthetic_diff.md)
CSS (style.css):
- §1: Add unified naming aliases (--terminal-green, --bg-primary, etc.)
- §2: Upgrade borders: modal 1px→3px double, btn/btn-sm/inputs 1px→2px
- §3: Add [ ] bracket decorations to .btn classes; primary keeps > prefix;
  hover lift -1px→-2px; padding 6px 14px→5px 12px
- §4: Fix glow definitions from 2-layer rgba to 3-layer solid stack
- §5: Section headers now symmetric ╠═══ TITLE ═══╣ (was one-sided)
- §6+§7: Modal border 3px double, corners ┌┐→╔╗, add glow shadow
- §11: Nav active state now amber tint (was green); hover remains green
- §15: Scanline opacity 0.13→0.15; flicker delay 45s→30s

JS (app.js):
- §18: Replace custom showToast() with lt.toast.* delegate wrapper

Templates (base.html):
- Load base.css and base.js (symlinked from web_template)
- Add lt-boot overlay for boot sequence animation (§13)

README: Remove completed pending convergence items

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 21:40:20 -04:00
jaredandClaude Sonnet 4.6 34758768a4 New features: stale banner, tab title alerts, health checks, DB housekeeping
static/app.js:
- Browser tab title updates to show alert count: '(3 CRIT) GANDALF' or '(2 WARN) GANDALF'
- Stale monitoring banner: injected into .main if last_check > 15 min old,
  warns operator that the monitor daemon may be down

static/style.css:
- .stale-banner: amber top-border warning strip

app.py:
- /health now checks DB connectivity and monitor freshness (last_check age)
  Returns 503 + degraded status if DB unreachable or monitor stale >20min

db.py:
- cleanup_expired_suppressions(): marks time-limited suppressions inactive when
  expires_at <= NOW() (was only filtered in SELECTs, never marked inactive)
- purge_old_resolved_events(days=90): deletes old resolved events to prevent
  unbounded table growth

monitor.py:
- Calls cleanup_expired_suppressions() and purge_old_resolved_events() each cycle

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 21:35:32 -04:00
jaredandClaude Sonnet 4.6 a7c3f97252 De-hardcode ticket URL and cluster name; improve diagnostic polling UX
app.py:
- Context processor injects config.ticket_api.web_url into all templates
  (falls back to 'http://t.lotusguild.org/ticket/' if not set in config)

templates/base.html:
- Inject GANDALF_CONFIG JS global with ticket_web_url before app.js loads

static/app.js:
- Use GANDALF_CONFIG.ticket_web_url instead of hardcoded domain

templates/index.html:
- Use {{ config.ticket_api.web_url }} Jinja var instead of hardcoded domain

monitor.py:
- CLUSTER_NAME constant kept as default; NetworkMonitor now reads cluster_name
  from config monitor.cluster_name, falling back to the constant
- All CLUSTER_NAME references inside class methods replaced with self.cluster_name

templates/inspector.html:
- pollDiagnostic() .catch() now clears interval and shows error message instead
  of silently ignoring network failures during active polling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 14:31:57 -04:00
jaredandClaude Sonnet 4.6 4413bb8b4c Add compound DB indexes for hot query paths
network_events: idx_event_lookup (event_type, target_name, target_detail, resolved_at)
  - Covers the upsert_event SELECT which runs every cycle per monitored entity
  - Replaces three separate single-column index scans with one covering lookup

suppression_rules: idx_sup_lookup (active, target_type, target_name, target_detail)
  - Covers is_suppressed() queries (now redundant for runtime due to in-memory
    check_suppressed, but ensures fast get_active_suppressions() loading per cycle)

Both indexes created on live DB (MariaDB LXC 149).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 14:24:40 -04:00
jaredandClaude Sonnet 4.6 df2a2cc48c Optimize suppression checks: load once per cycle, add error logging
db.py:
- Add check_suppressed(suppressions, ...) for in-memory suppression lookups
  against pre-loaded list (eliminates N*M DB queries per monitoring cycle)
- get_baseline(): log error instead of silently swallowing JSON parse failure

monitor.py:
- Load active suppressions once per cycle at the top of the alert loop
- Pass suppressions list to _process_interfaces, _process_unifi, _process_ping_hosts
- Replace all db.is_suppressed() calls with db.check_suppressed(suppressions, ...)
- Reduces DB queries from 100-600+ per cycle down to 1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 14:13:54 -04:00
jaredandClaude Sonnet 4.6 c51c24bbb5 Fix setDur implicit event, title XSS, hardcoded pulse URL, suppress error toast
- suppressions.html: setDur() now takes explicit element param instead of relying
  on implicit global event.target (which fails outside direct click handlers)
- suppressions.html: removeSuppression() now shows error toast on failed DELETE
- templates/index.html: escape description in title attribute with |e filter
  to prevent attribute breakout on quotes in description text
- diagnose.py: derive Pulse execution URL from pulse_client.url instead of
  hardcoding http://pulse.lotusguild.org

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 14:36:55 -04:00
jaredandClaude Sonnet 4.6 4ea5ae61e4 Fix port_idx type coercion and add logging to silent except blocks
- port_idx now coerced to int() with 400 on invalid type (prevents string/int mismatch)
- api_network and api_links bare except blocks now log errors instead of silently passing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 17:35:41 -04:00
jaredandClaude Sonnet 4.6 300b1cc68c Security and reliability fixes: input validation, logging, job cleanup
- C5: Validate host_ip (IPv4 check) and iface (allowlist regex) before SSH command builder
- H6: Upgrade Pulse failure logging from debug to error so operators see outages
- M6: Replace per-request O(n) purge with background daemon thread (runs every 2 min)
- M7: Background thread marks jobs stuck in 'running' > 5 min as errored

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 17:30:50 -04:00
jaredandClaude Sonnet 4.6 18112541c8 feat: deep link diagnostics via Pulse SSH
Adds comprehensive per-port link troubleshooting triggered from the
Inspector panel when a port has an LLDP-identified server counterpart.

- diagnose.py: DiagnosticsRunner with 15-section SSH command (carrier,
  operstate, sysfs counters, ethtool, ethtool -i/-a/-g/-S/-m, ip link,
  ip addr, ip route, dmesg, lldpctl); parsers for all sections; health
  analyzer with 14 check codes (NO_CARRIER, HALF_DUPLEX, SPEED_MISMATCH,
  SFP_RX_CRITICAL, CARRIER_FLAPPING, CRC_ERRORS_HIGH, LLDP_MISMATCH, etc.)
- monitor.py: PulseClient now tracks last_execution_id so callers can
  link back to the raw Pulse execution URL
- app.py: POST /api/diagnose + GET /api/diagnose/<job_id> with daemon
  thread background execution and 10-minute in-memory job store
- inspector.html: "Run Link Diagnostics" button (shown only when LLDP
  host is resolvable); full results panel: health banner, physical layer,
  SFP/DOM with power bars, NIC error counters, collapsible ethtool -S,
  flow control/ring buffers, driver info, LLDP 2-col validation,
  collapsible dmesg, switch port summary, "View in Pulse" link
- style.css: all .diag-* CSS classes with terminal aesthetic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:03:54 -05:00
jaredandClaude Sonnet 4.6 468dbda7a1 feat: inspector page, link debug enhancements, security hardening
- Add /inspector page: visual model-accurate switch chassis diagrams
  (USF5P, USL8A, US24PRO, USPPDUP, USMINI), clickable port blocks
  with color coding (green=up, amber=PoE, cyan=uplink, grey=down),
  detail panel with stats/PoE/LLDP, LLDP-based path debug side-by-side

- Link Debug: port number badges (#N), LLDP neighbor line, PoE class/max,
  collapsible host/switch panels with sessionStorage persistence

- monitor.py: collect LLDP neighbor map + PoE class/max/mode per switch
  port; PulseClient uses requests.Session() for HTTP keep-alive; add
  shlex.quote() around interface names (defense-in-depth)

- Security: suppress buttons use data-* attrs + delegated click handler
  instead of inline onclick with Jinja2 variable interpolation; remove
  | safe filter from user-controlled fields in suppressions.html;
  setDuration() takes explicit el param instead of implicit event global

- db.py: thread-local connection reuse with ping(reconnect=True) to
  avoid a new TCP handshake per query

- .gitignore: add config.json (contains credentials), __pycache__

- README: full rewrite covering architecture, all 4 pages, alert logic,
  config reference, deployment, troubleshooting, security notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 15:39:48 -05:00
jaredandClaude Sonnet 4.6 c8e183fc4f feat: terminal aesthetic rewrite + link debug page
- Full dark terminal aesthetic (Pulse/TinkerTickets style):
  - #0a0a0a background, #00ff41 green, #ffb000 amber, #00ffff cyan
  - CRT scanline overlay, phosphor glow, ASCII corner pseudoelements
  - Bracket-notation badges [CRITICAL], monospace font throughout
  - style.css, base.html, index.html, suppressions.html all rewritten

- New Link Debug page (/links, /api/links):
  - Per-host, per-interface cards with speed/duplex/port type/auto-neg
  - Traffic bars (TX cyan, RX green) with rate labels
  - Error/drop counters, carrier change history
  - SFP/DOM optical panel: vendor, temp, voltage, bias, TX/RX power dBm bars
  - RX-TX delta shown; color-coded warn/crit thresholds
  - Auto-refresh every 60s, anchor-jump to #hostname

- LinkStatsCollector in monitor.py:
  - SSHes to each host (one connection, all ifaces batched)
  - Parses ethtool + ethtool -m (SFP DOM) output
  - Merges with Prometheus traffic/error/carrier metrics
  - Stores as link_stats in monitor_state table

- config.json: added ssh section for ethtool collection
- app.js: terminal chip style consistency (uppercase, ● bullet)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-02 12:43:11 -05:00
jared 124a8c4ea2 chore: remove deploy test line from README 2026-03-02 12:08:16 -05:00
jared 565bd4ee24 chore: test auto-deploy pipeline 2026-03-02 12:05:59 -05:00
jared 150300ff91 chore: clean up deploy test line from README 2026-03-02 12:00:46 -05:00
jared 997c626c3c chore: trigger deploy test 2026-03-02 11:58:42 -05:00
jaredandClaude Sonnet 4.6 c201604dca docs: update README for storage-01 Prometheus migration
- storage-01 now monitored via Prometheus node_exporter (10.10.10.11:9100),
  removed from ping_hosts
- Updated data sources table (6 hosts via Prometheus, pbs only via ping)
- Added storage-01 to monitored hosts table
- Fixed Authelia reload command (restart, not reload)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 23:05:27 -05:00
jaredandClaude Sonnet 4.6 04a5182259 Complete rewrite: full-featured network monitoring dashboard
- Two-service architecture: Flask web app (gandalf.service) + background
  polling daemon (gandalf-monitor.service)
- Monitor polls Prometheus node_network_up for physical NIC states on all
  6 hypervisors (added storage-01 at 10.10.10.11:9100)
- UniFi API monitoring for switches, APs, and gateway device status
- Ping reachability for hosts without node_exporter (pbs only now)
- Smart baseline: interfaces first seen as down are never alerted on;
  only UP→DOWN regressions trigger tickets
- Cluster-wide P1 ticket when 3+ hosts have genuine simultaneous
  interface regressions (guards against false positives on startup)
- Tinker Tickets integration with 24-hour hash-based deduplication
- Alert suppression: manual toggle or timed windows (30m/1h/4h/8h)
- Authelia SSO via forward-auth headers, admin group required
- Network topology: Internet → UDM-Pro → Agg Switch (10G DAC) →
  PoE Switch (10G DAC) → Hosts
- MariaDB schema, suppression management UI, host/interface cards

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-01 23:03:18 -05:00
jared 4a66c41d18 added git ignore 2025-03-01 13:34:25 -05:00
jared 517843886b interface update 2025-02-08 00:32:25 -05:00
jared ec1e8055de fixed indent 2025-02-08 00:16:45 -05:00
jared 773802815a lots logs 2025-02-08 00:16:06 -05:00
jared 3688cf6d62 added raw back 2025-02-08 00:13:06 -05:00
jared c8fb605dbb more dynamic 2025-02-08 00:11:28 -05:00
jared 2498388715 fix syntax error 2025-02-08 00:04:42 -05:00
jared 640a3b9993 dynamic devices 2025-02-08 00:03:01 -05:00
jared a7b164f340 interfaces update 2025-02-07 23:57:34 -05:00
jared 6f79dc905d update index 2025-02-07 23:54:28 -05:00
jared 705be36c3f updated index html 2025-02-07 23:51:13 -05:00
jared 9188401374 update html 2025-02-07 23:38:49 -05:00
jared 59546cd080 json update 2025-02-07 23:26:41 -05:00
jared 6b0ef3fcb6 v2 api 2025-02-07 23:24:36 -05:00
jared 2b1af750f4 wrong indentation 2025-02-07 23:20:14 -05:00
jared 361e85fd69 Bearer token 2025-02-07 23:19:50 -05:00
jared b3b118395a Auth order plz 2025-02-07 23:17:12 -05:00
jared bc2590dc34 CSRF token 2025-02-07 23:14:36 -05:00
jared df460afe68 bearer token 2025-02-07 23:12:29 -05:00
jared 0438bd32ce updated stats 2025-02-07 23:09:11 -05:00
jared 05671360cd acquire site id 2025-02-07 23:06:49 -05:00
jared f3b9d26ec2 site id to default 2025-02-07 23:01:58 -05:00
jared 27a1707b81 DEBUG BROSKI 2025-02-07 22:54:43 -05:00
jared dfdafc07c4 fixed endpoint 2025-02-07 22:52:19 -05:00
jared 8c72a1494a new header 2025-02-07 22:50:52 -05:00
jared 3115b714aa raw data 2025-02-07 22:48:55 -05:00
jared 4edffca6af new endpoints again 2025-02-07 22:47:04 -05:00
jared 231de5714b legacy endpoints 2025-02-07 22:44:45 -05:00
jared bb2f6b96cf device and site id 2025-02-07 22:42:47 -05:00
jared d37e58af15 initializes the session before attempting to use it 2025-02-07 22:31:07 -05:00
jared 4f126f1ffb get site id 2025-02-07 22:29:26 -05:00
jared 071f5e954d x-api- 2025-02-07 22:27:26 -05:00
jared 646a1c3687 json instead of html 2025-02-07 22:24:32 -05:00
jared 0894f35cde raw response 2025-02-07 22:23:15 -05:00
jared 09823f8619 updated json parsing 2025-02-07 22:21:34 -05:00
jared 62a4ec16a2 revert 2025-02-07 22:20:20 -05:00
jared bd5707160a Bearer token 2025-02-07 22:17:27 -05:00
jared a21c21fbfc attempt new login method 2025-02-07 22:14:57 -05:00
jared 786c4be2ff updated auth 2025-02-07 22:09:09 -05:00
jared 1a4ffa107d updated api endpoint 2025-02-07 22:04:41 -05:00
jared 637b9876a3 more loggging 2025-02-07 22:03:19 -05:00
jared d7b7b1d17a debug api 2025-02-07 21:58:35 -05:00
jared b2a183b0e2 debug more 2025-02-07 21:56:30 -05:00
jared fa7ec4cfe2 identify device by ip in api 2025-02-07 21:53:12 -05:00
jared 0b068c824b more debug 2025-02-07 21:38:48 -05:00
jared d4a28b37fb added debug for api calls 2025-02-07 21:36:44 -05:00
jared 21bd5d8dbf added diag content 2025-02-07 21:33:55 -05:00
jared a35f0527e3 Added interface status 2025-02-07 21:28:54 -05:00
jared d671916100 Update file structure for Flask 2025-02-07 21:22:43 -05:00
jared c2cfa6370e sorted code 2025-02-07 21:03:31 -05:00
jared eefa162863 dynamic overview 2025-02-07 20:58:23 -05:00
jared 8c81c19785 integrated with unifi api 2025-02-07 20:55:38 -05:00
jared f3eab76c63 added troub 2025-02-07 20:31:56 -05:00
jared 9e091cd8fe added functionality 2025-01-04 01:42:16 -05:00
jared 5173302eb4 ui updates 2025-01-04 01:27:49 -05:00
jared d4cdad7006 added a paragraph 2025-01-04 01:16:06 -05:00
jared 8f73514297 test 2025-01-04 01:13:54 -05:00
jared 21dfad35bf made everything static 2025-01-04 01:07:18 -05:00
jared 81ba85845b test change 2025-01-04 00:57:42 -05:00
jared 109dff1cd0 test 2025-01-04 00:33:04 -05:00
jared a746140e8b first commit 2025-01-04 00:07:15 -05:00
30 changed files with 17906 additions and 0 deletions

No files matched your search

+7
View File
@@ -0,0 +1,7 @@
[run]
omit =
tests/*
*/site-packages/*
[report]
show_missing = True
+21
View File
@@ -0,0 +1,21 @@
{
"env": {
"browser": true,
"es2021": true
},
"globals": {
"lt": "readonly",
"GANDALF_CONFIG": "readonly",
"CSS": "readonly"
},
"rules": {
"no-undef": "error",
"no-unused-vars": ["warn", { "argsIgnorePattern": "^_", "varsIgnorePattern": "^_" }],
"no-console": "off",
"eqeqeq": ["error", "always", { "null": "ignore" }]
},
"parserOptions": {
"ecmaVersion": 2021,
"sourceType": "script"
}
}
+7
View File
@@ -0,0 +1,7 @@
[flake8]
max-line-length = 120
# E221: multiple spaces before operator — intentional column alignment
# E305: two blank lines after function — relaxed for module-level code
extend-ignore = E221, E305
exclude = __pycache__, .git, node_modules
extend-exclude = node_modules
+86
View File
@@ -0,0 +1,86 @@
name: Lint
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
jobs:
python-lint:
name: Python (flake8)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Python and flake8
run: |
apt-get update -qq
apt-get install -y -qq python3 python3-pip
pip3 install flake8
- name: Run flake8
run: flake8 . --exclude=__pycache__,.git
js-lint:
name: JS (eslint)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install ESLint
run: npm install --save-dev eslint@8
- name: Run ESLint
run: npx eslint --ext .js static/
notify-failure:
name: Notify on failure
runs-on: ubuntu-latest
needs: [python-lint, js-lint]
if: failure() && github.event_name == 'push'
steps:
- name: Send Matrix alert
env:
MATRIX_WEBHOOK_URL: ${{ secrets.MATRIX_WEBHOOK_URL }}
REPO: ${{ github.repository }}
BRANCH: ${{ github.ref_name }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
if [ -z "$MATRIX_WEBHOOK_URL" ] || [ "$MATRIX_WEBHOOK_URL" = "CONFIGURE_ME" ]; then exit 0; fi
curl -sf -X POST "$MATRIX_WEBHOOK_URL" \
-H "Content-Type: application/json" \
-d "{\"text\":\"CI FAILED: ${REPO} @ ${BRANCH} — ${RUN_URL}\"}"
deploy:
name: Deploy
runs-on: ubuntu-latest
needs: [python-lint, js-lint]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: write
steps:
- name: Trigger webhook
env:
WEBHOOK_SECRET: ${{ secrets.WEBHOOK_SECRET }}
GIT_REF: ${{ github.ref }}
run: |
PAYLOAD="{\"ref\":\"${GIT_REF}\"}"
SIG=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$WEBHOOK_SECRET" | awk '{print $2}')
curl -sf --connect-timeout 10 \
-X POST \
-H "Content-Type: application/json" \
-H "X-Gitea-Signature: ${SIG}" \
-d "$PAYLOAD" \
"http://10.10.10.61:9000/hooks/gandalf-deploy"
- name: Tag deployed commit
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="deploy-$(date -u +%Y.%m.%d)-${{ github.run_number }}"
curl -sf -X POST \
-H "Authorization: token $GITHUB_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"tag_name\":\"${TAG}\",\"target\":\"${{ github.sha }}\",\"message\":\"Deployed to production\"}" \
"https://code.lotusguild.org/api/v1/repos/${{ github.repository }}/tags"
+25
View File
@@ -0,0 +1,25 @@
name: Security
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
schedule:
- cron: '0 6 * * 1'
jobs:
bandit:
name: Python Security (bandit)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install bandit
run: |
apt-get update -qq
apt-get install -y -qq python3 python3-pip
pip3 install bandit
- name: Run bandit
run: bandit -r . --exclude .git,__pycache__,node_modules -ll
+24
View File
@@ -0,0 +1,24 @@
name: Test
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
jobs:
pytest:
name: Python Tests (pytest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Python and dependencies
run: |
apt-get update -qq
apt-get install -y -qq python3 python3-pip
pip3 install pytest pytest-cov
pip3 install -r requirements.txt --quiet
- name: Run pytest with coverage
run: python3 -m pytest tests/ -v --cov=. --cov-report=term-missing --cov-config=.coveragerc
+5
View File
@@ -0,0 +1,5 @@
log.txt
config.json
__pycache__/
*.pyc
node_modules/
Binary file not shown.

Before

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 212 KiB

+509
View File
@@ -0,0 +1,509 @@
# GANDALF (Global Advanced Network Detection And Link Facilitator)
[![Lint](https://code.lotusguild.org/LotusGuild/gandalf/actions/workflows/lint.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/gandalf/actions?workflow=lint.yml)
[![Test](https://code.lotusguild.org/LotusGuild/gandalf/actions/workflows/test.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/gandalf/actions?workflow=test.yml)
[![Security](https://code.lotusguild.org/LotusGuild/gandalf/actions/workflows/security.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/gandalf/actions?workflow=security.yml)
> Because it shall not let problems pass.
Network monitoring dashboard for the LotusGuild Proxmox cluster.
Deployed on **LXC 157** (monitor-02 / 10.10.10.9), reachable at `gandalf.lotusguild.org`.
**Design System**: [web_template](https://code.lotusguild.org/LotusGuild/web_template) — shared CSS, JS, and layout patterns for all LotusGuild apps
## Styling & Layout
GANDALF uses the **LotusGuild Terminal Design System**. For all styling, component, and layout documentation see:
- [`web_template/README.md`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/README.md) — full component reference, CSS variables, JS API
- [`web_template/base.css`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/base.css) — unified CSS (`.lt-*` classes)
- [`web_template/base.js`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/base.js) — `window.lt` utilities (toast, modal, auto-refresh, fetch helpers)
- [`web_template/python/base.html`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/python/base.html) — Jinja2 base template
- [`web_template/python/auth.py`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/python/auth.py) — `@require_auth` decorator pattern
---
## Architecture
Two processes share a MariaDB database:
| Process | Service | Role |
|---|---|---|
| `app.py` | `gandalf.service` | Flask web dashboard (gunicorn, port 8000) |
| `monitor.py` | `gandalf-monitor.service` | Background polling daemon |
```
[Prometheus :9090] ──▶
[UniFi Controller] ──▶ monitor.py ──▶ MariaDB ◀── app.py ──▶ nginx ──▶ Authelia ──▶ Browser
[Pulse Worker] ──▶
[SSH / ethtool] ──▶
```
### Data Sources
| Source | What it provides |
|---|---|
| **Prometheus** (`10.10.10.48:9090`) | Physical NIC link state + traffic/error rates via `node_exporter` |
| **UniFi API** (`https://10.10.10.1`) | Switch port stats, device status, LLDP neighbor table, PoE data |
| **Pulse Worker** | SSH relay — runs `ethtool` + SFP DOM queries on each Proxmox host |
| **Ping** | Reachability for hosts without `node_exporter` (e.g. PBS) |
### Monitored Hosts (Prometheus / node_exporter)
| Host | Prometheus Instance |
|---|---|
| large1 | 10.10.10.2:9100 |
| compute-storage-01 | 10.10.10.4:9100 |
| micro1 | 10.10.10.8:9100 |
| monitor-02 | 10.10.10.9:9100 |
| compute-storage-gpu-01 | 10.10.10.10:9100 |
| storage-01 | 10.10.10.11:9100 |
Ping-only (no node_exporter): **pbs** (10.10.10.3)
---
## Pages
### Dashboard (`/`)
- Real-time host status grid with per-NIC link state (UP / DOWN / degraded)
- Network topology diagram (Internet → Gateway → Switches → Hosts)
- UniFi device table (switches, APs, gateway)
- Active alerts table with severity, target, consecutive failures, ticket link
- Quick-suppress modal: apply timed or manual suppression from any alert row
- Auto-refreshes every 30 seconds via `/api/status` + `/api/network`
### Link Debug (`/links`)
Per-interface statistics collected every poll cycle. All panels are collapsible
(click header or use Collapse All / Expand All). Collapse state persists across
page refreshes via `sessionStorage`.
**Server NICs** (via Prometheus + SSH/ethtool):
- Speed, duplex, auto-negotiation, link detected
- TX/RX rate bars (bandwidth utilisation % of link capacity)
- TX/RX error and drop rates per second
- Carrier changes (cumulative since boot — watch for flapping)
- **SFP / Optical panel** (when SFP module present): vendor/PN, temp, voltage,
bias current, TX power (dBm), RX power (dBm), RX−TX delta, per-stat bars
**UniFi Switch Ports** (via UniFi API):
- Port number badge (`#N`), UPLINK badge, PoE draw badge
- LLDP neighbor line: `→ system_name (port_id)` when neighbor is detected
- PoE class and max wattage line
- Speed, duplex, auto-neg, TX/RX rates, errors, drops
### Inspector (`/inspector`)
Visual switch chassis diagrams. Each switch is rendered model-accurately using
layout config in the template (`SWITCH_LAYOUTS`).
**Port block colours:**
| Colour | State |
|---|---|
| Green | Up, no active PoE |
| Amber | Up with active PoE draw |
| Cyan | Uplink port (up) |
| Grey | Down |
| White outline | Currently selected |
**Clicking a port** opens the right-side detail panel showing:
- Link stats (status, speed, duplex, auto-neg, media type)
- PoE (class, max wattage, current draw, mode)
- Traffic (TX/RX rates)
- Errors/drops per second
- **LLDP Neighbor** section (system name, port ID, chassis ID, management IPs)
- **Path Debug** (auto-appears when LLDP `system_name` matches a known server):
two-column comparison of the switch port stats vs. the server NIC stats,
including SFP DOM data if the server side has an SFP module
**LLDP path debug requirements:**
1. Server must run `lldpd`: `apt install lldpd && systemctl enable --now lldpd`
2. `lldpd` hostname must match the key in `data.hosts` (set via `config.json → hosts`)
3. Switch has LLDP enabled (UniFi default: on)
**Supported switch models** (set `SWITCH_LAYOUTS` keys to your UniFi model codes):
| Key | Model | Layout |
|---|---|---|
| `USF5P` | UniFi Switch Flex 5 PoE | 4×RJ45 + 1×SFP uplink |
| `USL8A` | UniFi Switch Lite 8 PoE | 8×SFP (2 rows of 4) |
| `US24PRO` | UniFi Switch Pro 24 | 24×RJ45 staggered + 2×SFP |
| `USPPDUP` | Custom/other | Single-port fallback |
| `USMINI` | UniFi Switch Mini | 5-port row |
Add new layouts by adding a key to `SWITCH_LAYOUTS` matching the `model` field
returned by the UniFi API for that device.
### Suppressions (`/suppressions`)
- Create timed (30 min / 1 hr / 4 hr / 8 hr) or manual suppressions
- Target types: host, interface, UniFi device, or global
- Active suppressions table with one-click removal
- Suppression history (last 50)
- Available targets reference grid (all known hosts + interfaces)
---
## Alert Logic
### Ticket Triggers
| Condition | Priority |
|---|---|
| UniFi device offline (≥2 consecutive checks) | P2 High |
| Proxmox host NIC link-down regression (≥2 consecutive checks) | P2 High |
| Host unreachable via ping (≥2 consecutive checks) | P2 High |
| ≥3 hosts simultaneously reporting interface failures | P1 Critical |
### Baseline Tracking
Interfaces that are **down on first observation** (unused ports, unplugged cables)
are recorded as `initial_down` and never alerted. Only **UP→DOWN regressions**
generate tickets. Baseline is stored in MariaDB and survives daemon restarts.
### Suppression Targets
| Type | Suppresses |
|---|---|
| `host` | All interface alerts for a named host |
| `interface` | A specific NIC on a specific host |
| `unifi_device` | A specific UniFi device |
| `all` | Everything (global maintenance mode) |
Suppressions can be manual (persist until removed) or timed (auto-expire).
Expired suppressions are checked at evaluation time — no background cleanup needed.
---
## Configuration (`config.json`)
Shared by both processes. Located in the working directory (`/var/www/html/prod/`).
```json
{
"database": {
"host": "10.10.10.50",
"port": 3306,
"user": "gandalf",
"password": "...",
"name": "gandalf"
},
"prometheus": {
"url": "http://10.10.10.48:9090"
},
"unifi": {
"controller": "https://10.10.10.1",
"api_key": "...",
"site_id": "default"
},
"ticket_api": {
"url": "https://t.lotusguild.org/api/tickets",
"api_key": "..."
},
"pulse": {
"url": "http://<pulse-host>:<port>",
"api_key": "...",
"worker_id": "...",
"timeout": 45
},
"auth": {
"allowed_groups": ["admin"]
},
"hosts": [
{ "name": "large1", "prometheus_instance": "10.10.10.2:9100" },
{ "name": "compute-storage-01", "prometheus_instance": "10.10.10.4:9100" },
{ "name": "micro1", "prometheus_instance": "10.10.10.8:9100" },
{ "name": "monitor-02", "prometheus_instance": "10.10.10.9:9100" },
{ "name": "compute-storage-gpu-01", "prometheus_instance": "10.10.10.10:9100" },
{ "name": "storage-01", "prometheus_instance": "10.10.10.11:9100" }
],
"monitor": {
"poll_interval": 120,
"failure_threshold": 2,
"cluster_threshold": 3,
"ping_hosts": [
{ "name": "pbs", "ip": "10.10.10.3" }
]
}
}
```
### Key Config Fields
| Key | Description |
|---|---|
| `database.*` | MariaDB credentials (LXC 149 at 10.10.10.50) |
| `prometheus.url` | Prometheus base URL |
| `unifi.controller` | UniFi controller base URL (HTTPS, self-signed cert ignored) |
| `unifi.api_key` | UniFi API key from controller Settings → API |
| `unifi.site_id` | UniFi site ID (default: `default`) |
| `ticket_api.api_key` | Tinker Tickets bearer token |
| `pulse.url` | Pulse worker API base URL (for SSH relay) |
| `pulse.worker_id` | Which Pulse worker runs ethtool collection |
| `pulse.timeout` | Max seconds to wait for SSH collection per host |
| `auth.allowed_groups` | Authelia groups that may access Gandalf |
| `hosts` | Maps Prometheus instance labels → display hostnames |
| `monitor.poll_interval` | Seconds between full check cycles (default: 120) |
| `monitor.failure_threshold` | Consecutive failures before creating ticket (default: 2) |
| `monitor.cluster_threshold` | Hosts with failures to trigger cluster-wide P1 (default: 3) |
| `monitor.ping_hosts` | Hosts checked only by ping (no node_exporter) |
---
## Deployment (LXC 157)
### 1. Database — MariaDB LXC 149 (`10.10.10.50`)
```sql
CREATE DATABASE gandalf CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'gandalf'@'10.10.10.61' IDENTIFIED BY 'your_password';
GRANT ALL PRIVILEGES ON gandalf.* TO 'gandalf'@'10.10.10.61';
FLUSH PRIVILEGES;
```
Import schema:
```bash
mysql -h 10.10.10.50 -u gandalf -p gandalf < schema.sql
```
### 2. LXC 157 — Install dependencies
```bash
pip3 install -r requirements.txt
# Ensure sshpass is available (used by deploy scripts)
apt install sshpass
```
### 3. Deploy files
```bash
# From dev machine / root/code/gandalf:
for f in app.py db.py monitor.py config.json schema.sql \
static/style.css static/app.js \
templates/*.html; do
sshpass -p 'yourpass' scp -o StrictHostKeyChecking=no \
"$f" "root@10.10.10.61:/var/www/html/prod/$f"
done
systemctl restart gandalf gandalf-monitor
```
### 4. systemd services
**`gandalf.service`** (Flask/gunicorn web app):
```ini
[Unit]
Description=Gandalf Web Dashboard
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/www/html/prod
ExecStart=/usr/bin/python3 -m gunicorn --workers 1 --bind 127.0.0.1:8000 app:app
Restart=always
[Install]
WantedBy=multi-user.target
```
**`gandalf-monitor.service`** (background polling daemon):
```ini
[Unit]
Description=Gandalf Network Monitor Daemon
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/www/html/prod
ExecStart=/usr/bin/python3 monitor.py
Restart=always
[Install]
WantedBy=multi-user.target
```
### 5. Authelia rule (LXC 167)
```yaml
access_control:
rules:
- domain: gandalf.lotusguild.org
policy: one_factor
subject:
- group:admin
```
```bash
systemctl restart authelia
```
### 6. NPM reverse proxy
- **Domain:** `gandalf.lotusguild.org`
- **Forward to:** `http://10.10.10.61:8000` (gunicorn direct, no nginx needed on LXC)
- **Forward Auth:** Authelia at `http://10.10.10.167:9091`
- **WebSockets:** Not required
---
## Service Management
```bash
# Status
systemctl status gandalf gandalf-monitor
# Logs (live)
journalctl -u gandalf -f
journalctl -u gandalf-monitor -f
# Restart after code or config changes
systemctl restart gandalf gandalf-monitor
```
---
## Troubleshooting
### Monitor not creating tickets
- Verify `config.json → ticket_api.api_key` is set and valid
- Check `journalctl -u gandalf-monitor` for `Ticket creation failed` lines
- Confirm the Tinker Tickets API is reachable from LXC 157
### Link Debug shows no data / "Loading…" forever
- Check `gandalf-monitor.service` is running and has completed at least one cycle
- Check `journalctl -u gandalf-monitor` for Prometheus or UniFi errors
- Verify Prometheus is reachable: `curl http://10.10.10.48:9090/api/v1/query?query=up`
### Link Debug: SFP DOM panel missing
- SFP data requires Pulse worker + SSH access to hosts
- Verify `config.json → pulse.*` is configured and the Pulse worker is running
- Confirm `sshpass` + SSH access from the Pulse worker to each Proxmox host
- Only interfaces with physical SFP modules return DOM data (`ethtool -m`)
### Inspector: path debug section not appearing
- Requires LLDP: run `apt install lldpd && systemctl enable --now lldpd` on each server
- The LLDP `system_name` broadcast by `lldpd` must match the hostname in `config.json → hosts[].name`
- Override: `echo 'configure system hostname large1' > /etc/lldpd.d/hostname.conf && systemctl restart lldpd`
- Allow up to 2 poll cycles (240s) after installing lldpd for LLDP table to populate
### Inspector: switch chassis shows as flat list (no layout)
- The switch's `model` field from UniFi doesn't match any key in `SWITCH_LAYOUTS` in `inspector.html`
- Check the UniFi API: the model appears in the `link_stats` API response under `unifi_switches.<name>.model`
- Add the model key to `SWITCH_LAYOUTS` in `inspector.html` with the correct row/SFP layout
### Baseline re-initializing on every restart
- `interface_baseline` is stored in the `monitor_state` DB table; survives restarts
- If it appears to reset: check DB connectivity from the monitor daemon
### Interface stuck at "initial_down" forever
- This means the interface was down when the monitor first saw it
- It will begin tracking once it comes up; or manually clear it:
```sql
-- In MariaDB on 10.10.10.50:
UPDATE monitor_state SET value='{}' WHERE key_name='interface_baseline';
```
Then restart the monitor: `systemctl restart gandalf-monitor`
### Prometheus data missing for a host
```bash
# On the affected host:
systemctl status prometheus-node-exporter
# Verify it's scraped:
curl http://10.10.10.48:9090/api/v1/query?query=up | jq '.data.result[] | select(.metric.job=="node")'
```
---
## Development Notes
### File Layout
```
gandalf/
├── app.py # Flask web app (routes, auth, API endpoints)
├── monitor.py # Background daemon (Prometheus, UniFi, Pulse, alert logic)
├── db.py # Database operations (MariaDB via pymysql, thread-local conn reuse)
├── schema.sql # Database schema (network_events, suppression_rules, monitor_state)
├── config.json # Runtime configuration (not committed with secrets)
├── requirements.txt # Python dependencies
├── static/
│ ├── style.css # Terminal aesthetic CSS (CRT scanlines, green-on-black)
│ └── app.js # Dashboard JS (auto-refresh, host grid, events, suppress modal)
└── templates/
├── base.html # Shared layout (header, nav, footer)
├── index.html # Dashboard page
├── links.html # Link Debug page (server NICs + UniFi switch ports)
├── inspector.html # Visual switch inspector + LLDP path debug
└── suppressions.html # Suppression management page
```
### Adding a New Monitored Host
1. Install `prometheus-node-exporter` on the host
2. Add a scrape target to Prometheus config
3. Add an entry to `config.json → hosts`:
```json
{ "name": "newhost", "prometheus_instance": "10.10.10.X:9100" }
```
4. Restart monitor: `systemctl restart gandalf-monitor`
5. For SFP DOM / ethtool: ensure the host is SSH-accessible from the Pulse worker
### Adding a New Switch Layout (Inspector)
Find the UniFi model code for the switch (it appears in the `/api/links` JSON response
under `unifi_switches.<switch_name>.model`), then add to `SWITCH_LAYOUTS` in
`templates/inspector.html`:
```javascript
'MYNEWMODEL': {
rows: [[1,2,3,4,5,6,7,8], [9,10,11,12,13,14,15,16]], // port_idx by row
sfp_section: [17, 18], // separate SFP cage ports (rendered below rows)
sfp_ports: [], // port_idx values that are SFP-type within rows
},
```
### Database Schema Notes
- `network_events`: one row per active event; `resolved_at` is set when recovered
- `suppression_rules`: `active=FALSE` when removed; `expires_at` checked at query time
- `monitor_state`: key/value store; `interface_baseline` and `link_stats` are JSON blobs
### Security Notes
- **XSS prevention**: all user-controlled data in dynamically generated HTML uses
`escHtml()` (JS) or Jinja2 auto-escaping (Python). Suppress buttons use `data-*`
attributes + a single delegated click listener rather than inline `onclick` with
interpolated strings.
- **Interface name validation**: `monitor.py` validates SSH interface names against
`^[a-zA-Z0-9_.@-]+$` before use, and additionally wraps them with `shlex.quote()`
for defense-in-depth.
- **DB parameters**: all SQL uses parameterised queries via pymysql — no string
concatenation into SQL.
- **Auth**: Authelia enforces admin-only access at the nginx/LXC 167 layer; the Flask
app additionally checks the `Remote-User` header via `@require_auth`.
### Known Limitations
- Single gunicorn worker (`--workers 1`) — required because `db.py` uses thread-local
connection reuse (one connection per thread). Multiple workers would each have their
own connection, which is fine, but the thread-local optimisation only helps within
one worker.
- No CSRF tokens on API endpoints — mitigated by Authelia session cookies being
`SameSite=Strict` and the site being admin-only.
- SSH collection via Pulse is synchronous — if Pulse is slow, the entire monitor cycle
is delayed. The `pulse.timeout` config controls the max wait.
- UniFi LLDP data is only as fresh as the last monitor poll (120s default).
---
## CI / CD
| Workflow | Purpose | Triggers |
|---|---|---|
| `lint.yml` (python-lint) | flake8 on all `.py` files | Every push and PR |
| `lint.yml` (js-lint) | ESLint on `static/` | Every push and PR |
| `test.yml` | pytest — 33 tests for `diagnose.py` static methods | Every push and PR |
| `security.yml` | bandit `-ll` (medium+ severity) | Every push, PR, and weekly Monday 6am |
| `deploy` job in `lint.yml` | Calls the `gandalf-deploy` webhook on CT157 (10.10.10.61) | Push to `main` only, after both lint jobs pass |
Branch protection is enabled on `main` — both lint jobs must pass before any PR can merge.
Tests live in `tests/test_diagnose.py` and cover `DiagnosticsRunner` static methods:
`build_ssh_command`, `parse_output`, `parse_sysfs_stats`, `parse_ethtool`, and variants.
+691
View File
@@ -0,0 +1,691 @@
"""Gandalf – Global Advanced Network Detection And Link Facilitator.
Flask web application serving the monitoring dashboard and suppression
management UI. Authentication via Authelia forward-auth headers.
All monitoring and alerting is handled by the separate monitor.py daemon.
"""
import hashlib
import html
import ipaddress
import json
import logging
import os
import re
import tempfile
import threading
import time
import uuid
from datetime import datetime, timezone
from functools import wraps
from flask import Flask, jsonify, make_response, render_template, request, send_file
import db
import diagnose
from monitor import PulseClient
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s %(levelname)s %(name)s %(message)s',
)
logger = logging.getLogger('gandalf.web')
app = Flask(__name__)
_AVATAR_COLORS = ['lt-avatar--orange', 'lt-avatar--green', 'lt-avatar--purple', '']
@app.template_filter('avatar_color')
def avatar_color_filter(name: str) -> str:
return _AVATAR_COLORS[int(hashlib.md5(name.encode()).hexdigest(), 16) % len(_AVATAR_COLORS)] # nosec B324
_cfg = None
_cfg_lock = threading.Lock()
@app.context_processor
def inject_config():
"""Inject safe config values into all templates."""
cfg = _config()
return {
'config': {
'ticket_api': {
'web_url': cfg.get('ticket_api', {}).get('web_url', 'http://t.lotusguild.org/ticket/'),
}
}
}
# In-memory diagnostic job store { job_id: { status, result, created_at } }
_diag_jobs: dict = {}
_diag_lock = threading.Lock()
# Per-user rate-limit: { username: [epoch_float, ...] } — cleaned inside _diag_lock
_diag_rate: dict = {}
def _purge_old_jobs_loop():
"""Background thread: remove stale diagnostic jobs and mark stuck ones done."""
while True:
time.sleep(120)
cutoff = time.time() - 600
stuck_cutoff = time.time() - 300 # 5 min: job still 'running' → thread must have crashed
with _diag_lock:
stale = [jid for jid, j in _diag_jobs.items() if j.get('created_at', 0) < cutoff]
for jid in stale:
del _diag_jobs[jid]
for jid, j in list(_diag_jobs.items()):
if j['status'] == 'running' and j.get('created_at', 0) < stuck_cutoff:
j['status'] = 'done'
j['result'] = {'status': 'error', 'error': 'Diagnostic abandoned — no activity for 5 minutes.'}
logger.error(f'Diagnostic job {jid} stuck (no activity for 5 min); marked done/error')
_purge_thread = threading.Thread(target=_purge_old_jobs_loop, daemon=True)
_purge_thread.start()
def _config() -> dict:
global _cfg
if _cfg is None:
with _cfg_lock:
if _cfg is None:
with open('config.json') as f:
_cfg = json.load(f)
return _cfg
@app.after_request
def add_security_headers(response):
response.headers.setdefault('X-Content-Type-Options', 'nosniff')
response.headers.setdefault('X-Frame-Options', 'DENY')
response.headers.setdefault('Referrer-Policy', 'strict-origin-when-cross-origin')
return response
def _daemon_ok(last_check: str) -> bool:
"""Return True if monitor last checked within 20 minutes."""
if not last_check or last_check == 'Never':
return False
try:
ts = datetime.strptime(last_check, '%Y-%m-%d %H:%M:%S UTC').replace(tzinfo=timezone.utc)
return (datetime.now(timezone.utc) - ts).total_seconds() < 1200
except Exception:
return False
# ---------------------------------------------------------------------------
# Auth helpers
# ---------------------------------------------------------------------------
def _get_user() -> dict:
return {
'username': request.headers.get('Remote-User', ''),
'name': request.headers.get('Remote-Name', ''),
'email': request.headers.get('Remote-Email', ''),
'groups': [
g.strip()
for g in request.headers.get('Remote-Groups', '').split(',')
if g.strip()
],
}
def require_auth(f):
@wraps(f)
def wrapper(*args, **kwargs):
user = _get_user()
if not user['username']:
return (
'<h1>401 – Not authenticated</h1>'
'<p>Please access Gandalf through '
'<a href="https://auth.lotusguild.org">auth.lotusguild.org</a>.</p>',
401,
)
allowed = _config().get('auth', {}).get('allowed_groups', ['admin'])
if not any(g in allowed for g in user['groups']):
safe_user = html.escape(user['username'])
safe_groups = html.escape(', '.join(allowed))
return (
f'<h1>403 – Access denied</h1>'
f'<p>Your account ({safe_user}) is not in an allowed group '
f'({safe_groups}).</p>',
403,
)
return f(*args, **kwargs)
return wrapper
def require_admin(f):
"""Decorator: require require_auth AND membership in the 'admin' group."""
@wraps(f)
def wrapper(*args, **kwargs):
user = _get_user()
if 'admin' not in user.get('groups', []):
return jsonify({'error': 'Admin access required'}), 403
return f(*args, **kwargs)
return wrapper
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
_PAGE_LIMIT = 200 # max events returned per request
def _annotate_suppressions(events: list, suppressions: list) -> None:
"""Annotate each event dict in-place with an is_suppressed bool.
Mirrors the suppression check order in monitor.py exactly:
interface_down → interface OR host
unifi_device_* → unifi_device
everything else → host
"""
for ev in events:
etype = ev.get('event_type', '')
name = ev.get('target_name', '')
detail = ev.get('target_detail', '') or ''
if etype == 'interface_down':
ev['is_suppressed'] = (
db.check_suppressed(suppressions, 'interface', name, detail) or
db.check_suppressed(suppressions, 'host', name)
)
elif etype == 'unifi_device_offline':
ev['is_suppressed'] = db.check_suppressed(suppressions, 'unifi_device', name, detail)
else:
ev['is_suppressed'] = db.check_suppressed(suppressions, 'host', name, detail)
# ---------------------------------------------------------------------------
# Page routes
# ---------------------------------------------------------------------------
@app.route('/')
@require_auth
def index():
user = _get_user()
events = db.get_active_events(limit=_PAGE_LIMIT)
total_active = db.count_active_events()
summary = db.get_status_summary()
snapshot_raw = db.get_state('network_snapshot')
last_check = db.get_state('last_check', 'Never')
try:
snapshot = json.loads(snapshot_raw) if snapshot_raw else {}
except Exception as e:
logger.error(f'Failed to parse network_snapshot JSON: {e}')
snapshot = {}
suppressions = db.get_active_suppressions()
_annotate_suppressions(events, suppressions)
recent_resolved = db.get_recent_resolved(hours=24, limit=10)
return render_template(
'index.html',
user=user,
events=events,
total_active=total_active,
summary=summary,
snapshot=snapshot,
last_check=last_check,
suppressions=suppressions,
recent_resolved=recent_resolved,
daemon_ok=_daemon_ok(last_check),
)
@app.route('/links')
@require_auth
def links_page():
user = _get_user()
return render_template('links.html', user=user)
@app.route('/inspector')
@require_auth
def inspector():
user = _get_user()
return render_template('inspector.html', user=user)
@app.route('/suppressions')
@require_auth
@require_admin
def suppressions_page():
user = _get_user()
active = db.get_active_suppressions()
history = db.get_suppression_history(limit=50)
snapshot_raw = db.get_state('network_snapshot')
try:
snapshot = json.loads(snapshot_raw) if snapshot_raw else {}
except Exception as e:
logger.error(f'Failed to parse network_snapshot JSON: {e}')
snapshot = {}
return render_template(
'suppressions.html',
user=user,
active=active,
history=history,
snapshot=snapshot,
)
# ---------------------------------------------------------------------------
# API routes
# ---------------------------------------------------------------------------
@app.route('/api/status')
@require_auth
def api_status():
active = db.get_active_events(limit=_PAGE_LIMIT)
suppressions = db.get_active_suppressions()
_annotate_suppressions(active, suppressions)
last_check = db.get_state('last_check', 'Never')
return jsonify({
'summary': db.get_status_summary(),
'last_check': last_check,
'events': active,
'total_active': db.count_active_events(),
'daemon_ok': _daemon_ok(last_check),
})
@app.route('/api/network')
@require_auth
def api_network():
raw = db.get_state('network_snapshot')
if raw:
try:
return jsonify(json.loads(raw))
except Exception:
logger.error('Failed to parse network_snapshot JSON')
return jsonify({'hosts': {}, 'unifi': [], 'updated': None})
@app.route('/api/links')
@require_auth
def api_links():
raw = db.get_state('link_stats')
if raw:
if len(raw) > 10_000_000:
logger.error(f'link_stats exceeds 10 MB ({len(raw)} bytes); possible corruption')
return jsonify({'error': 'Invalid cached data'}), 503
try:
return jsonify(json.loads(raw))
except Exception as e:
logger.error(f'Failed to parse link_stats JSON: {e}')
return jsonify({'hosts': {}, 'unifi_switches': {}, 'updated': None})
@app.route('/api/events')
@require_auth
def api_events():
try:
limit = min(int(request.args.get('limit', _PAGE_LIMIT)), 1000)
offset = max(int(request.args.get('offset', 0)), 0)
except ValueError:
return jsonify({'error': 'limit and offset must be integers'}), 400
status_filter = request.args.get('status', 'active')
if status_filter not in ('active', 'resolved', 'all'):
return jsonify({'error': 'status must be active, resolved, or all'}), 400
result: dict = {}
if status_filter in ('active', 'all'):
result['active'] = db.get_active_events(limit=limit, offset=offset)
result['total_active'] = db.count_active_events()
if status_filter in ('resolved', 'all'):
result['resolved'] = db.get_recent_resolved(hours=24, limit=30)
return jsonify(result)
@app.route('/api/suppressions', methods=['GET'])
@require_auth
def api_get_suppressions():
return jsonify(db.get_active_suppressions())
@app.route('/api/suppressions', methods=['POST'])
@require_auth
@require_admin
def api_create_suppression():
user = _get_user()
data = request.get_json(silent=True) or {}
target_type = data.get('target_type', 'host')
target_name = (data.get('target_name') or '').strip()
target_detail = (data.get('target_detail') or '').strip()
reason = (data.get('reason') or '').strip()
expires_minutes = data.get('expires_minutes') # None = manual/permanent
if target_type not in ('host', 'interface', 'unifi_device', 'all'):
return jsonify({'error': 'Invalid target_type'}), 400
if target_type != 'all' and not target_name:
return jsonify({'error': 'target_name required'}), 400
if not reason:
return jsonify({'error': 'reason required'}), 400
if len(reason) > 500:
return jsonify({'error': 'reason must be 500 characters or fewer'}), 400
if len(target_name) > 255:
return jsonify({'error': 'target_name must be 255 characters or fewer'}), 400
if len(target_detail) > 255:
return jsonify({'error': 'target_detail must be 255 characters or fewer'}), 400
if expires_minutes is not None:
try:
expires_minutes = int(expires_minutes)
if expires_minutes <= 0 or expires_minutes > 43200:
return jsonify({'error': 'expires_minutes must be between 1 and 43200 (30 days)'}), 400
except (ValueError, TypeError):
return jsonify({'error': 'expires_minutes must be a valid integer'}), 400
sup_id = db.create_suppression(
target_type=target_type,
target_name=target_name,
target_detail=target_detail,
reason=reason,
suppressed_by=user['username'],
expires_minutes=expires_minutes,
)
logger.info(
f'Suppression #{sup_id} created by {user["username"]}: '
f'{target_type}/{target_name}/{target_detail} – {reason}'
)
return jsonify({'success': True, 'id': sup_id})
@app.route('/api/suppressions/<int:sup_id>', methods=['DELETE'])
@require_auth
@require_admin
def api_delete_suppression(sup_id: int):
user = _get_user()
db.deactivate_suppression(sup_id)
logger.info(f'Suppression #{sup_id} removed by {user["username"]}')
return jsonify({'success': True})
@app.route('/api/diagnose', methods=['POST'])
@require_auth
def api_diagnose_start():
"""Start a link diagnostic job. Returns {job_id}."""
data = request.get_json(silent=True) or {}
switch_name = (data.get('switch_name') or '').strip()
try:
port_idx = int(data.get('port_idx'))
except (TypeError, ValueError):
return jsonify({'error': 'port_idx must be an integer'}), 400
if not switch_name:
return jsonify({'error': 'switch_name and port_idx required'}), 400
# Look up switch + port in cached link_stats
raw = db.get_state('link_stats')
if not raw:
return jsonify({'error': 'No link_stats data available'}), 503
try:
link_data = json.loads(raw)
except Exception as e:
logger.error(f'Failed to parse link_stats JSON in /api/diagnose: {e}')
return jsonify({'error': 'Internal data error'}), 500
switches = link_data.get('unifi_switches', {})
sw = switches.get(switch_name)
if not sw:
return jsonify({'error': f'Switch "{switch_name}" not found'}), 404
# Find port by port_idx
port_data = None
for pname, pd in sw.get('ports', {}).items():
if pd.get('port_idx') == port_idx:
port_data = dict(pd)
port_data['name'] = pname
break
if not port_data:
return jsonify({'error': f'Port {port_idx} not found on switch "{switch_name}"'}), 404
# LLDP neighbor required to know which host+iface to SSH into
lldp = port_data.get('lldp')
if not lldp or not lldp.get('system_name'):
return jsonify({'error': 'No LLDP neighbor data for this port'}), 400
server_name = lldp['system_name']
if not re.fullmatch(r'[a-zA-Z0-9._-]+', server_name):
logger.error(f'Refusing diagnostic: invalid server_name from LLDP: {server_name!r}')
return jsonify({'error': 'LLDP neighbor name contains invalid characters'}), 400
lldp_port_id = lldp.get('port_id', '')
# Find matching host + interface in link_stats hosts
hosts = link_data.get('hosts', {})
server_ifaces = hosts.get(server_name)
if not server_ifaces:
return jsonify({'error': f'Host "{server_name}" not in link stats'}), 404
# Match interface by LLDP port_id (exact then fuzzy)
matched_iface = None
if lldp_port_id and lldp_port_id in server_ifaces:
matched_iface = lldp_port_id
if not matched_iface and lldp_port_id:
matched_iface = next(
(k for k in server_ifaces if lldp_port_id in k or k in lldp_port_id),
None
)
if not matched_iface:
matched_iface = next(iter(server_ifaces), None)
if not matched_iface:
return jsonify({'error': 'Cannot determine server interface'}), 400
# Resolve host IP from link_stats host data
host_ip = (server_ifaces.get(matched_iface) or {}).get('host_ip')
if not host_ip:
# Fallback: use first valid IP from LLDP mgmt IPs
for candidate in (lldp.get('mgmt_ips') or []):
try:
ipaddress.ip_address(candidate)
host_ip = candidate
break
except ValueError:
continue
if not host_ip:
return jsonify({'error': 'Cannot determine host IP for SSH'}), 400
# Validate resolved values before passing to SSH command builder
try:
ipaddress.ip_address(host_ip)
except ValueError:
logger.error(f'Refusing diagnostic: invalid host_ip "{host_ip}" for {server_name}')
return jsonify({'error': 'Resolved host IP is not a valid IP address'}), 400
if not re.fullmatch(r'[a-zA-Z0-9._-]+', matched_iface):
logger.error(f'Refusing diagnostic: invalid iface "{matched_iface}" for {server_name}')
return jsonify({'error': 'Resolved interface name contains invalid characters'}), 400
job_id = str(uuid.uuid4())
requesting_user = _get_user()['username']
now = time.time()
with _diag_lock:
# Rate limit: max 5 diagnostic jobs per user per minute; prune stale user entries
stale_users = [u for u, ts in _diag_rate.items() if not ts or max(ts) < now - 3600]
for u in stale_users:
del _diag_rate[u]
recent = [t for t in _diag_rate.get(requesting_user, []) if now - t < 60]
if len(recent) >= 5:
return jsonify({'error': 'Rate limit exceeded: max 5 diagnostics per minute'}), 429
recent.append(now)
_diag_rate[requesting_user] = recent
_diag_jobs[job_id] = {
'status': 'running', 'result': None,
'created_at': now, 'user': requesting_user,
}
def _run():
try:
cfg = _config()
pulse = PulseClient(cfg)
runner = diagnose.DiagnosticsRunner(pulse)
result = runner.run(host_ip, server_name, matched_iface, port_data)
except Exception as e:
logger.error(f'Diagnostic job {job_id} failed: {e}', exc_info=True)
result = {'status': 'error', 'error': 'Diagnostic failed; check server logs.'}
with _diag_lock:
if job_id in _diag_jobs:
_diag_jobs[job_id]['status'] = 'done'
_diag_jobs[job_id]['result'] = result
t = threading.Thread(target=_run, daemon=True)
t.start()
return jsonify({'job_id': job_id})
@app.route('/api/diagnose/<job_id>', methods=['GET'])
@require_auth
def api_diagnose_poll(job_id: str):
"""Poll a diagnostic job. Returns {status, result}."""
current_user = _get_user()['username']
with _diag_lock:
job = _diag_jobs.get(job_id)
if not job:
return jsonify({'error': 'Job not found'}), 404
if job.get('user') != current_user:
return jsonify({'error': 'Forbidden'}), 403
snapshot = {'status': job['status'], 'result': job.get('result')}
return jsonify(snapshot)
@app.route('/api/avatar')
@require_auth
def api_avatar():
"""Serve the current user's LDAP avatar photo (JPEG), cached to disk."""
username = request.headers.get('Remote-User', '').strip()
if not username:
return '', 404
ldap_cfg = _config().get('ldap', {})
if not ldap_cfg.get('host') or not ldap_cfg.get('bind_dn'):
return '', 404
# Build a safe cache filename from the username (alphanumeric + - _ .)
safe_name = re.sub(r'[^a-zA-Z0-9._-]', '_', username)
cache_dir = os.path.abspath(
ldap_cfg.get('cache_dir', os.path.join(tempfile.gettempdir(), 'gandalf_avatars'))
)
os.makedirs(cache_dir, exist_ok=True)
cache_file = os.path.abspath(os.path.join(cache_dir, f'user_{safe_name}.jpg'))
sentinel = os.path.abspath(os.path.join(cache_dir, f'user_{safe_name}.none'))
# Guard against path escape (shouldn't happen with sanitised safe_name, but be explicit)
if not cache_file.startswith(cache_dir + os.sep) or not sentinel.startswith(cache_dir + os.sep):
logger.error(f'Avatar path escape detected for user {username!r}')
return '', 404
try:
cache_ttl = int(ldap_cfg.get('cache_ttl', 3600))
except (ValueError, TypeError):
logger.warning('Invalid cache_ttl in ldap config; using default 3600')
cache_ttl = 3600
now = time.time()
# Serve cached image if fresh
if os.path.exists(cache_file) and now - os.path.getmtime(cache_file) < cache_ttl:
return send_file(cache_file, mimetype='image/jpeg',
max_age=cache_ttl, conditional=True)
# Skip LDAP if we already know this user has no avatar
try:
if os.path.exists(sentinel) and now - os.path.getmtime(sentinel) < cache_ttl:
return '', 404
except OSError:
pass
# Query lldap
bind_pw = ldap_cfg.get('bind_pw', '')
if not bind_pw:
logger.error('LDAP bind_pw not configured — avatar lookup disabled')
return '', 404
avatar_data = None
conn = None
try:
import ldap3
server = ldap3.Server(ldap_cfg['host'], port=int(ldap_cfg.get('port', 3890)))
conn = ldap3.Connection(server,
user=ldap_cfg['bind_dn'],
password=bind_pw,
auto_bind=True, receive_timeout=5)
safe_uid = ldap3.utils.conv.escape_filter_chars(username)
conn.search(ldap_cfg.get('user_base', 'ou=people,dc=example,dc=com'),
f'(uid={safe_uid})', attributes=['avatar'])
if conn.entries and conn.entries[0]['avatar'].value:
avatar_data = conn.entries[0]['avatar'].value
except ImportError:
logger.error('ldap3 not installed — run: pip install ldap3')
return '', 404
except Exception as e:
logger.error(f'LDAP avatar lookup failed for {username}: {e}')
return '', 404
finally:
if conn is not None:
try:
conn.unbind()
except Exception:
pass
if not avatar_data or len(avatar_data) < 100:
with open(sentinel, 'w'):
pass
return '', 404
# Validate JPEG magic bytes (FF D8 FF)
if isinstance(avatar_data, str):
avatar_data = avatar_data.encode('latin-1')
if avatar_data[:3] != b'\xFF\xD8\xFF':
logger.warning(f'Non-JPEG avatar data for {username}')
with open(sentinel, 'w'):
pass
return '', 404
with open(cache_file, 'wb') as f:
f.write(avatar_data)
if os.path.exists(sentinel):
os.unlink(sentinel)
resp = make_response(avatar_data)
resp.headers['Content-Type'] = 'image/jpeg'
resp.headers['Cache-Control'] = f'private, max-age={cache_ttl}'
return resp
@app.route('/health')
def health():
"""Health check endpoint (no auth). Checks DB and monitor freshness."""
checks = {}
overall = 'ok'
# DB connectivity
try:
db.get_state('last_check')
checks['db'] = 'ok'
except Exception as e:
logger.error(f'Health check db error: {e}')
checks['db'] = 'error'
overall = 'degraded'
# Monitor freshness: fail if last_check is older than 20 minutes
try:
last_check = db.get_state('last_check', '')
if last_check:
ts = datetime.strptime(last_check, '%Y-%m-%d %H:%M:%S UTC').replace(tzinfo=timezone.utc)
age_s = (datetime.now(timezone.utc) - ts).total_seconds()
if age_s > 1200:
checks['monitor'] = 'stale'
overall = 'degraded'
else:
checks['monitor'] = 'ok'
else:
checks['monitor'] = 'no data yet'
except Exception as e:
logger.error(f'Health check monitor error: {e}')
checks['monitor'] = 'error'
overall = 'degraded'
status_code = 200 if overall == 'ok' else 503
return jsonify({'status': overall, 'service': 'gandalf', 'checks': checks}), status_code
if __name__ == '__main__':
app.run(debug=True, host='0.0.0.0', port=5000) # nosec B201 B104 — dev runner only; production uses gunicorn
+385
View File
@@ -0,0 +1,385 @@
"""Database operations for Gandalf network monitor."""
import json
import logging
import threading
from contextlib import contextmanager
from datetime import datetime, timedelta, timezone
from typing import Optional
import pymysql
import pymysql.cursors
logger = logging.getLogger(__name__)
_config_cache = None
_local = threading.local()
def _config() -> dict:
global _config_cache
if _config_cache is None:
with open('config.json') as f:
_config_cache = json.load(f)['database']
return _config_cache
def _new_conn(cfg: dict):
return pymysql.connect(
host=cfg['host'],
port=cfg.get('port', 3306),
user=cfg['user'],
password=cfg['password'],
database=cfg['name'],
autocommit=True,
cursorclass=pymysql.cursors.DictCursor,
connect_timeout=10,
charset='utf8mb4',
)
@contextmanager
def get_conn():
"""Yield a per-thread cached database connection, reconnecting as needed."""
cfg = _config()
conn = getattr(_local, 'conn', None)
if conn is None:
conn = _new_conn(cfg)
_local.conn = conn
else:
try:
conn.ping(reconnect=True)
except Exception:
try:
conn.close()
except Exception:
pass
_local.conn = None
conn = _new_conn(cfg)
_local.conn = conn
yield conn
# ---------------------------------------------------------------------------
# Monitor state (key/value store)
# ---------------------------------------------------------------------------
def set_state(key: str, value) -> None:
if not isinstance(value, str):
value = json.dumps(value, default=str)
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""INSERT INTO monitor_state (key_name, value)
VALUES (%s, %s)
ON DUPLICATE KEY UPDATE value=VALUES(value), updated_at=NOW()""",
(key, value),
)
def get_state(key: str, default=None):
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute('SELECT value FROM monitor_state WHERE key_name=%s', (key,))
row = cur.fetchone()
return row['value'] if row else default
# ---------------------------------------------------------------------------
# Interface baseline tracking
# ---------------------------------------------------------------------------
def get_baseline() -> dict:
raw = get_state('interface_baseline')
if raw:
try:
return json.loads(raw)
except Exception:
logger.error('Failed to parse interface_baseline JSON; resetting baseline')
return {}
def set_baseline(baseline: dict) -> None:
set_state('interface_baseline', json.dumps(baseline))
# ---------------------------------------------------------------------------
# Network events
# ---------------------------------------------------------------------------
def upsert_event(
event_type: str,
severity: str,
source_type: str,
target_name: str,
target_detail: str,
description: str,
) -> tuple:
"""Insert or update a network event. Returns (id, is_new, consecutive_failures)."""
detail = target_detail or ''
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""SELECT id, consecutive_failures FROM network_events
WHERE event_type=%s AND target_name=%s AND target_detail=%s
AND resolved_at IS NULL LIMIT 1""",
(event_type, target_name, detail),
)
existing = cur.fetchone()
if existing:
new_count = existing['consecutive_failures'] + 1
cur.execute(
"""UPDATE network_events
SET last_seen=NOW(), consecutive_failures=%s, description=%s
WHERE id=%s""",
(new_count, description, existing['id']),
)
return existing['id'], False, new_count
else:
cur.execute(
"""INSERT INTO network_events
(event_type, severity, source_type, target_name, target_detail, description)
VALUES (%s, %s, %s, %s, %s, %s)""",
(event_type, severity, source_type, target_name, detail, description),
)
return cur.lastrowid, True, 1
def resolve_event(event_type: str, target_name: str, target_detail: str = '') -> None:
detail = target_detail or ''
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""UPDATE network_events SET resolved_at=NOW()
WHERE event_type=%s AND target_name=%s AND target_detail=%s
AND resolved_at IS NULL""",
(event_type, target_name, detail),
)
def set_ticket_id(event_id: int, ticket_id: str) -> None:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
'UPDATE network_events SET ticket_id=%s WHERE id=%s',
(ticket_id, event_id),
)
def get_active_events(limit: int = 200, offset: int = 0) -> list:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""SELECT * FROM network_events
WHERE resolved_at IS NULL
ORDER BY
FIELD(severity,'critical','warning','info'),
first_seen DESC
LIMIT %s OFFSET %s""",
(limit, offset),
)
rows = cur.fetchall()
for r in rows:
for k in ('first_seen', 'last_seen'):
if r.get(k) and hasattr(r[k], 'isoformat'):
r[k] = r[k].isoformat() + 'Z'
return rows
def count_active_events() -> int:
"""Return count of all unresolved events (for pagination)."""
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"SELECT COUNT(*) AS n FROM network_events WHERE resolved_at IS NULL"
)
return cur.fetchone()['n']
def get_recent_resolved(hours: int = 24, limit: int = 50) -> list:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""SELECT * FROM network_events
WHERE resolved_at IS NOT NULL
AND resolved_at > DATE_SUB(NOW(), INTERVAL %s HOUR)
ORDER BY resolved_at DESC LIMIT %s""",
(hours, limit),
)
rows = cur.fetchall()
for r in rows:
for k in ('first_seen', 'last_seen', 'resolved_at'):
if r.get(k) and hasattr(r[k], 'isoformat'):
r[k] = r[k].isoformat() + 'Z'
return rows
def get_status_summary() -> dict:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""SELECT severity, COUNT(*) as cnt FROM network_events
WHERE resolved_at IS NULL GROUP BY severity"""
)
counts = {r['severity']: r['cnt'] for r in cur.fetchall()}
cur.execute(
"""SELECT COUNT(*) as cnt FROM network_events
WHERE resolved_at IS NOT NULL
AND resolved_at > DATE_SUB(NOW(), INTERVAL 24 HOUR)"""
)
resolved_24h = cur.fetchone()['cnt']
return {
'critical': counts.get('critical', 0),
'warning': counts.get('warning', 0),
'info': counts.get('info', 0),
'resolved_24h': resolved_24h,
}
# ---------------------------------------------------------------------------
# Suppression rules
# ---------------------------------------------------------------------------
def get_active_suppressions() -> list:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""SELECT * FROM suppression_rules
WHERE active=TRUE AND (expires_at IS NULL OR expires_at > NOW())
ORDER BY created_at DESC"""
)
rows = cur.fetchall()
for r in rows:
for k in ('created_at', 'expires_at'):
if r.get(k) and hasattr(r[k], 'isoformat'):
r[k] = r[k].isoformat() + 'Z'
return rows
def get_suppression_history(limit: int = 50) -> list:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
'SELECT * FROM suppression_rules ORDER BY created_at DESC LIMIT %s',
(limit,),
)
rows = cur.fetchall()
for r in rows:
for k in ('created_at', 'expires_at'):
if r.get(k) and hasattr(r[k], 'isoformat'):
r[k] = r[k].isoformat() + 'Z'
return rows
def create_suppression(
target_type: str,
target_name: str,
target_detail: str,
reason: str,
suppressed_by: str,
expires_minutes: Optional[int] = None,
) -> int:
expires_at = None
if expires_minutes:
expires_at = datetime.now(timezone.utc) + timedelta(minutes=int(expires_minutes))
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""INSERT INTO suppression_rules
(target_type, target_name, target_detail, reason, suppressed_by, expires_at, active)
VALUES (%s, %s, %s, %s, %s, %s, TRUE)""",
(target_type, target_name or '', target_detail or '', reason, suppressed_by, expires_at),
)
return cur.lastrowid
def deactivate_suppression(sup_id: int) -> None:
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
'UPDATE suppression_rules SET active=FALSE WHERE id=%s', (sup_id,)
)
def cleanup_expired_suppressions() -> int:
"""Mark expired time-limited suppressions as inactive. Returns count deactivated."""
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""UPDATE suppression_rules
SET active=FALSE
WHERE active=TRUE AND expires_at IS NOT NULL AND expires_at <= NOW()"""
)
n = cur.rowcount
if n:
logger.info(f'Deactivated {n} expired suppression(s)')
return n
def purge_old_resolved_events(days: int = 90) -> int:
"""Delete resolved events older than `days` days. Returns count deleted."""
with get_conn() as conn:
with conn.cursor() as cur:
cur.execute(
"""DELETE FROM network_events
WHERE resolved_at IS NOT NULL
AND resolved_at < DATE_SUB(NOW(), INTERVAL %s DAY)""",
(days,),
)
n = cur.rowcount
if n:
logger.info(f'Purged {n} old resolved event(s) (>{days}d)')
return n
def check_suppressed(suppressions: list, target_type: str, target_name: str, target_detail: str = '') -> bool:
"""Check suppression against a pre-loaded list (avoids per-call DB queries)."""
for s in suppressions:
if s['target_type'] == 'all':
return True
if s['target_type'] == target_type and s['target_name'] == target_name:
if not (s.get('target_detail') or ''):
return True
if target_detail and s.get('target_detail') == target_detail:
return True
return False
def is_suppressed(target_type: str, target_name: str, target_detail: str = '') -> bool:
with get_conn() as conn:
with conn.cursor() as cur:
# Global suppression (all)
cur.execute(
"""SELECT id FROM suppression_rules
WHERE active=TRUE AND (expires_at IS NULL OR expires_at > NOW())
AND target_type='all' LIMIT 1"""
)
if cur.fetchone():
return True
if not target_name:
return False
# Host-level suppression (covers all interfaces on that host)
cur.execute(
"""SELECT id FROM suppression_rules
WHERE active=TRUE AND (expires_at IS NULL OR expires_at > NOW())
AND target_type=%s AND target_name=%s
AND target_detail='' LIMIT 1""",
(target_type, target_name),
)
if cur.fetchone():
return True
# Interface/device-specific suppression
if target_detail:
cur.execute(
"""SELECT id FROM suppression_rules
WHERE active=TRUE AND (expires_at IS NULL OR expires_at > NOW())
AND target_type=%s AND target_name=%s AND target_detail=%s LIMIT 1""",
(target_type, target_name, target_detail),
)
if cur.fetchone():
return True
return False
+546
View File
@@ -0,0 +1,546 @@
"""Gandalf – Link Diagnostics module.
Runs a comprehensive SSH-based diagnostic against a server NIC and
analyses the result against switch port data to surface root causes.
Executed in a background thread; results stored in _diag_jobs (app.py).
"""
import re
import shlex
import logging
from typing import Dict, List, Optional
logger = logging.getLogger('gandalf.diagnose')
# sysfs counters collected per interface
_SYSFS_STATS = [
'rx_bytes', 'tx_bytes', 'rx_errors', 'tx_errors',
'rx_dropped', 'tx_dropped', 'rx_crc_errors',
'rx_frame_errors', 'rx_fifo_errors', 'tx_carrier_errors',
'collisions', 'rx_missed_errors',
]
class DiagnosticsRunner:
"""Build and run a link diagnostic against a server NIC via PulseClient."""
def __init__(self, pulse_client):
self.pulse = pulse_client
# ------------------------------------------------------------------
# SSH command builder
# ------------------------------------------------------------------
@staticmethod
def build_ssh_command(host_ip: str, iface: str) -> str:
"""Return a single-line SSH command that collects all diagnostic data."""
q = shlex.quote(iface)
ip_q = shlex.quote(host_ip)
sysfs_loop = '; '.join(
f'echo "{s}:$(cat /sys/class/net/{q}/statistics/{s} 2>/dev/null || echo 0)"'
for s in _SYSFS_STATS
)
remote_cmd = (
f'echo "=== carrier ===";'
f' cat /sys/class/net/{q}/carrier 2>/dev/null || echo "?";'
f' echo "=== operstate ===";'
f' cat /sys/class/net/{q}/operstate 2>/dev/null || echo "?";'
f' echo "=== sysfs_stats ===";'
f' {sysfs_loop};'
f' echo "=== carrier_changes ===";'
f' cat /sys/class/net/{q}/carrier_changes 2>/dev/null || echo "0";'
f' echo "=== ethtool ===";'
f' ethtool {q} 2>/dev/null;'
f' echo "=== ethtool_driver ===";'
f' ethtool -i {q} 2>/dev/null;'
f' echo "=== ethtool_pause ===";'
f' ethtool -a {q} 2>/dev/null;'
f' echo "=== ethtool_ring ===";'
f' ethtool -g {q} 2>/dev/null;'
f' echo "=== ethtool_stats ===";'
f' ethtool -S {q} 2>/dev/null;'
f' echo "=== ethtool_dom ===";'
f' ethtool -m {q} 2>/dev/null;'
f' echo "=== ip_link ===";'
f' ip -s link show {q} 2>/dev/null;'
f' echo "=== ip_addr ===";'
f' ip addr show {q} 2>/dev/null;'
f' echo "=== ip_route ===";'
f' ip route show dev {q} 2>/dev/null;'
f' echo "=== dmesg ===";'
f' dmesg 2>/dev/null | grep -F -- {q} | tail -50;'
f' echo "=== lldpctl ===";'
f' lldpctl 2>/dev/null || echo "lldpd not running";'
f' echo "=== end ==="'
)
return (
f'ssh -o StrictHostKeyChecking=accept-new -o ConnectTimeout=5 '
f'-o BatchMode=yes -o LogLevel=ERROR '
f'-o ServerAliveInterval=10 -o ServerAliveCountMax=2 '
f'root@{ip_q} {shlex.quote(remote_cmd)}'
)
# ------------------------------------------------------------------
# Main entry point
# ------------------------------------------------------------------
def run(self, host_ip: str, host_name: str, iface: str,
switch_port_data: dict) -> dict:
"""Execute diagnostics and return structured result dict."""
cmd = self.build_ssh_command(host_ip, iface)
logger.info(f'Running link diagnostic: {host_name}/{iface} via Pulse')
# Reset execution_id before call
self.pulse.last_execution_id = None
output = self.pulse.run_command(cmd)
execution_id = getattr(self.pulse, 'last_execution_id', None)
if output is None:
return {
'status': 'error',
'error': 'Pulse command failed or timed out',
'host': host_name,
'iface': iface,
'pulse_execution_id': execution_id,
}
sections = self.parse_output(output)
health = self.analyze(sections, switch_port_data)
pulse_url = None
if execution_id:
base = getattr(self.pulse, 'url', '').rstrip('/')
pulse_url = f'{base}/executions/{execution_id}' if base else None
return {
'status': 'done',
'host': host_name,
'iface': iface,
'sections': sections,
'health': health,
'pulse_execution_id': execution_id,
'pulse_url': pulse_url,
'switch_port': switch_port_data,
}
# ------------------------------------------------------------------
# Output parser (splits on === SECTION_NAME === sentinels)
# ------------------------------------------------------------------
@staticmethod
def parse_output(raw: str) -> dict:
sections: Dict[str, str] = {}
current: Optional[str] = None
buf: List[str] = []
for line in raw.splitlines():
m = re.match(r'^=== (.+?) ===$', line.strip())
if m:
if current and current != 'end':
sections[current] = '\n'.join(buf).strip()
name = m.group(1)
if name == 'end':
current = None
else:
current = name
buf = []
elif current:
buf.append(line)
if current and current != 'end':
sections[current] = '\n'.join(buf).strip()
parsed: dict = {}
# Simple string sections
parsed['carrier'] = sections.get('carrier', '?').strip()
parsed['operstate'] = sections.get('operstate', '?').strip()
# carrier_changes
cc_raw = sections.get('carrier_changes', '0').strip()
try:
parsed['carrier_changes'] = int(cc_raw)
except ValueError:
parsed['carrier_changes'] = None
# Structured sections
parsed['sysfs_stats'] = DiagnosticsRunner.parse_sysfs_stats(sections.get('sysfs_stats', ''))
parsed['ethtool'] = DiagnosticsRunner.parse_ethtool(sections.get('ethtool', ''))
parsed['ethtool_driver'] = DiagnosticsRunner.parse_ethtool_driver(sections.get('ethtool_driver', ''))
parsed['ethtool_pause'] = DiagnosticsRunner.parse_ethtool_pause(sections.get('ethtool_pause', ''))
parsed['ethtool_ring'] = DiagnosticsRunner.parse_ethtool_ring(sections.get('ethtool_ring', ''))
parsed['ethtool_stats'] = DiagnosticsRunner.parse_nic_stats(sections.get('ethtool_stats', ''))
parsed['ethtool_dom'] = DiagnosticsRunner.parse_ethtool_dom(sections.get('ethtool_dom', ''))
parsed['ip_link'] = DiagnosticsRunner.parse_ip_link(sections.get('ip_link', ''))
parsed['ip_addr'] = sections.get('ip_addr', '').strip()
parsed['ip_route'] = sections.get('ip_route', '').strip()
parsed['dmesg'] = DiagnosticsRunner.parse_dmesg(sections.get('dmesg', ''))
parsed['lldpctl'] = DiagnosticsRunner.parse_lldpctl(sections.get('lldpctl', ''))
return parsed
# ------------------------------------------------------------------
# Individual parsers
# ------------------------------------------------------------------
@staticmethod
def parse_sysfs_stats(text: str) -> dict:
result: dict = {}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if key in _SYSFS_STATS:
try:
result[key] = int(val)
except ValueError:
result[key] = 0
return result
@staticmethod
def parse_ethtool(text: str) -> dict:
"""Parse ethtool <iface> output."""
data: dict = {}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if key == 'Speed':
m = re.match(r'(\d+)\s*Mb/s', val)
if m:
data['speed_mbps'] = int(m.group(1))
elif 'Unknown' in val or 'unknown' in val:
data['speed_mbps'] = None
elif key == 'Duplex':
data['duplex'] = val.lower()
elif key == 'Port':
data['port_type'] = val
elif key == 'Auto-negotiation':
data['auto_neg'] = (val.lower() == 'on')
elif key == 'Link detected':
data['link_detected'] = (val.lower() == 'yes')
return data
@staticmethod
def parse_ethtool_driver(text: str) -> dict:
data: dict = {}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if key == 'driver':
data['driver'] = val
elif key == 'version':
data['version'] = val
elif key == 'firmware-version':
data['firmware_version'] = val
elif key == 'bus-info':
data['bus_info'] = val
return data
@staticmethod
def parse_ethtool_pause(text: str) -> dict:
data = {'rx_pause': False, 'tx_pause': False}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip().lower()
if key == 'RX':
data['rx_pause'] = (val == 'on')
elif key == 'TX':
data['tx_pause'] = (val == 'on')
return data
@staticmethod
def parse_ethtool_ring(text: str) -> dict:
data: dict = {}
in_current = False
for line in text.splitlines():
if 'Current hardware settings' in line:
in_current = True
continue
if 'Pre-set maximums' in line:
in_current = False
continue
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
try:
v = int(val)
except ValueError:
continue
if in_current:
if 'RX' in key and 'rx_current' not in data:
data['rx_current'] = v
elif 'TX' in key and 'tx_current' not in data:
data['tx_current'] = v
else:
if 'RX' in key and 'rx_max' not in data:
data['rx_max'] = v
elif 'TX' in key and 'tx_max' not in data:
data['tx_max'] = v
return data
@staticmethod
def parse_nic_stats(text: str) -> dict:
"""Parse ethtool -S output into {key: int} dict."""
data: dict = {}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
try:
data[key] = int(val)
except ValueError:
pass
return data
@staticmethod
def parse_ethtool_dom(text: str) -> dict:
"""Parse ethtool -m (SFP DOM) output."""
if not text:
return {}
lower = text.lower()
if any(s in lower for s in ('cannot get', 'not supported', 'no sfp', 'operation not supported')):
return {}
data: dict = {}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if key == 'Vendor name':
data['vendor'] = val
elif key == 'Vendor PN':
data['part_no'] = val
elif key == 'Identifier':
m = re.search(r'\((.+?)\)', val)
if m:
data['sfp_type'] = m.group(1)
elif key == 'Connector':
m = re.search(r'\((.+?)\)', val)
if m:
data['connector'] = m.group(1)
elif key == 'Laser wavelength':
m = re.match(r'(\d+)', val)
if m:
data['wavelength_nm'] = int(m.group(1))
elif key == 'Laser bias current':
m = re.match(r'([\d.]+)\s+mA', val)
if m:
data['bias_ma'] = float(m.group(1))
elif key == 'Laser output power':
m = re.search(r'/\s*([-\d.]+)\s*dBm', val)
if m:
try:
data['tx_power_dbm'] = float(m.group(1))
except ValueError:
pass
elif 'receiver' in key.lower() and ('power' in key.lower() or 'optical' in key.lower()):
m = re.search(r'/\s*([-\d.]+)\s*dBm', val)
if m:
try:
data['rx_power_dbm'] = float(m.group(1))
except ValueError:
pass
elif key == 'Module temperature':
m = re.match(r'([\d.]+)\s+degrees', val)
if m:
data['temp_c'] = float(m.group(1))
elif key == 'Module voltage':
m = re.match(r'([\d.]+)\s+V', val)
if m:
data['voltage_v'] = float(m.group(1))
return data
@staticmethod
def parse_ip_link(text: str) -> dict:
"""Parse ip -s link show output for basic link state and counters."""
data: dict = {}
lines = text.splitlines()
for i, line in enumerate(lines):
# MTU and state: "2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 ..."
m = re.search(r'mtu\s+(\d+)', line)
if m:
data['mtu'] = int(m.group(1))
m = re.search(r'state\s+(\S+)', line)
if m:
data['state'] = m.group(1).lower()
# RX line follows "RX:" label
if line.strip().startswith('RX:') and i + 1 < len(lines):
vals = lines[i + 1].split()
if len(vals) >= 5:
try:
data['ip_rx_bytes'] = int(vals[0])
data['ip_rx_packets'] = int(vals[1])
data['ip_rx_errors'] = int(vals[2])
data['ip_rx_dropped'] = int(vals[3])
except (ValueError, IndexError):
pass
if line.strip().startswith('TX:') and i + 1 < len(lines):
vals = lines[i + 1].split()
if len(vals) >= 5:
try:
data['ip_tx_bytes'] = int(vals[0])
data['ip_tx_packets'] = int(vals[1])
data['ip_tx_errors'] = int(vals[2])
data['ip_tx_dropped'] = int(vals[3])
except (ValueError, IndexError):
pass
return data
@staticmethod
def parse_dmesg(text: str) -> List[dict]:
"""Parse dmesg lines into [{timestamp, msg, severity}]."""
events = []
for line in text.splitlines():
if not line.strip():
continue
# Extract timestamp from [ 123.456789]
m = re.match(r'^\[\s*([\d.]+)\]\s*(.*)', line)
if m:
ts = m.group(1)
msg = m.group(2)
else:
ts = ''
msg = line
lower = msg.lower()
if any(w in lower for w in ('error', 'fail', 'reset', 'panic', 'oops', 'hung', 'timeout')):
severity = 'error'
elif any(w in lower for w in ('warn', 'drop', 'lost', 'miss')):
severity = 'warn'
else:
severity = 'info'
events.append({'timestamp': ts, 'msg': msg, 'severity': severity})
return events
@staticmethod
def parse_lldpctl(text: str) -> dict:
"""Extract neighbor info from lldpctl output."""
if not text or 'lldpd not running' in text or 'not found' in text.lower():
return {'available': False}
data: dict = {'available': True}
for line in text.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if 'SysName' in key:
data['neighbor_system'] = val
elif 'PortID' in key and 'neighbor_port' not in data:
data['neighbor_port'] = val
elif 'ChassisID' in key and 'neighbor_chassis_id' not in data:
data['neighbor_chassis_id'] = val
return data
# ------------------------------------------------------------------
# Health analysis
# ------------------------------------------------------------------
@staticmethod
def analyze(sections: dict, switch_port_data: dict) -> dict:
"""Return {issues: [...], warnings: [...], info: [...]} health analysis."""
issues: List[dict] = []
warnings: List[dict] = []
info: List[dict] = []
def add(collection, code, message):
collection.append({'code': code, 'message': message})
carrier = sections.get('carrier', '?')
eth = sections.get('ethtool', {})
sysfs = sections.get('sysfs_stats', {})
dom = sections.get('ethtool_dom', {})
dmesg = sections.get('dmesg', [])
lldp = sections.get('lldpctl', {})
cc = sections.get('carrier_changes')
# Physical carrier
if carrier == '0':
add(issues, 'NO_CARRIER',
'No physical carrier — cable/SFP disconnected or switch port disabled')
elif eth.get('link_detected') is False and carrier != '0':
add(issues, 'LINK_NOT_DETECTED',
'NIC does not detect link signal despite carrier sysfs reading non-zero')
# Duplex
if eth.get('duplex') == 'half':
add(issues, 'HALF_DUPLEX',
'Half-duplex detected — likely duplex mismatch; force full-duplex on both ends')
# Speed mismatch (switch vs server NIC)
sw_speed = switch_port_data.get('speed_mbps', 0) or 0
srv_speed = eth.get('speed_mbps', 0) or 0
if sw_speed > 0 and srv_speed > 0 and sw_speed != srv_speed:
add(warnings, 'SPEED_MISMATCH',
f'Speed mismatch: switch reports {sw_speed} Mbps, NIC reports {srv_speed} Mbps')
# SFP DOM power levels
rx_dbm = dom.get('rx_power_dbm')
tx_dbm = dom.get('tx_power_dbm')
if rx_dbm is not None:
if rx_dbm < -25:
add(issues, 'SFP_RX_CRITICAL',
f'RX power critically low ({rx_dbm:.2f} dBm) — fiber not connected or SFP failed')
elif rx_dbm < -18:
add(warnings, 'SFP_RX_LOW',
f'RX power low ({rx_dbm:.2f} dBm) — check fiber cleanliness and SFP seating')
if tx_dbm is not None and tx_dbm < -10:
add(warnings, 'SFP_TX_LOW',
f'TX power low ({tx_dbm:.2f} dBm) — SFP may be failing or requires cleaning')
# Carrier changes (flapping)
if cc is not None:
if cc > 100:
add(issues, 'CARRIER_FLAPPING',
f'Link has flapped {cc} times — severe physical instability')
elif cc > 20:
add(warnings, 'CARRIER_FLAPS',
f'Link has flapped {cc} times — intermittent physical issue')
# CRC errors
crc = sysfs.get('rx_crc_errors', 0) or 0
if crc > 100:
add(issues, 'CRC_ERRORS_HIGH',
f'High CRC error count ({crc}) — dirty fiber/connector or cable damage')
elif crc > 10:
add(warnings, 'CRC_ERRORS_LOW',
f'CRC errors present ({crc}) — cable or SFP quality issue')
# Kernel events
err_events = [e for e in dmesg if e['severity'] == 'error']
if err_events:
add(warnings, 'KERNEL_EVENTS',
f'{len(err_events)} recent kernel error event(s) for this interface in dmesg')
# LLDP validation
if lldp.get('available'):
sw_lldp = switch_port_data.get('lldp') or {}
sw_system = (sw_lldp.get('system_name') or '').lower()
srv_neighbor = (lldp.get('neighbor_system') or '').lower()
if sw_system and srv_neighbor and sw_system not in srv_neighbor and srv_neighbor not in sw_system:
add(warnings, 'LLDP_MISMATCH',
f'LLDP mismatch: switch sees "{sw_lldp.get("system_name")}" but '
f'server lldpctl sees "{lldp.get("neighbor_system")}" — cross-cabled port?')
else:
add(info, 'LLDP_MISSING',
'lldpd not running on server — install lldpd for full path validation')
return {'issues': issues, 'warnings': warnings, 'info': info}
+22
View File
@@ -0,0 +1,22 @@
[Unit]
Description=Gandalf Network Monitor Daemon
Documentation=https://gitea.lotusguild.org/LotusGuild/gandalf
After=network.target
Wants=network-online.target
[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/html/prod
ExecStart=/usr/bin/python3 /var/www/html/prod/monitor.py
Restart=on-failure
RestartSec=30
TimeoutStopSec=10
# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=gandalf-monitor
[Install]
WantedBy=multi-user.target
+987
View File
@@ -0,0 +1,987 @@
#!/usr/bin/env python3
"""Gandalf network monitor daemon.
Polls Prometheus (node_exporter) and the UniFi controller for network
interface and device state. Creates tickets in Tinker Tickets when issues
are detected, with deduplication and suppression support.
Run as a separate systemd service alongside the Flask web app.
"""
import json
import logging
import re
import shlex
import time
from datetime import datetime, timezone
from typing import Dict, List, Optional
import requests
from urllib3.exceptions import InsecureRequestWarning
import db
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s %(levelname)s %(name)s %(message)s',
)
logger = logging.getLogger('gandalf.monitor')
# --------------------------------------------------------------------------
# Interface filtering
# --------------------------------------------------------------------------
_SKIP_PREFIXES = (
'lo', 'veth', 'tap', 'fwbr', 'fwln', 'fwpr',
'docker', 'dummy', 'br-', 'virbr', 'vmbr',
)
_VLAN_SUFFIX = re.compile(r'\.\d+$')
def is_physical_interface(name: str) -> bool:
"""Return True for physical/bond interfaces worth monitoring."""
if any(name.startswith(p) for p in _SKIP_PREFIXES):
return False
if _VLAN_SUFFIX.search(name):
return False
return True
# --------------------------------------------------------------------------
# Prometheus client
# --------------------------------------------------------------------------
class PrometheusClient:
def __init__(self, url: str):
self.url = url.rstrip('/')
def query(self, promql: str) -> list:
try:
resp = requests.get(
f'{self.url}/api/v1/query',
params={'query': promql},
timeout=15,
)
resp.raise_for_status()
data = resp.json()
if data.get('status') == 'success':
return data['data']['result']
except Exception as e:
logger.error(f'Prometheus query failed ({promql!r}): {e}')
return []
def get_interface_states(self) -> Dict[str, Dict[str, bool]]:
"""Return {instance: {device: is_up}} for physical interfaces."""
results = self.query('node_network_up')
hosts: Dict[str, Dict[str, bool]] = {}
for r in results:
instance = r['metric'].get('instance', '')
device = r['metric'].get('device', '')
if not is_physical_interface(device):
continue
hosts.setdefault(instance, {})[device] = (r['value'][1] == '1')
return hosts
# --------------------------------------------------------------------------
# UniFi client
# --------------------------------------------------------------------------
class UnifiClient:
def __init__(self, cfg: dict):
self.base_url = cfg['controller']
self.site_id = cfg.get('site_id', 'default')
self.session = requests.Session()
self.session.verify = cfg.get('verify_ssl', True)
if not self.session.verify:
requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
self.headers = {
'X-API-KEY': cfg['api_key'],
'Accept': 'application/json',
}
def get_devices(self) -> Optional[List[dict]]:
"""Return list of UniFi devices, or None if the controller is unreachable."""
try:
url = f'{self.base_url}/proxy/network/v2/api/site/{self.site_id}/device'
resp = self.session.get(url, headers=self.headers, timeout=15)
resp.raise_for_status()
data = resp.json()
devices = []
for d in data.get('network_devices', []):
state = d.get('state', 1)
devices.append({
'name': d.get('name') or d.get('mac', 'unknown'),
'mac': d.get('mac', ''),
'ip': d.get('ip', ''),
'type': d.get('type', 'unknown'),
'model': d.get('model', ''),
'state': state,
'connected': state == 1,
})
return devices
except Exception as e:
logger.error(f'UniFi API error: {e}')
return None
def get_switch_ports(self) -> Optional[Dict[str, dict]]:
"""Return per-port stats for all UniFi switches, keyed by switch name.
Uses the v1 stat API which includes full port_table data.
Returns {switch_name: {'ip': str, 'model': str, 'ports': {port_name: {...}}}}.
"""
try:
url = f'{self.base_url}/proxy/network/api/s/{self.site_id}/stat/device'
resp = self.session.get(url, headers=self.headers, timeout=15)
resp.raise_for_status()
devices = resp.json().get('data', [])
result: Dict[str, dict] = {}
for dev in devices:
if dev.get('type', '').lower() != 'usw':
continue
sw_name = dev.get('name') or dev.get('mac', 'unknown')
sw_ip = dev.get('ip', '')
sw_model = dev.get('model', '')
ports: Dict[str, dict] = {}
# Build LLDP neighbor map (keyed by port_idx)
lldp_map: Dict[int, dict] = {}
for entry in dev.get('lldp_table', []):
pidx = entry.get('lldp_port_idx')
if pidx is not None:
lldp_map[int(pidx)] = {
'chassis_id': entry.get('chassis_id', ''),
'system_name': entry.get('system_name', ''),
'port_id': entry.get('port_id', ''),
'port_desc': entry.get('port_desc', ''),
'mgmt_ips': entry.get('management_ips', []),
}
for port in dev.get('port_table', []):
idx = port.get('port_idx', 0)
pname = port.get('name') or f'Port {idx}'
raw_poe = port.get('poe_power')
raw_poe_max = port.get('poe_max_power')
ports[pname] = {
'port_idx': idx,
'switch_ip': sw_ip,
'up': port.get('up', False),
'speed_mbps': port.get('speed', 0),
'full_duplex': port.get('full_duplex', False),
'autoneg': port.get('autoneg', False),
'is_uplink': port.get('is_uplink', False),
'media': port.get('media', ''),
'poe_power': float(raw_poe) if raw_poe is not None else None,
'poe_class': port.get('poe_class'),
'poe_max_power': float(raw_poe_max) if raw_poe_max is not None else None,
'poe_mode': port.get('poe_mode', ''),
'lldp': lldp_map.get(idx),
'tx_bytes': port.get('tx_bytes', 0),
'rx_bytes': port.get('rx_bytes', 0),
'tx_errors': port.get('tx_errors', 0),
'rx_errors': port.get('rx_errors', 0),
'tx_dropped': port.get('tx_dropped', 0),
'rx_dropped': port.get('rx_dropped', 0),
}
if ports:
result[sw_name] = {'ip': sw_ip, 'model': sw_model, 'ports': ports}
return result
except Exception as e:
logger.error(f'UniFi switch port stats error: {e}')
return None
# --------------------------------------------------------------------------
# Ticket client
# --------------------------------------------------------------------------
class TicketClient:
def __init__(self, cfg: dict):
self.url = cfg.get('url', '')
self.api_key = cfg.get('api_key', '')
def create(self, title: str, description: str, priority: str = '2') -> Optional[str]:
if not self.api_key or not self.url:
logger.warning('Ticket API not configured – skipping ticket creation')
return None
try:
resp = requests.post(
self.url,
json={
'title': title,
'description': description,
'status': 'Open',
'priority': priority,
'category': 'Network',
'type': 'Issue',
},
headers={'Authorization': f'Bearer {self.api_key}'},
timeout=15,
)
resp.raise_for_status()
data = resp.json()
if data.get('success'):
tid = data.get('ticket_id')
if not tid:
logger.warning(f'Ticket API success but no ticket_id in response: {data}')
return None
logger.info(f'Created ticket #{tid}: {title}')
return tid
if data.get('existing_ticket_id'):
logger.info(f'Duplicate suppressed by API – existing #{data["existing_ticket_id"]}')
return data['existing_ticket_id']
logger.warning(f'Unexpected ticket API response: {data}')
except Exception as e:
logger.error(f'Ticket creation failed: {e}')
return None
# --------------------------------------------------------------------------
# Pulse HTTP client (delegates SSH commands to Pulse worker)
# --------------------------------------------------------------------------
class PulseClient:
"""Submit a command to a Pulse worker via the internal M2M API and poll for result."""
def __init__(self, cfg: dict):
p = cfg.get('pulse', {})
self.url = p.get('url', '').rstrip('/')
self.api_key = p.get('api_key', '')
self.worker_id = p.get('worker_id', '')
self.timeout = p.get('timeout', 45)
self.last_execution_id: Optional[str] = None
self.session = requests.Session()
self.session.headers.update({
'X-Gandalf-API-Key': self.api_key,
'Content-Type': 'application/json',
})
def run_command(self, command: str, _retry: bool = True) -> Optional[str]:
"""Submit *command* to Pulse, poll until done, return stdout or None.
Retries once automatically on transient submit failures or timeouts.
"""
self.last_execution_id = None
if not self.url or not self.api_key or not self.worker_id:
return None
# Submit
try:
resp = self.session.post(
f'{self.url}/api/internal/command',
json={'worker_id': self.worker_id, 'command': command},
timeout=10,
)
resp.raise_for_status()
execution_id = resp.json().get('execution_id')
if not execution_id:
logger.error('Pulse submit response missing execution_id')
return None
self.last_execution_id = execution_id
except Exception as e:
logger.error(f'Pulse command submit failed: {e}')
if _retry:
logger.info('Retrying Pulse command submit in 5s...')
time.sleep(5)
return self.run_command(command, _retry=False)
return None
# Poll
deadline = time.time() + self.timeout
while time.time() < deadline:
time.sleep(2)
try:
r = self.session.get(
f'{self.url}/api/internal/executions/{execution_id}',
timeout=10,
)
r.raise_for_status()
data = r.json()
status = data.get('status')
if status == 'completed':
logs = data.get('logs', [])
for entry in logs:
if entry.get('action') == 'command_result':
return entry.get('stdout', '')
return ''
if status in ('failed', 'timed_out', 'cancelled'):
logger.error(
f'Pulse execution {execution_id} ended with status={status!r}; '
f'view at {self.url}/executions/{execution_id}'
)
if _retry and status != 'cancelled':
logger.info('Retrying failed Pulse command in 5s...')
time.sleep(5)
return self.run_command(command, _retry=False)
return None
except Exception as e:
logger.error(f'Pulse poll failed for {execution_id}: {e}')
logger.warning(
f'Pulse command timed out after {self.timeout}s '
f'(execution_id={execution_id}); '
f'view at {self.url}/executions/{execution_id}'
)
if _retry:
logger.info('Retrying timed-out Pulse command in 5s...')
time.sleep(5)
return self.run_command(command, _retry=False)
return None
def ping(self, ip: str, count: int = 3, timeout: int = 2) -> bool:
"""Ping *ip* via the Pulse worker. Returns True if host responds."""
ip_q = shlex.quote(ip)
output = self.run_command(
f'ping -c {count} -W {timeout} {ip_q} >/dev/null 2>&1 && echo REACHABLE || echo UNREACHABLE'
)
return output is not None and output.strip() == 'REACHABLE'
# --------------------------------------------------------------------------
# Link stats collector (ethtool + Prometheus traffic metrics)
# --------------------------------------------------------------------------
class LinkStatsCollector:
"""Collects detailed per-interface statistics via SSH (ethtool) and Prometheus,
plus per-port stats from UniFi switches."""
def __init__(self, cfg: dict, prom: 'PrometheusClient',
unifi: Optional['UnifiClient'] = None):
self.cfg = cfg
self.prom = prom
self.pulse = PulseClient(cfg)
self.unifi = unifi
# State for UniFi rate calculation (previous snapshot + timestamp)
self._prev_unifi: Dict[str, dict] = {}
self._prev_unifi_time: float = 0.0
# ------------------------------------------------------------------
# SSH collection (via Pulse worker)
# ------------------------------------------------------------------
def _ssh_batch(self, ip: str, ifaces: List[str]) -> Dict[str, dict]:
"""
Delegate one SSH session to the Pulse worker to collect ethtool + SFP DOM
data for all *ifaces*. Returns {iface: {speed_mbps, duplex, ..., sfp: {...}}}.
"""
if not ifaces or not self.pulse.url:
return {}
# Validate interface names (kernel names: [a-zA-Z0-9_.-], max 15 chars per IFNAMSIZ)
safe_ifaces = [i for i in ifaces if re.match(r'^[a-zA-Z0-9_.-]{1,15}$', i)]
if not safe_ifaces:
return {}
# Build a single shell command: for each iface output ethtool + -m with sentinels
parts = []
for iface in safe_ifaces:
q = shlex.quote(iface)
parts.append(
f'echo "___IFACE:{iface}___";'
f' ethtool {q} 2>/dev/null;'
f' echo "___DOM:{iface}___";'
f' ethtool -m {q} 2>/dev/null;'
f' echo "___END___"'
)
shell_cmd = ' '.join(parts)
ssh_cmd = (
f'ssh -o StrictHostKeyChecking=accept-new -o ConnectTimeout=5 '
f'-o BatchMode=yes -o LogLevel=ERROR '
f'-o ServerAliveInterval=10 -o ServerAliveCountMax=2 '
f'root@{ip} {shlex.quote(shell_cmd)}'
)
output = self.pulse.run_command(ssh_cmd)
if output is None:
logger.error(f'Pulse ethtool collection returned None for {ip}')
return {}
return self._parse_ssh_output(output)
@staticmethod
def _parse_ssh_output(output: str) -> Dict[str, dict]:
result: Dict[str, dict] = {}
current_iface: Optional[str] = None
current_section: Optional[str] = None
buf: List[str] = []
def flush(iface, section, lines):
if not iface or not lines:
return
text = '\n'.join(lines)
if section == 'ethtool':
result.setdefault(iface, {}).update(
LinkStatsCollector._parse_ethtool(text)
)
elif section == 'dom':
sfp = LinkStatsCollector._parse_ethtool_m(text)
if sfp:
result.setdefault(iface, {})['sfp'] = sfp
for line in output.splitlines():
if line.startswith('___IFACE:') and line.endswith('___'):
flush(current_iface, current_section, buf)
current_iface = line[9:-3]
current_section = 'ethtool'
buf = []
elif line.startswith('___DOM:') and line.endswith('___'):
flush(current_iface, current_section, buf)
current_iface = line[7:-3]
current_section = 'dom'
buf = []
elif line == '___END___':
flush(current_iface, current_section, buf)
current_iface = None
current_section = None
buf = []
else:
buf.append(line)
flush(current_iface, current_section, buf)
return result
@staticmethod
def _parse_ethtool(output: str) -> dict:
data: dict = {}
for line in output.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if key == 'Speed':
m = re.match(r'(\d+)Mb/s', val)
if m:
data['speed_mbps'] = int(m.group(1))
elif key == 'Duplex':
data['duplex'] = val.lower()
elif key == 'Port':
data['port_type'] = val
elif key == 'Auto-negotiation':
data['auto_neg'] = (val.lower() == 'on')
elif key == 'Link detected':
data['link_detected'] = (val.lower() == 'yes')
return data
@staticmethod
def _parse_ethtool_m(output: str) -> dict:
"""Parse ethtool -m (SFP DOM / digital optical monitoring) output."""
if not output:
return {}
# Skip if module diagnostics unsupported
lower = output.lower()
if 'cannot get' in lower or 'not supported' in lower or 'no sfp' in lower:
return {}
data: dict = {}
for line in output.splitlines():
if ':' not in line:
continue
key, _, val = line.partition(':')
key = key.strip()
val = val.strip()
if key == 'Vendor name':
data['vendor'] = val
elif key == 'Vendor PN':
data['part_no'] = val
elif key == 'Identifier':
m = re.search(r'\((.+?)\)', val)
if m:
data['sfp_type'] = m.group(1)
elif key == 'Connector':
m = re.search(r'\((.+?)\)', val)
if m:
data['connector'] = m.group(1)
elif key == 'Laser wavelength':
m = re.match(r'(\d+)', val)
if m:
data['wavelength_nm'] = int(m.group(1))
elif key == 'Laser bias current':
# e.g. "4.340 mA"
m = re.match(r'([\d.]+)\s+mA', val)
if m:
data['bias_ma'] = float(m.group(1))
elif key == 'Laser output power':
# e.g. "0.1234 mW / -9.09 dBm"
m = re.search(r'/\s*([-\d.]+)\s*dBm', val)
if m:
try:
data['tx_power_dbm'] = float(m.group(1))
except ValueError:
pass
elif 'receiver' in key.lower() and ('power' in key.lower() or 'optical' in key.lower()):
m = re.search(r'/\s*([-\d.]+)\s*dBm', val)
if m:
try:
data['rx_power_dbm'] = float(m.group(1))
except ValueError:
pass
elif key == 'Module temperature':
# e.g. "36.00 degrees C / 96.80 degrees F"
m = re.match(r'([\d.]+)\s+degrees', val)
if m:
data['temp_c'] = float(m.group(1))
elif key == 'Module voltage':
# e.g. "3.3180 V"
m = re.match(r'([\d.]+)\s+V', val)
if m:
data['voltage_v'] = float(m.group(1))
return data
# ------------------------------------------------------------------
# Prometheus traffic / error metrics
# ------------------------------------------------------------------
def _collect_prom_metrics(self) -> Dict[str, Dict[str, dict]]:
"""Return {instance: {device: {tx_bytes_rate, rx_bytes_rate, ...}}}."""
metrics: Dict[str, Dict[str, dict]] = {}
queries = [
('tx_bytes_rate', 'rate(node_network_transmit_bytes_total[5m])'),
('rx_bytes_rate', 'rate(node_network_receive_bytes_total[5m])'),
('tx_errs_rate', 'rate(node_network_transmit_errs_total[5m])'),
('rx_errs_rate', 'rate(node_network_receive_errs_total[5m])'),
('tx_drops_rate', 'rate(node_network_transmit_drop_total[5m])'),
('rx_drops_rate', 'rate(node_network_receive_drop_total[5m])'),
('carrier_changes', 'node_network_carrier_changes_total'),
]
for field, promql in queries:
for r in self.prom.query(promql):
instance = r['metric'].get('instance', '')
device = r['metric'].get('device', '')
if not is_physical_interface(device):
continue
raw = r['value'][1]
try:
val: Optional[float] = float(raw)
if val != val: # NaN
val = None
except (ValueError, TypeError):
val = None
metrics.setdefault(instance, {}).setdefault(device, {})[field] = val
return metrics
# ------------------------------------------------------------------
# Main collection entry point
# ------------------------------------------------------------------
def collect(self, instance_map: Dict[str, str]) -> dict:
"""
Collect full link stats for all Prometheus-monitored hosts.
*instance_map*: ``{'10.10.10.2:9100': 'large1', ...}``
Returns a dict suitable for ``db.set_state('link_stats', ...)``.
"""
prom_metrics = self._collect_prom_metrics()
result_hosts: Dict[str, Dict[str, dict]] = {}
exclude_ips = set(self.cfg.get('monitor', {}).get('links_exclude_ips', []))
for instance, iface_metrics in prom_metrics.items():
host_ip = instance.split(':')[0]
if host_ip in exclude_ips:
continue
host = instance_map.get(instance, host_ip)
ifaces = list(iface_metrics.keys())
# SSH ethtool collection via Pulse worker — only for explicitly configured
# hosts (instance_map keys). Hosts like postgresql/matrix may report
# node_exporter metrics to Prometheus but don't need link diagnostics.
ethtool_data: Dict[str, dict] = {}
if self.pulse.url and ifaces and instance in instance_map:
try:
ethtool_data = self._ssh_batch(host_ip, ifaces)
except Exception as e:
logger.warning(f'ethtool collection failed for {host} ({host_ip}): {e}')
# Merge Prometheus metrics + ethtool data per interface
merged: Dict[str, dict] = {}
for iface in ifaces:
d: dict = {'host_ip': host_ip}
d.update(iface_metrics.get(iface, {}))
eth = ethtool_data.get(iface, {})
for k, v in eth.items():
if k != 'sfp':
d[k] = v
if 'sfp' in eth:
d['sfp'] = eth['sfp']
merged[iface] = d
result_hosts[host] = merged
# Collect UniFi switch port stats
unifi_switches: dict = {}
if self.unifi:
try:
raw = self.unifi.get_switch_ports()
if raw is not None:
now = time.time()
unifi_switches = self._compute_unifi_rates(raw, now)
self._prev_unifi = raw
self._prev_unifi_time = now
except Exception as e:
logger.warning(f'UniFi switch port collection failed: {e}')
return {
'hosts': result_hosts,
'unifi_switches': unifi_switches,
'updated': datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC'),
}
def _compute_unifi_rates(self, raw: Dict[str, dict], now: float) -> Dict[str, dict]:
"""Compute per-port byte/error rates from delta against previous snapshot."""
dt = now - self._prev_unifi_time if self._prev_unifi_time > 0 else 0
def rate(new_val: int, old_val: int) -> Optional[float]:
if dt <= 0:
return None
return max(0.0, (new_val - old_val) / dt)
result: Dict[str, dict] = {}
for sw_name, sw_data in raw.items():
prev_ports = self._prev_unifi.get(sw_name, {}).get('ports', {})
merged_ports: Dict[str, dict] = {}
for pname, d in sw_data['ports'].items():
entry = dict(d)
prev = prev_ports.get(pname, {})
entry['tx_bytes_rate'] = rate(d['tx_bytes'], prev.get('tx_bytes', 0))
entry['rx_bytes_rate'] = rate(d['rx_bytes'], prev.get('rx_bytes', 0))
entry['tx_errs_rate'] = rate(d['tx_errors'], prev.get('tx_errors', 0))
entry['rx_errs_rate'] = rate(d['rx_errors'], prev.get('rx_errors', 0))
entry['tx_drops_rate'] = rate(d['tx_dropped'], prev.get('tx_dropped', 0))
entry['rx_drops_rate'] = rate(d['rx_dropped'], prev.get('rx_dropped', 0))
merged_ports[pname] = entry
result[sw_name] = {'ip': sw_data['ip'], 'model': sw_data['model'],
'ports': merged_ports}
return result
# --------------------------------------------------------------------------
# Helpers
# --------------------------------------------------------------------------
def _now_utc() -> str:
return datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S UTC')
# --------------------------------------------------------------------------
# Monitor
# --------------------------------------------------------------------------
CLUSTER_NAME = 'proxmox-cluster'
class NetworkMonitor:
def __init__(self):
with open('config.json') as f:
self.cfg = json.load(f)
prom_url = self.cfg['prometheus']['url']
self.prom = PrometheusClient(prom_url)
self.unifi = UnifiClient(self.cfg['unifi'])
self.tickets = TicketClient(self.cfg.get('ticket_api', {}))
self.link_stats = LinkStatsCollector(self.cfg, self.prom, self.unifi)
self.pulse = self.link_stats.pulse # convenience alias
mon = self.cfg.get('monitor', {})
self.poll_interval = mon.get('poll_interval', 120)
self.fail_thresh = mon.get('failure_threshold', 2)
self.cluster_thresh = mon.get('cluster_threshold', 3)
self.cluster_name = mon.get('cluster_name', CLUSTER_NAME)
# Build Prometheus instance → hostname lookup
self._instance_map: Dict[str, str] = {
h['prometheus_instance']: h['name']
for h in self.cfg.get('hosts', [])
if 'prometheus_instance' in h
}
def _hostname(self, instance: str) -> str:
return self._instance_map.get(instance, instance.split(':')[0])
# ------------------------------------------------------------------
# Interface monitoring (Prometheus)
# ------------------------------------------------------------------
def _process_interfaces(self, states: Dict[str, Dict[str, bool]], suppressions: list) -> None:
baseline = db.get_baseline()
new_baseline = {k: dict(v) for k, v in baseline.items()}
# Only count hosts with genuine regressions (UP→DOWN) toward cluster threshold
hosts_with_regression: List[str] = []
for instance, ifaces in states.items():
if instance not in self._instance_map:
continue # skip unconfigured Prometheus instances
host = self._hostname(instance)
new_baseline.setdefault(host, {})
host_has_regression = False
for iface, is_up in ifaces.items():
prev = baseline.get(host, {}).get(iface) # 'up', 'initial_down', or None
if is_up:
new_baseline[host][iface] = 'up'
db.resolve_event('interface_down', host, iface)
else:
if prev is None:
# First observation is down – could be unused port, don't alert
new_baseline[host][iface] = 'initial_down'
elif prev == 'initial_down':
# Persistently down since first observation – no alert
pass
else: # prev == 'up'
# Regression: was UP, now DOWN
host_has_regression = True
sup = (
db.check_suppressed(suppressions, 'interface', host, iface) or
db.check_suppressed(suppressions, 'host', host)
)
event_id, is_new, consec = db.upsert_event(
'interface_down', 'critical', 'prometheus',
host, iface,
f'Interface {iface} on {host} went link-down ({_now_utc()})',
)
if not sup and consec >= self.fail_thresh:
self._ticket_interface(event_id, host, iface, consec)
if host_has_regression:
hosts_with_regression.append(host)
db.set_baseline(new_baseline)
# Cluster-wide check – only genuine regressions count
if len(hosts_with_regression) >= self.cluster_thresh:
sup = db.check_suppressed(suppressions, 'all', '')
event_id, is_new, consec = db.upsert_event(
'cluster_network_issue', 'critical', 'prometheus',
self.cluster_name, '',
f'{len(hosts_with_regression)} hosts reporting simultaneous interface failures: '
f'{", ".join(hosts_with_regression)}',
)
if not sup and is_new:
title = (
f'[{self.cluster_name}][auto][production][issue][network][cluster-wide] '
f'Multiple hosts reporting interface failures'
)
desc = (
f'Cluster Network Alert\n{"=" * 40}\n\n'
f'Affected hosts: {", ".join(hosts_with_regression)}\n'
f'Detected: {_now_utc()}\n\n'
f'{len(hosts_with_regression)} Proxmox hosts simultaneously reported '
f'interface regressions (link-down on interfaces previously known UP).\n'
f'This likely indicates a switch or upstream network failure.\n\n'
f'Please check the core and management switches immediately.'
)
tid = self.tickets.create(title, desc, priority='1')
if tid:
db.set_ticket_id(event_id, tid)
else:
db.resolve_event('cluster_network_issue', self.cluster_name, '')
def _ticket_interface(
self, event_id: int, host: str, iface: str, consec: int
) -> None:
title = (
f'[{host}][auto][production][issue][network][single-node] '
f'Interface {iface} link-down'
)
desc = (
f'Network Interface Alert\n{"=" * 40}\n\n'
f'Host: {host}\n'
f'Interface: {iface}\n'
f'Detected: {_now_utc()}\n'
f'Consecutive check failures: {consec}\n\n'
f'Interface {iface} on {host} is reporting link-down state via '
f'Prometheus node_exporter.\n\n'
f'Note: {host} may still be reachable via its other network interface.\n'
f'Please inspect the cable/SFP/switch port for {host}/{iface}.'
)
tid = self.tickets.create(title, desc, priority='2')
if tid:
db.set_ticket_id(event_id, tid)
# ------------------------------------------------------------------
# UniFi device monitoring
# ------------------------------------------------------------------
def _process_unifi(self, devices: Optional[List[dict]], suppressions: list) -> None:
if devices is None:
logger.warning('UniFi API unreachable this cycle')
return
for d in devices:
name = d['name']
if not d['connected']:
sup = db.check_suppressed(suppressions, 'unifi_device', name)
event_id, is_new, consec = db.upsert_event(
'unifi_device_offline', 'critical', 'unifi',
name, d.get('type', ''),
f'UniFi {name} ({d.get("ip","")}) offline ({_now_utc()})',
)
if not sup and consec >= self.fail_thresh:
self._ticket_unifi(event_id, d)
else:
db.resolve_event('unifi_device_offline', name, d.get('type', ''))
def _ticket_unifi(self, event_id: int, device: dict) -> None:
name = device['name']
title = (
f'[{name}][auto][production][issue][network][single-node] '
f'UniFi device offline'
)
desc = (
f'UniFi Device Alert\n{"=" * 40}\n\n'
f'Device: {name}\n'
f'Type: {device.get("type","unknown")}\n'
f'Model: {device.get("model","")}\n'
f'Last Known IP: {device.get("ip","unknown")}\n'
f'Detected: {_now_utc()}\n\n'
f'The UniFi device {name} is offline per the UniFi controller.\n'
f'Please check power and cable connectivity.'
)
tid = self.tickets.create(title, desc, priority='2')
if tid:
db.set_ticket_id(event_id, tid)
# ------------------------------------------------------------------
# Ping-only hosts (no node_exporter)
# ------------------------------------------------------------------
def _process_ping_hosts(self, suppressions: list, ping_states: Dict[str, bool]) -> None:
for h in self.cfg.get('monitor', {}).get('ping_hosts', []):
name, ip = h['name'], h['ip']
reachable = ping_states.get(name, False)
if not reachable:
sup = db.check_suppressed(suppressions, 'host', name)
event_id, is_new, consec = db.upsert_event(
'host_unreachable', 'critical', 'ping',
name, ip,
f'Host {name} ({ip}) unreachable via ping ({_now_utc()})',
)
if not sup and consec >= self.fail_thresh:
self._ticket_unreachable(event_id, name, ip, consec)
else:
db.resolve_event('host_unreachable', name, ip)
def _ticket_unreachable(
self, event_id: int, name: str, ip: str, consec: int
) -> None:
title = (
f'[{name}][auto][production][issue][network][single-node] '
f'Host unreachable'
)
desc = (
f'Host Reachability Alert\n{"=" * 40}\n\n'
f'Host: {name}\n'
f'IP: {ip}\n'
f'Detected: {_now_utc()}\n'
f'Consecutive check failures: {consec}\n\n'
f'Host {name} ({ip}) is not responding to ping from the Gandalf monitor.\n'
f'This host does not have a Prometheus node_exporter, so interface-level '
f'detail is unavailable.\n\n'
f'Please check the host power, management interface, and network connectivity.'
)
tid = self.tickets.create(title, desc, priority='2')
if tid:
db.set_ticket_id(event_id, tid)
# ------------------------------------------------------------------
# Snapshot collection (for dashboard)
# ------------------------------------------------------------------
def _collect_snapshot(
self, iface_states: Dict[str, Dict[str, bool]],
unifi_devices: Optional[List[dict]] = None,
ping_states: Optional[Dict[str, bool]] = None,
) -> dict:
# Accept pre-fetched devices; fall back to empty list if unavailable
display_unifi = unifi_devices if unifi_devices is not None else []
hosts = {}
for instance, ifaces in iface_states.items():
if instance not in self._instance_map:
continue # skip Prometheus instances not in config (e.g. LXC app servers)
host = self._hostname(instance)
phys = {k: v for k, v in ifaces.items()}
up_count = sum(1 for v in phys.values() if v)
total = len(phys)
if total == 0 or up_count == total:
status = 'up'
elif up_count == 0:
status = 'down'
else:
status = 'degraded'
hosts[host] = {
'ip': instance.split(':')[0],
'interfaces': {k: ('up' if v else 'down') for k, v in phys.items()},
'status': status,
'source': 'prometheus',
}
for h in self.cfg.get('monitor', {}).get('ping_hosts', []):
name, ip = h['name'], h['ip']
reachable = (ping_states or {}).get(name, False)
hosts[name] = {
'ip': ip,
'interfaces': {},
'status': 'up' if reachable else 'down',
'source': 'ping',
}
return {
'hosts': hosts,
'unifi': display_unifi,
'updated': datetime.now(timezone.utc).isoformat().replace('+00:00', 'Z'),
}
# ------------------------------------------------------------------
# Main loop
# ------------------------------------------------------------------
def run(self) -> None:
logger.info(
f'Gandalf monitor started – poll_interval={self.poll_interval}s '
f'fail_thresh={self.fail_thresh}'
)
while True:
try:
logger.info('Starting network check cycle')
# 1. Fetch interface states once — shared by snapshot and alert processing
iface_states = self.prom.get_interface_states()
# 2. Fetch UniFi devices once — used by both snapshot and alert processing
unifi_devices = self.unifi.get_devices()
# 3a. Ping-only hosts once — shared by snapshot and alert processing
ping_states: Dict[str, bool] = {
h['name']: self.pulse.ping(h['ip'])
for h in self.cfg.get('monitor', {}).get('ping_hosts', [])
}
# 3b. Collect and store snapshot for dashboard
snapshot = self._collect_snapshot(iface_states, unifi_devices, ping_states)
db.set_state('network_snapshot', snapshot)
db.set_state('last_check', _now_utc())
# 4. Collect link stats (ethtool + traffic metrics)
try:
link_data = self.link_stats.collect(self._instance_map)
db.set_state('link_stats', link_data)
except Exception as e:
logger.error(f'Link stats collection failed: {e}', exc_info=True)
# 5. Process alerts using already-fetched data
suppressions = db.get_active_suppressions()
self._process_interfaces(iface_states, suppressions)
self._process_unifi(unifi_devices, suppressions)
self._process_ping_hosts(suppressions, ping_states)
# Housekeeping: deactivate expired suppressions and purge old resolved events
db.cleanup_expired_suppressions()
db.purge_old_resolved_events(days=90)
logger.info('Network check cycle complete')
except Exception as e:
logger.error(f'Monitor loop error: {e}', exc_info=True)
time.sleep(30)
continue
time.sleep(self.poll_interval)
if __name__ == '__main__':
monitor = NetworkMonitor()
monitor.run()
+1140
View File
File diff suppressed because it is too large. Load diff
+5
View File
@@ -0,0 +1,5 @@
{
"devDependencies": {
"eslint": "^8.57.1"
}
}
+6
View File
@@ -0,0 +1,6 @@
flask>=2.2.0
gunicorn>=20.1.0
ldap3>=2.9
pymysql>=1.1.0
requests>=2.31.0
urllib3>=2.0.0
+54
View File
@@ -0,0 +1,54 @@
-- Gandalf Network Monitor – Database Schema
-- Run on MariaDB LXC 149 (10.10.10.50)
CREATE DATABASE IF NOT EXISTS gandalf
CHARACTER SET utf8mb4
COLLATE utf8mb4_unicode_ci;
USE gandalf;
-- ── Network events (open and resolved alerts) ─────────────────────────
CREATE TABLE IF NOT EXISTS network_events (
id INT AUTO_INCREMENT PRIMARY KEY,
event_type VARCHAR(60) NOT NULL,
severity ENUM('critical','warning','info') NOT NULL DEFAULT 'warning',
source_type VARCHAR(20) NOT NULL, -- 'prometheus', 'unifi', 'ping'
target_name VARCHAR(255) NOT NULL, -- hostname or device name
target_detail VARCHAR(255) NOT NULL DEFAULT '', -- interface name, device type, IP
description TEXT,
first_seen TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
last_seen TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
resolved_at TIMESTAMP NULL,
consecutive_failures INT NOT NULL DEFAULT 1,
ticket_id VARCHAR(20) NULL,
INDEX idx_active (resolved_at),
INDEX idx_target (target_name, target_detail),
INDEX idx_type (event_type),
-- Compound index for hot upsert_event lookup: event_type+target+resolved_at
INDEX idx_event_lookup (event_type, target_name, target_detail, resolved_at)
) ENGINE=InnoDB;
-- ── Suppression rules ─────────────────────────────────────────────────
CREATE TABLE IF NOT EXISTS suppression_rules (
id INT AUTO_INCREMENT PRIMARY KEY,
target_type VARCHAR(50) NOT NULL, -- 'host', 'interface', 'unifi_device', 'all'
target_name VARCHAR(255) NOT NULL DEFAULT '',
target_detail VARCHAR(255) NOT NULL DEFAULT '',
reason TEXT NOT NULL,
suppressed_by VARCHAR(255) NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
expires_at TIMESTAMP NULL, -- NULL = manual (never auto-expires)
active BOOLEAN NOT NULL DEFAULT TRUE,
INDEX idx_active_exp (active, expires_at),
-- For suppression lookup by type+name
INDEX idx_sup_lookup (active, target_type, target_name, target_detail)
) ENGINE=InnoDB;
-- ── Monitor state (key/value store for snapshot + baseline) ───────────
CREATE TABLE IF NOT EXISTS monitor_state (
key_name VARCHAR(100) PRIMARY KEY,
value MEDIUMTEXT NOT NULL,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB;
+13
View File
@@ -0,0 +1,13 @@
{
"env": { "browser": true, "es2021": true },
"extends": "eslint:recommended",
"parserOptions": { "ecmaVersion": 2021, "sourceType": "script" },
"rules": {
"no-unused-vars": "warn",
"no-undef": "off",
"no-empty": "warn",
"no-inner-declarations": "warn",
"semi": ["error", "always"],
"eqeqeq": "warn"
}
}
+390
View File
@@ -0,0 +1,390 @@
'use strict';
// ── Auto-redirect on auth timeout ─────────────────────────────────────
// Wraps fetch so a 401 (Authelia session expired) forces a full reload.
// lt.api uses fetch internally, so this covers all API calls too.
(function () {
const _fetch = window.fetch;
window.fetch = async function (...args) {
const resp = await _fetch(...args);
if (resp.status === 401) {
window.location.reload();
throw new Error('Session expired — reloading');
}
return resp;
};
})();
// ── Toast notifications — thin wrapper over lt.toast ──────────────────
function showToast(msg, type = 'success') {
if (type === 'error') return lt.toast.error(msg);
if (type === 'warning') return lt.toast.warning(msg);
if (type === 'info') return lt.toast.info(msg);
return lt.toast.success(msg);
}
// ── Normalise UTC timestamp string for Date() parsing ─────────────────
// Server returns "2026-03-14 14:14:21 UTC"; Date() needs ISO 8601.
function _toIso(s) {
if (!s) return s;
return s.replace(' UTC', 'Z').replace(' ', 'T');
}
// ── Dashboard auto-refresh ────────────────────────────────────────────
async function refreshAll() {
const refreshBtn = document.querySelector('[data-action="refresh"]');
const spinner = document.getElementById('refresh-spinner');
if (refreshBtn) refreshBtn.classList.add('is-loading');
if (spinner) spinner.style.display = '';
try {
const [netResult, statusResult] = await Promise.allSettled([
lt.api.get('/api/network'),
lt.api.get('/api/status'),
]);
if (netResult.status === 'fulfilled') {
const net = netResult.value;
updateHostGrid(net.hosts || {});
updateUnifiTable(net.unifi || []);
updateTopology(net.hosts || {});
} else {
showToast('Network data unavailable', 'warning');
}
if (statusResult.status === 'fulfilled') {
const status = statusResult.value;
updateEventsTable(status.events || [], status.total_active);
updateStatusBar(status.summary || {}, status.last_check || '', status.daemon_ok);
if (typeof window.gandalfNotifUpdate === 'function') window.gandalfNotifUpdate(status.events || []);
} else {
showToast('Status data unavailable', 'warning');
}
} finally {
if (refreshBtn) refreshBtn.classList.remove('is-loading');
if (spinner) spinner.style.display = 'none';
}
}
function updateStatusBar(summary, lastCheck, daemonOk) {
const bar = document.querySelector('.status-chips');
if (!bar) return;
const chips = [];
if (daemonOk === false) chips.push('<span class="chip chip-critical">⚠ MONITOR OFFLINE</span>');
if (summary.critical) chips.push(`<span class="chip chip-critical">● ${summary.critical} CRITICAL</span>`);
if (summary.warning) chips.push(`<span class="chip chip-warning">● ${summary.warning} WARNING</span>`);
if (!summary.critical && !summary.warning && daemonOk !== false) chips.push('<span class="chip chip-ok">✔ ALL SYSTEMS NOMINAL</span>');
bar.innerHTML = chips.join('');
const lc = document.getElementById('last-check');
if (lc && lastCheck) lc.textContent = lastCheck;
const critCount = summary.critical || 0;
const warnCount = summary.warning || 0;
if (critCount) document.title = `(${critCount} CRIT) GANDALF`;
else if (warnCount) document.title = `(${warnCount} WARN) GANDALF`;
else document.title = 'GANDALF';
const alertBadge = document.getElementById('alert-count-badge');
if (alertBadge) {
const total = critCount + warnCount;
alertBadge.textContent = total;
alertBadge.style.display = total ? '' : 'none';
}
// Update stat cards
const scCrit = document.getElementById('stat-critical-val');
const scWarn = document.getElementById('stat-warning-val');
const scRes = document.getElementById('stat-resolved-val');
if (scCrit) scCrit.textContent = critCount;
if (scWarn) scWarn.textContent = warnCount;
if (scRes && summary.resolved_24h !== null && summary.resolved_24h !== undefined) scRes.textContent = summary.resolved_24h;
const statCritCard = document.getElementById('stat-critical');
if (statCritCard) statCritCard.classList.toggle('lt-stat-card--alert', critCount > 0);
// Stale data banner: warn if last_check is older than 15 minutes
let staleBanner = document.getElementById('stale-banner');
if (lastCheck) {
const checkAge = (Date.now() - new Date(_toIso(lastCheck))) / 1000;
if (checkAge > 900) {
if (!staleBanner) {
staleBanner = document.createElement('div');
staleBanner.id = 'stale-banner';
staleBanner.className = 'lt-alert lt-alert--warning';
staleBanner.innerHTML = '<span class="lt-alert-icon">⚠</span><div class="lt-alert-body"><div class="lt-alert-msg"></div></div>';
document.querySelector('.lt-main').prepend(staleBanner);
}
const mins = Math.floor(checkAge / 60);
staleBanner.querySelector('.lt-alert-msg').textContent =
`Monitoring data is stale — last check was ${mins} minute${mins !== 1 ? 's' : ''} ago. The monitor daemon may be down.`;
staleBanner.style.display = '';
} else if (staleBanner) {
staleBanner.style.display = 'none';
}
}
}
function updateHostGrid(hosts) {
const scHosts = document.getElementById('stat-hosts-val');
if (scHosts) scHosts.textContent = Object.keys(hosts).length;
for (const [name, host] of Object.entries(hosts)) {
const card = document.querySelector(`.host-card[data-host="${CSS.escape(name)}"]`);
if (!card) continue;
card.className = card.className.replace(/host-card-(up|down|degraded|unknown)/g, '');
card.classList.add(`host-card-${host.status}`);
const dot = card.querySelector('.host-status-dot');
if (dot) dot.className = `host-status-dot dot-${host.status}`;
const ifaceList = card.querySelector('.iface-list');
if (ifaceList && host.interfaces && Object.keys(host.interfaces).length > 0) {
ifaceList.innerHTML = Object.entries(host.interfaces)
.sort(([a], [b]) => a.localeCompare(b))
.map(([iface, state]) => `
<div class="iface-row">
<span class="iface-dot dot-${state}"></span>
<span class="iface-name">${lt.escHtml(iface)}</span>
<span class="iface-state state-${state}">${state}</span>
</div>
`).join('');
}
}
}
function updateTopology(hosts) {
document.querySelectorAll('.topo-host').forEach(node => {
const name = node.dataset.host;
const host = hosts[name];
if (!host) return;
node.className = node.className.replace(/topo-v2-status-(up|down|degraded|unknown)/g, '');
node.classList.add(`topo-v2-status-${host.status}`);
const badge = node.querySelector('.topo-badge');
if (badge) {
badge.className = `topo-badge topo-badge-${host.status}`;
badge.textContent = host.status;
}
// Animate the 10G drop-wire red+dashed when host is down
document.querySelectorAll(`.topo-v2-wire-10g[data-host="${CSS.escape(name)}"]`).forEach(wire => {
wire.classList.toggle('wire-down', host.status === 'down');
});
});
}
function updateUnifiTable(devices) {
const tbody = document.querySelector('#unifi-table tbody');
if (!tbody || !devices.length) return;
tbody.innerHTML = devices.map(d => {
const statusClass = d.connected ? '' : 'row-critical';
const dotClass = d.connected ? 'dot-up' : 'dot-down';
const statusText = d.connected ? 'ONLINE' : 'OFFLINE';
const suppressBtn = !d.connected
? `<button class="lt-btn lt-btn-ghost lt-btn-sm btn-suppress"
data-sup-type="unifi_device"
data-sup-name="${lt.escHtml(d.name)}"
data-sup-detail=""
aria-label="Suppress alerts for ${lt.escHtml(d.name)}">🔕 Suppress</button>`
: '';
return `
<tr class="${statusClass}">
<td><span class="${dotClass}"></span> ${statusText}</td>
<td><strong>${lt.escHtml(d.name)}</strong></td>
<td>${lt.escHtml(d.type)}</td>
<td>${lt.escHtml(d.model)}</td>
<td>${lt.escHtml(d.ip)}</td>
<td>${suppressBtn}</td>
</tr>`;
}).join('');
}
function updateEventsTable(events, totalActive) {
const wrap = document.getElementById('events-table-wrap');
if (!wrap) return;
const active = (events || []).filter(e => e.severity !== 'info');
if (!active.length) {
wrap.innerHTML = '<div class="lt-empty-state lt-empty-state--sm"><div class="lt-empty-state-icon">✔</div><div class="lt-empty-state-title">No active alerts</div></div>';
return;
}
const truncated = totalActive !== null && totalActive !== undefined && totalActive > active.length;
const countNotice = truncated
? `<div class="pagination-notice">Showing ${active.length} of ${totalActive} active alerts &mdash; <a href="/api/events?limit=1000">view all via API</a></div>`
: '';
const rows = active.map(e => {
const supType = e.event_type === 'unifi_device_offline' ? 'unifi_device'
: e.event_type === 'interface_down' ? 'interface'
: 'host';
const ticketBase = (typeof GANDALF_CONFIG !== 'undefined' && GANDALF_CONFIG.ticket_web_url)
? GANDALF_CONFIG.ticket_web_url : 'http://t.lotusguild.org/ticket/';
const ticket = e.ticket_id
? `<a href="${lt.escHtml(ticketBase)}${lt.escHtml(String(e.ticket_id))}" target="_blank"
class="ticket-link">#${lt.escHtml(String(e.ticket_id))}</a>`
: '–';
const supBadge = e.is_suppressed
? `<span class="lt-badge badge-suppressed" title="Alert suppressed">🔕 sup</span>`
: '';
return `
<tr class="row-${e.severity}${e.is_suppressed ? ' row-suppressed' : ''}">
<td><span class="lt-badge badge-${e.severity}">${e.severity}</span>${supBadge}</td>
<td>${lt.escHtml(e.event_type.replace(/_/g,' '))}</td>
<td><strong>${lt.escHtml(e.target_name)}</strong></td>
<td>${lt.escHtml(e.target_detail || '–')}</td>
<td class="desc-cell" title="${lt.escHtml(e.description || '')}">${lt.escHtml((e.description||'').substring(0,60))}${(e.description||'').length>60?'…':''}</td>
<td class="ts-cell" title="${lt.escHtml(e.first_seen||'')}">${lt.time.ago(_toIso(e.first_seen))}</td>
<td class="ts-cell" title="${lt.escHtml(e.last_seen||'')}">${lt.time.ago(_toIso(e.last_seen))}</td>
<td>${e.consecutive_failures}</td>
<td>${ticket}</td>
<td>
<button class="lt-btn lt-btn-ghost lt-btn-sm btn-suppress"
data-sup-type="${lt.escHtml(supType)}"
data-sup-name="${lt.escHtml(e.target_name)}"
data-sup-detail="${lt.escHtml(e.target_detail||'')}"
aria-label="Suppress alert for ${lt.escHtml(e.target_name)}">🔕</button>
</td>
</tr>`;
}).join('');
wrap.innerHTML = `
${countNotice}
<div class="lt-table-wrap">
<table class="lt-table" id="events-table">
<caption class="lt-sr-only">Active network alerts</caption>
<thead>
<tr>
<th>Sev</th><th>Type</th><th>Target</th><th>Detail</th>
<th>Description</th><th>First Seen</th><th>Last Seen</th><th>Failures</th><th>Ticket</th><th>Actions</th>
</tr>
</thead>
<tbody>${rows}</tbody>
</table>
</div>`;
}
// ── Suppression modal ─────────────────────────────────────────────────
function openSuppressModal(type, name, detail) {
const modal = document.getElementById('suppress-modal');
if (!modal) return;
document.getElementById('sup-type').value = type;
document.getElementById('sup-name').value = name;
document.getElementById('sup-detail').value = detail;
document.getElementById('sup-reason').value = '';
document.getElementById('sup-expires').value = '';
updateSuppressForm();
lt.modal.open('suppress-modal');
document.querySelectorAll('#suppress-modal .pill').forEach(p => {
p.classList.remove('active');
p.setAttribute('aria-pressed', 'false');
});
const manualPill = document.querySelector('#suppress-modal .pill-manual');
if (manualPill) { manualPill.classList.add('active'); manualPill.setAttribute('aria-pressed', 'true'); }
const hint = document.getElementById('duration-hint');
if (hint) hint.textContent = 'Suppression will persist until manually removed.';
}
function closeSuppressModal() {
lt.modal.close('suppress-modal');
}
function updateSuppressForm() {
const type = document.getElementById('sup-type').value;
const nameGrp = document.getElementById('sup-name-group');
const detailGrp = document.getElementById('sup-detail-group');
const nameInput = document.getElementById('sup-name');
const detailInput = document.getElementById('sup-detail');
if (nameGrp) nameGrp.style.display = (type === 'all') ? 'none' : '';
if (detailGrp) detailGrp.style.display = (type === 'interface') ? '' : 'none';
if (nameInput) {
const req = (type !== 'all');
nameInput.required = req;
nameInput.setAttribute('aria-required', String(req));
}
if (detailInput) {
const req = (type === 'interface');
detailInput.required = req;
detailInput.setAttribute('aria-required', String(req));
}
}
function setDuration(mins, el, opts) {
const o = opts || {};
const expiresEl = document.getElementById(o.expiresId || 'sup-expires');
const pillSel = o.pillSel || '#suppress-modal .pill';
const hint = document.getElementById(o.hintId || 'duration-hint');
if (expiresEl) expiresEl.value = mins || '';
document.querySelectorAll(pillSel).forEach(p => {
p.classList.remove('active');
p.setAttribute('aria-pressed', 'false');
});
if (el) { el.classList.add('active'); el.setAttribute('aria-pressed', 'true'); }
if (hint) {
if (mins) {
const h = Math.floor(mins / 60), m = mins % 60;
hint.textContent = `Expires in ${h ? h + 'h ' : ''}${m ? m + 'm' : ''}.`;
} else {
hint.textContent = 'Suppression will persist until manually removed.';
}
}
}
async function submitSuppress(e) {
e.preventDefault();
const type = document.getElementById('sup-type').value;
const name = document.getElementById('sup-name').value;
const detail = document.getElementById('sup-detail').value;
const reason = document.getElementById('sup-reason').value;
const expires = document.getElementById('sup-expires').value;
if (!reason.trim()) { showToast('Reason is required', 'error'); return; }
if (type !== 'all' && !name.trim()) { showToast('Target name is required', 'error'); return; }
try {
await lt.api.post('/api/suppressions', {
target_type: type,
target_name: name,
target_detail: detail,
reason,
expires_minutes: expires ? parseInt(expires) : null,
});
closeSuppressModal();
showToast('Suppression applied ✔', 'success');
setTimeout(refreshAll, 500);
} catch (err) {
showToast(err.message || 'Failed to apply suppression', 'error');
}
}
// ── Suppress form – wired here so the modal works from any page ──────
document.getElementById('suppress-form')?.addEventListener('submit', submitSuppress);
document.getElementById('sup-type')?.addEventListener('change', updateSuppressForm);
// ── Global click delegation ───────────────────────────────────────────
document.addEventListener('click', e => {
// Refresh button
if (e.target.closest('[data-action="refresh"]')) {
lt.autoRefresh.now();
return;
}
// Duration pills (data-duration="" = manual/forever)
const pill = e.target.closest('.pill[data-duration]');
if (pill) {
const val = pill.dataset.duration;
setDuration(val ? parseInt(val) : null, pill);
return;
}
// Suppress buttons
const btn = e.target.closest('.btn-suppress[data-sup-type]');
if (btn) {
openSuppressModal(
btn.dataset.supType || '',
btn.dataset.supName || '',
btn.dataset.supDetail || '',
);
}
});
+5614
View File
File diff suppressed because it is too large. Load diff
+2947
View File
File diff suppressed because it is too large. Load diff
+1066
View File
File diff suppressed because it is too large. Load diff
+521
View File
@@ -0,0 +1,521 @@
<!DOCTYPE html>
<html lang="en" data-theme="dark">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
<meta name="theme-color" content="#030508">
<meta name="robots" content="noindex, nofollow">
<title>{% block title %}GANDALF{% endblock %} — GANDALF</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=JetBrains+Mono:ital,wght@0,400;0,600;0,700;1,400&family=VT323&display=swap" rel="stylesheet">
<link rel="stylesheet" href="{{ url_for('static', filename='base.css') }}">
<link rel="stylesheet" href="{{ url_for('static', filename='style.css') }}">
<!-- base.js loaded in head so lt.* is available for inline scripts -->
<script src="{{ url_for('static', filename='base.js') }}"></script>
</head>
<body>
<a class="lt-skip-link" href="#main-content">Skip to main content</a>
<!-- BOOT OVERLAY -->
<div id="lt-boot" class="lt-boot-overlay" data-app-name="GANDALF" style="display:none" aria-hidden="true">
<pre id="lt-boot-text" class="lt-boot-text"></pre>
</div>
<!-- MOBILE NAV DRAWER -->
<div id="lt-nav-drawer" class="lt-nav-drawer" aria-hidden="true" role="dialog" aria-modal="true" aria-label="Navigation menu">
<div class="lt-nav-drawer-header">
<span class="lt-brand-title">GANDALF</span>
<button type="button" class="lt-nav-drawer-close" id="lt-nav-drawer-close" aria-label="Close navigation">&#x2715;</button>
</div>
<nav class="lt-nav-drawer-links" aria-label="Mobile navigation">
<a href="{{ url_for('index') }}"
class="lt-nav-drawer-link{% if request.endpoint == 'index' %} active{% endif %}"
{% if request.endpoint == 'index' %}aria-current="page"{% endif %}>Dashboard</a>
<a href="{{ url_for('links_page') }}"
class="lt-nav-drawer-link{% if request.endpoint == 'links_page' %} active{% endif %}"
{% if request.endpoint == 'links_page' %}aria-current="page"{% endif %}>Link Debug</a>
<a href="{{ url_for('inspector') }}"
class="lt-nav-drawer-link{% if request.endpoint == 'inspector' %} active{% endif %}"
{% if request.endpoint == 'inspector' %}aria-current="page"{% endif %}>Inspector</a>
{% if user.groups and 'admin' in user.groups %}
<a href="{{ url_for('suppressions_page') }}"
class="lt-nav-drawer-link{% if request.endpoint == 'suppressions_page' %} active{% endif %}"
{% if request.endpoint == 'suppressions_page' %}aria-current="page"{% endif %}>Suppressions</a>
{% endif %}
</nav>
</div>
<div id="lt-nav-overlay" class="lt-nav-drawer-overlay"></div>
<!-- PRIMARY HEADER -->
<header class="lt-header" role="banner">
<div class="lt-header-left">
<!-- Hamburger (mobile) -->
<button type="button"
class="lt-menu-btn"
id="lt-menu-btn"
data-action="open-nav-drawer"
aria-label="Open navigation menu"
aria-expanded="false"
aria-controls="lt-nav-drawer">
<span class="lt-menu-btn-bar"></span>
<span class="lt-menu-btn-bar"></span>
<span class="lt-menu-btn-bar"></span>
</button>
<!-- Brand -->
<div class="lt-brand">
<a href="{{ url_for('index') }}"
class="lt-brand-title lt-glitch"
data-text="GANDALF"
aria-label="GANDALF home">GANDALF</a>
<span class="lt-brand-subtitle">Network Monitor // LotusGuild</span>
</div>
<!-- Desktop nav -->
<nav class="lt-nav" aria-label="Main navigation">
<a href="{{ url_for('index') }}"
class="lt-nav-link{% if request.endpoint == 'index' %} active{% endif %}"
{% if request.endpoint == 'index' %}aria-current="page"{% endif %}>
Dashboard
</a>
<a href="{{ url_for('links_page') }}"
class="lt-nav-link{% if request.endpoint == 'links_page' %} active{% endif %}"
{% if request.endpoint == 'links_page' %}aria-current="page"{% endif %}>
Link Debug
</a>
<a href="{{ url_for('inspector') }}"
class="lt-nav-link{% if request.endpoint == 'inspector' %} active{% endif %}"
{% if request.endpoint == 'inspector' %}aria-current="page"{% endif %}>
Inspector
</a>
{% if user.groups and 'admin' in user.groups %}
<a href="{{ url_for('suppressions_page') }}"
class="lt-nav-link{% if request.endpoint == 'suppressions_page' %} active{% endif %}"
{% if request.endpoint == 'suppressions_page' %}aria-current="page"{% endif %}>
Suppressions
</a>
{% endif %}
</nav>
</div>
<div class="lt-header-right">
{% set _uname = user.name or user.username %}
{% set _words = _uname.split() %}
{% set _initials = (_words[0][0] ~ (_words[1][0] if _words|length > 1 else ''))|upper %}
<div class="lt-avatar lt-avatar--sm {{ _uname | avatar_color }}"
aria-hidden="true" title="{{ _uname }}">
<img src="{{ url_for('api_avatar') }}" alt="" class="lt-avatar-img">
<span class="lt-avatar-initials">{{ _initials }}</span>
</div>
<span class="lt-header-user">{{ _uname }}</span>
{% if user.groups and 'admin' in user.groups %}
<span class="lt-badge lt-badge-admin" aria-label="Administrator">ADMIN</span>
{% endif %}
<!-- Notification bell — shows active monitoring alerts -->
<div class="lt-notif-dropdown-wrap" id="lt-notif-wrap">
<button type="button"
class="lt-btn lt-btn-ghost lt-btn-sm lt-notif-bell-btn"
id="lt-notif-bell"
aria-label="Active alerts"
aria-expanded="false"
aria-controls="lt-notif-panel"
title="Active alerts">&#x1F514;</button>
<div class="lt-notif-panel" id="lt-notif-panel" aria-hidden="true" role="dialog" aria-label="Active alerts">
<div class="lt-notif-panel-header">
<span>Active Alerts</span>
<button type="button" class="lt-notif-panel-clear" id="lt-notif-clear-btn">Mark all read</button>
</div>
<div class="lt-notif-panel-list" id="lt-notif-list">
<div class="lt-notif-empty">Loading&hellip;</div>
</div>
<div class="lt-notif-panel-footer">
<a href="{{ url_for('index') }}" class="lt-btn lt-btn-ghost lt-btn-sm lt-notif-view-all">View dashboard</a>
</div>
</div>
</div>
<!-- ⌘K affordance -->
<button type="button"
class="lt-btn lt-btn-ghost lt-btn-sm lt-cmd-hint-btn"
data-action="open-cmdpalette"
title="Command palette (Ctrl+K)"
aria-label="Open command palette">&#x2315;&nbsp;K</button>
<button type="button" class="lt-theme-btn" id="lt-theme-btn"
aria-label="Toggle theme" title="Toggle light/dark mode">&#x2600;</button>
</div>
</header>
<!-- COMMAND PALETTE -->
<div id="lt-cmd-overlay" class="lt-cmd-overlay" role="dialog" aria-modal="true" aria-label="Command palette" aria-hidden="true">
<div class="lt-cmd-palette" id="lt-cmd-palette">
<div class="lt-cmd-input-wrap">
<span class="lt-cmd-prompt">&gt;</span>
<input id="lt-cmd-input" class="lt-cmd-input" type="text"
placeholder="Search commands&hellip;" autocomplete="off"
spellcheck="false" aria-label="Search commands">
</div>
<div class="lt-cmd-results" id="lt-cmd-results">
<div class="lt-cmd-empty">Start typing to search&hellip;</div>
</div>
<div class="lt-cmd-footer">
<span><kbd>&#x2191;</kbd><kbd>&#x2193;</kbd> Navigate</span>
<span><kbd>Enter</kbd> Select</span>
<span><kbd>Esc</kbd> Close</span>
</div>
</div>
</div>
<!-- MAIN CONTENT -->
<main class="lt-main lt-container" id="main-content">
{% block content %}{% endblock %}
</main>
<!-- FOOTER — context-sensitive per page -->
<footer class="lt-footer" role="contentinfo">
<nav class="lt-footer-hints" aria-label="Keyboard shortcuts">
{% if request.endpoint == 'index' %}
<span class="lt-footer-hint"><span class="lt-footer-key">[ R ]</span> REFRESH</span>
<span class="lt-footer-sep">|</span>
<span class="lt-footer-hint"><span class="lt-footer-key">[ S ]</span> SUPPRESS</span>
<span class="lt-footer-sep">|</span>
{% elif request.endpoint in ('links_page', 'inspector') %}
<span class="lt-footer-hint"><span class="lt-footer-key">[ R ]</span> REFRESH</span>
<span class="lt-footer-sep">|</span>
{% endif %}
<button type="button" class="lt-footer-hint" data-action="open-settings"><span class="lt-footer-key">[ * ]</span> CFG</button>
<span class="lt-footer-sep">|</span>
<button type="button" class="lt-footer-hint" data-action="show-keyboard-help"><span class="lt-footer-key">[ ? ]</span> HELP</button>
</nav>
<span>GANDALF &mdash; TDS v1.2</span>
</footer>
<!-- QUICK-SUPPRESS MODAL — available on all pages via [S] shortcut -->
<div id="suppress-modal" class="lt-modal-overlay"
role="dialog" aria-modal="true" aria-labelledby="suppress-modal-title" aria-hidden="true">
<div class="lt-modal">
<div class="lt-modal-header">
<h3 class="lt-modal-title" id="suppress-modal-title">Suppress Alert</h3>
<button type="button" class="lt-modal-close" data-modal-close aria-label="Close">&#x2715;</button>
</div>
<form id="suppress-form">
<div class="lt-modal-body">
<div class="lt-form-group">
<label class="lt-label" for="sup-type">Target Type</label>
<select class="lt-select" id="sup-type" name="target_type">
<option value="host">Host (all interfaces)</option>
<option value="interface">Specific Interface</option>
<option value="unifi_device">UniFi Device</option>
<option value="all">Global Maintenance</option>
</select>
</div>
<div class="lt-form-group" id="sup-name-group">
<label class="lt-label" for="sup-name">Target Name</label>
<input type="text" class="lt-input" id="sup-name" name="target_name" placeholder="e.g. large1">
</div>
<div class="lt-form-group" id="sup-detail-group" style="display:none">
<label class="lt-label" for="sup-detail">Interface <span class="lt-field-hint">(interface type only)</span></label>
<input type="text" class="lt-input" id="sup-detail" name="target_detail" placeholder="e.g. enp35s0">
</div>
<div class="lt-form-group">
<label class="lt-label" for="sup-reason">Reason <span class="required">*</span></label>
<input type="text" class="lt-input" id="sup-reason" name="reason"
placeholder="e.g. Planned switch reboot" required aria-required="true">
</div>
<div class="lt-form-group lt-form-group--last">
<label class="lt-label">Duration</label>
<div class="duration-pills" role="group" aria-label="Select suppression duration">
<button type="button" class="pill" data-duration="30" aria-pressed="false" aria-label="30 minutes">30 min</button>
<button type="button" class="pill" data-duration="60" aria-pressed="false" aria-label="1 hour">1 hr</button>
<button type="button" class="pill" data-duration="240" aria-pressed="false" aria-label="4 hours">4 hr</button>
<button type="button" class="pill" data-duration="480" aria-pressed="false" aria-label="8 hours">8 hr</button>
<button type="button" class="pill pill-manual active" data-duration="" aria-pressed="true" aria-label="Manual, no expiry">Manual &#x221E;</button>
</div>
<input type="hidden" id="sup-expires" name="expires_minutes" value="">
<div class="lt-field-hint" id="duration-hint">Persists until manually removed.</div>
</div>
</div>
<div class="lt-modal-footer">
<button type="button" class="lt-btn lt-btn-secondary" data-modal-close>Cancel</button>
<button type="submit" class="lt-btn lt-btn-primary">Apply</button>
</div>
</form>
</div>
</div>
<!-- KEYBOARD SHORTCUTS MODAL -->
<div id="lt-keys-help" class="lt-modal-overlay" aria-hidden="true">
<div class="lt-modal" role="dialog" aria-modal="true" aria-labelledby="keys-help-title">
<div class="lt-modal-header">
<span class="lt-modal-title" id="keys-help-title">Keyboard Shortcuts</span>
<button type="button" class="lt-modal-close" data-modal-close aria-label="Close">&#x2715;</button>
</div>
<div class="lt-modal-body">
<table class="lt-table">
<thead><tr><th>Shortcut</th><th>Action</th></tr></thead>
<tbody>
<tr><td>Ctrl / &#x2318; + K</td><td>Command palette</td></tr>
<tr><td>R</td><td>Refresh data (Dashboard / Link Debug / Inspector)</td></tr>
<tr><td>S</td><td>Quick-suppress alert (Dashboard)</td></tr>
<tr><td>*</td><td>Open settings</td></tr>
<tr><td>?</td><td>Show this help</td></tr>
<tr><td>ESC</td><td>Close modal / panel</td></tr>
</tbody>
</table>
</div>
<div class="lt-modal-footer">
<button type="button" class="lt-btn" data-modal-close>Close</button>
</div>
</div>
</div>
<!-- SETTINGS MODAL -->
<div id="lt-settings-modal" class="lt-modal-overlay" aria-hidden="true">
<div class="lt-modal" role="dialog" aria-modal="true" aria-labelledby="settings-modal-title">
<div class="lt-modal-header">
<span class="lt-modal-title" id="settings-modal-title">Settings</span>
<button type="button" class="lt-modal-close" data-modal-close aria-label="Close">&#x2715;</button>
</div>
<div class="lt-modal-body">
<div class="lt-form-group">
<label class="lt-label">Auto-refresh interval</label>
<div class="duration-pills" id="settings-refresh-pills" role="group" aria-label="Select auto-refresh interval">
<button type="button" class="pill" data-refresh-interval="15" aria-pressed="false">15 s</button>
<button type="button" class="pill" data-refresh-interval="30" aria-pressed="false">30 s</button>
<button type="button" class="pill" data-refresh-interval="60" aria-pressed="false">1 min</button>
<button type="button" class="pill" data-refresh-interval="300" aria-pressed="false">5 min</button>
<button type="button" class="pill" data-refresh-interval="0" aria-pressed="false">Off</button>
</div>
<div class="lt-field-hint" id="settings-refresh-hint"></div>
</div>
<div class="lt-divider lt-divider--compact"></div>
<div class="lt-kv-grid">
<span class="lt-kv-key">User</span>
<span class="lt-kv-val lt-kv-val--cyan">{{ user.name or user.username }}</span>
{% if user.groups %}
<span class="lt-kv-key">Groups</span>
<span class="lt-kv-val">{{ user.groups | join(', ') }}</span>
{% endif %}
</div>
</div>
<div class="lt-modal-footer">
<button type="button" class="lt-btn" data-modal-close>Close</button>
</div>
</div>
</div>
<script>
const GANDALF_CONFIG = {
ticket_web_url: {{ config.get('ticket_api', {}).get('web_url', 'http://t.lotusguild.org/ticket/') | tojson }}
};
</script>
<script src="{{ url_for('static', filename='app.js') }}"></script>
{% block scripts %}{% endblock %}
<script>
if (window.lt) {
lt.init({ bootName: 'GANDALF' });
// Theme toggle
const themeBtn = document.getElementById('lt-theme-btn');
if (themeBtn) themeBtn.addEventListener('click', function() { lt.theme.toggle(); });
// Command palette
lt.cmdPalette.init([
{ id: 'nav-dashboard', group: 'Navigate', icon: '~', label: 'Dashboard', action: function() { window.location.href = '/'; } },
{ id: 'nav-links', group: 'Navigate', icon: '↗', label: 'Link Debug', action: function() { window.location.href = '/links'; } },
{ id: 'nav-inspector', group: 'Navigate', icon: '⬡', label: 'Inspector', action: function() { window.location.href = '/inspector'; } },
{ id: 'nav-suppressions', group: 'Navigate', icon: '🔕', label: 'Suppressions', action: function() { window.location.href = '/suppressions'; } },
{ id: 'action-refresh', group: 'Actions', icon: '↻', label: 'Refresh Data', kbd: 'R', action: function() { lt.autoRefresh.now(); } },
{ id: 'action-suppress', group: 'Actions', icon: '🔕', label: 'New Suppression', kbd: 'S', action: function() { if (typeof openSuppressModal === 'function') openSuppressModal('host','',''); else window.location.href='/suppressions'; } },
{ id: 'help-shortcuts', group: 'Help', icon: '?', label: 'Keyboard Shortcuts', kbd: '?', action: function() { lt.modal.open('lt-keys-help'); } },
{ id: 'help-settings', group: 'Help', icon: '*', label: 'Settings', kbd: '*', action: function() { lt.modal.open('lt-settings-modal'); } },
{ id: 'help-theme', group: 'Help', icon: '☀', label: 'Toggle Theme', action: function() { lt.theme.toggle(); } },
]);
}
// ── Global footer + key actions ───────────────────────────────────────
document.addEventListener('click', function(e) {
const btn = e.target.closest('[data-action]');
if (!btn) return;
const action = btn.getAttribute('data-action');
if (action === 'open-cmdpalette' && window.lt && lt.cmdPalette) lt.cmdPalette.open();
if (action === 'show-keyboard-help' && window.lt) lt.modal.open('lt-keys-help');
if (action === 'open-settings' && window.lt) lt.modal.open('lt-settings-modal');
});
lt.keys.on('r', function() { lt.autoRefresh.now(); });
lt.keys.on('?', function() { if (window.lt) lt.modal.open('lt-keys-help'); });
lt.keys.on('*', function() { if (window.lt) lt.modal.open('lt-settings-modal'); });
lt.keys.on('s', function() {
if (typeof openSuppressModal === 'function') openSuppressModal('host', '', '');
});
// ── Avatar image error fallback ───────────────────────────────────────
document.addEventListener('error', function(e) {
if (e.target.tagName === 'IMG' && e.target.closest('.lt-avatar')) {
e.target.classList.add('lt-avatar-img-err');
}
}, true);
// ── Settings modal ────────────────────────────────────────────────────
(function() {
const LS_KEY = 'gandalf_settings';
const DEFAULT = { refreshInterval: 30 };
function loadSettings() {
try { return Object.assign({}, DEFAULT, JSON.parse(localStorage.getItem(LS_KEY) || '{}')); }
catch(_) { return Object.assign({}, DEFAULT); }
}
function saveSettings(s) {
try { localStorage.setItem(LS_KEY, JSON.stringify(s)); } catch(_) {}
if (typeof window.onGandalfSettingsChanged === 'function') window.onGandalfSettingsChanged(s);
}
function applyRefreshPillUI(interval) {
document.querySelectorAll('#settings-refresh-pills .pill').forEach(function(p) {
const isActive = parseInt(p.dataset.refreshInterval) === interval;
p.classList.toggle('active', isActive);
p.setAttribute('aria-pressed', isActive ? 'true' : 'false');
});
const hint = document.getElementById('settings-refresh-hint');
if (hint) {
if (interval === 0) hint.textContent = 'Auto-refresh disabled.';
else if (interval < 60) hint.textContent = 'Refreshes every ' + interval + ' seconds.';
else hint.textContent = 'Refreshes every ' + Math.floor(interval/60) + ' minute' + (interval > 60 ? 's' : '') + '.';
}
}
// Init pill UI from saved settings
const _settings = loadSettings();
applyRefreshPillUI(_settings.refreshInterval);
// Expose for pages that need to read it (e.g. index.html for autoRefresh)
window.gandalfSettings = _settings;
document.addEventListener('click', function(e) {
const pill = e.target.closest('#settings-refresh-pills .pill[data-refresh-interval]');
if (!pill) return;
const interval = parseInt(pill.dataset.refreshInterval);
_settings.refreshInterval = interval;
saveSettings(_settings);
applyRefreshPillUI(interval);
});
})();
// ── Notification Bell — shows active monitoring alerts ────────────────
(function() {
const bell = document.getElementById('lt-notif-bell');
const panel = document.getElementById('lt-notif-panel');
const list = document.getElementById('lt-notif-list');
const clearBtn = document.getElementById('lt-notif-clear-btn');
const wrapEl = document.getElementById('lt-notif-wrap');
if (!bell || !panel) return;
let _open = false;
let _lastEvents = [];
const LS_READ_KEY = 'gandalf_notif_read_before';
function getReadBefore() {
try { return parseInt(localStorage.getItem(LS_READ_KEY) || '0'); } catch(_) { return 0; }
}
function setReadBefore(ts) {
try { localStorage.setItem(LS_READ_KEY, String(ts)); } catch(_) {}
}
function esc(s) {
return (window.lt && lt.escHtml) ? lt.escHtml(String(s)) : String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;');
}
function toMs(dateStr) {
if (!dateStr) return 0;
return new Date(dateStr.replace(' UTC','Z').replace(' ','T')).getTime();
}
function fmtAgo(dateStr) {
const diff = Math.floor((Date.now() - toMs(dateStr)) / 1000);
if (diff < 60) return diff + 's ago';
if (diff < 3600) return Math.floor(diff/60) + 'm ago';
if (diff < 86400) return Math.floor(diff/3600) + 'h ago';
return Math.floor(diff/86400) + 'd ago';
}
const SEV_DOT = { critical: 'var(--red)', warning: 'var(--amber)' };
function renderAlerts(events) {
_lastEvents = events || [];
const readBefore = getReadBefore();
const active = _lastEvents.filter(function(e) { return e.severity !== 'info'; });
const unreadCount = active.filter(function(e) { return toMs(e.last_seen) > readBefore; }).length;
lt.notif.set(bell, unreadCount);
if (!active.length) {
list.innerHTML = '<div class="lt-notif-empty">&#x2714; No active alerts</div>';
return;
}
list.innerHTML = active.slice(0, 25).map(function(e) {
const isUnread = toMs(e.last_seen) > readBefore;
const dotColor = SEV_DOT[e.severity] || 'var(--text-muted)';
return '<div class="lt-notif-item' + (isUnread ? ' lt-notif-item--unread' : '') + '">' +
'<div class="lt-notif-dot' + (isUnread ? '' : ' lt-notif-dot--read') + '" style="background:' + dotColor + '"></div>' +
'<div class="lt-notif-item-body">' +
'<div class="lt-notif-item-title">' + esc(e.target_name) + (e.target_detail ? ' &middot; ' + esc(e.target_detail) : '') + '</div>' +
'<div class="lt-notif-item-time">' + esc(e.event_type.replace(/_/g,' ')) + ' &middot; ' + fmtAgo(e.last_seen) + '</div>' +
'</div></div>';
}).join('');
}
function fetchAlerts(andRender) {
fetch('/api/status', { credentials: 'same-origin' })
.then(function(r) { return r.json(); })
.then(function(data) {
const events = data.events || [];
if (andRender) {
renderAlerts(events);
} else {
_lastEvents = events;
const readBefore = getReadBefore();
const active = events.filter(function(e) { return e.severity !== 'info'; });
const unread = active.filter(function(e) { return toMs(e.last_seen) > readBefore; }).length;
lt.notif.set(bell, unread);
}
})
.catch(function() {
if (andRender) list.innerHTML = '<div class="lt-notif-empty">Could not load</div>';
});
}
function openPanel() { _open = true; panel.removeAttribute('aria-hidden'); bell.setAttribute('aria-expanded','true'); fetchAlerts(true); }
function closePanel() { _open = false; panel.setAttribute('aria-hidden','true'); bell.setAttribute('aria-expanded','false'); }
bell.addEventListener('click', function(e) { e.stopPropagation(); _open ? closePanel() : openPanel(); });
if (clearBtn) {
clearBtn.addEventListener('click', function() {
setReadBefore(Date.now());
renderAlerts(_lastEvents);
});
}
document.addEventListener('click', function(e) { if (_open && wrapEl && !wrapEl.contains(e.target)) closePanel(); });
document.addEventListener('keydown', function(e) { if (e.key === 'Escape' && _open) closePanel(); });
// Initial badge load + poll every 60 s
fetchAlerts(false);
setInterval(function() { fetchAlerts(_open); }, 60000);
// Allow refreshAll() to also push fresh events to the bell
window.gandalfNotifUpdate = function(events) { renderAlerts(events); };
})();
</script>
</body>
</html>
+589
View File
@@ -0,0 +1,589 @@
{% extends "base.html" %}
{% block title %}Dashboard – GANDALF{% endblock %}
{% block content %}
<!-- ── Status bar ──────────────────────────────────────────────────── -->
<div class="status-bar">
<div class="status-chips" id="status-chips" aria-live="polite" aria-atomic="true">
{% if not daemon_ok %}
<span class="chip chip-critical">⚠ MONITOR OFFLINE</span>
{% endif %}
{% if summary.critical %}
<span class="chip chip-critical">● {{ summary.critical }} CRITICAL</span>
{% endif %}
{% if summary.warning %}
<span class="chip chip-warning">● {{ summary.warning }} WARNING</span>
{% endif %}
{% if daemon_ok and not summary.critical and not summary.warning %}
<span class="chip chip-ok">✔ ALL SYSTEMS NOMINAL</span>
{% endif %}
</div>
<div class="status-meta">
<span class="last-check" id="last-check" data-tooltip="Last monitor poll timestamp" data-tooltip-pos="bottom">{{ last_check }}</span>
<span class="lt-spinner lt-spinner--sm lt-spinner--cyan" id="refresh-spinner" style="display:none"></span>
<button class="lt-btn lt-btn-ghost lt-btn-sm" data-action="refresh" data-tooltip="Refresh all data now" data-tooltip-pos="bottom">↻ REFRESH</button>
</div>
</div>
<!-- ── Stats summary cards ──────────────────────────────────────────── -->
<div class="lt-stats-grid">
<div class="lt-stat-card{% if summary.critical %} lt-stat-card--alert{% endif %}"
id="stat-critical" role="button" tabindex="0"
data-stat-filter="critical" aria-label="{{ summary.critical or 0 }} critical alerts"
aria-controls="events-table-wrap">
<span class="lt-stat-icon lt-text-red" aria-hidden="true">●</span>
<div class="lt-stat-info">
<span class="lt-stat-value lt-text-red" id="stat-critical-val">{{ summary.critical or 0 }}</span>
<span class="lt-stat-label">Critical</span>
</div>
</div>
<div class="lt-stat-card"
id="stat-warning" role="button" tabindex="0"
data-stat-filter="warning" aria-label="{{ summary.warning or 0 }} warning alerts"
aria-controls="events-table-wrap">
<span class="lt-stat-icon lt-text-amber" aria-hidden="true">●</span>
<div class="lt-stat-info">
<span class="lt-stat-value lt-text-amber" id="stat-warning-val">{{ summary.warning or 0 }}</span>
<span class="lt-stat-label">Warning</span>
</div>
</div>
<div class="lt-stat-card" id="stat-hosts" aria-label="Monitored hosts">
<span class="lt-stat-icon lt-text-cyan" aria-hidden="true">⬡</span>
<div class="lt-stat-info">
<span class="lt-stat-value lt-text-cyan" id="stat-hosts-val">{{ snapshot.hosts | length }}</span>
<span class="lt-stat-label">Hosts</span>
</div>
</div>
<div class="lt-stat-card" id="stat-resolved" aria-label="{{ recent_resolved | length }} alerts resolved in last 24 hours">
<span class="lt-stat-icon lt-text-green" aria-hidden="true">✔</span>
<div class="lt-stat-info">
<span class="lt-stat-value lt-text-green" id="stat-resolved-val">{{ recent_resolved | length }}</span>
<span class="lt-stat-label">Resolved 24h</span>
</div>
</div>
</div>
<!-- ── Active alerts ─────────────────────────────── (above the fold) -->
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">Active Alerts</h2>
<span class="g-section-badge" id="alert-count-badge"
{% if not summary.critical and not summary.warning %}style="display:none"{% endif %}>{{ (summary.critical or 0) + (summary.warning or 0) }}</span>
</div>
<div class="lt-toolbar">
<div class="lt-toolbar-left">
<div class="lt-search">
<input type="search" class="lt-input lt-search-input" id="events-search"
placeholder="Filter by target, type, description…" autocomplete="off"
aria-label="Filter active alerts">
</div>
<div class="sev-pills" role="group" aria-label="Filter by severity">
<button type="button" class="pill active" data-sev="" aria-pressed="true">All</button>
<button type="button" class="pill" data-sev="critical" aria-pressed="false">Critical</button>
<button type="button" class="pill" data-sev="warning" aria-pressed="false">Warning</button>
</div>
</div>
</div>
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
<span class="lt-frame-br">&#x255D;</span>
<div class="lt-section-header">Alert Queue</div>
<div id="events-table-wrap">
{% if events %}
{% if total_active is defined and total_active > events|length %}
<div class="pagination-notice">Showing {{ events|length }} of {{ total_active }} active alerts — use the search box to filter, or <a href="/api/events?limit=1000" target="_blank" rel="noopener">export all as JSON</a></div>
{% endif %}
<div class="lt-table-wrap">
<table class="lt-table" id="events-table">
<caption class="lt-sr-only">Active network alerts</caption>
<thead>
<tr>
<th data-tooltip="Alert severity level" data-tooltip-pos="bottom">Sev</th>
<th>Type</th>
<th>Target</th>
<th>Detail</th>
<th>Description</th>
<th data-tooltip="When this alert was first raised" data-tooltip-pos="bottom">First Seen</th>
<th data-tooltip="Most recent check failure" data-tooltip-pos="bottom">Last Seen</th>
<th data-tooltip="Consecutive check failures since first seen" data-tooltip-pos="bottom">Failures</th>
<th>Ticket</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for e in events %}
{% if e.severity != 'info' %}
<tr class="row-{{ e.severity }}{% if e.is_suppressed %} row-suppressed{% endif %}">
<td>
<span class="lt-badge badge-{{ e.severity }}">{{ e.severity }}</span>
{% if e.is_suppressed %}<span class="lt-badge badge-suppressed" title="Alert suppressed">🔕 sup</span>{% endif %}
</td>
<td>{{ e.event_type | replace('_', ' ') }}</td>
<td><strong>{{ e.target_name }}</strong></td>
<td>{{ e.target_detail or '–' }}</td>
<td class="desc-cell" title="{{ e.description | e }}">{{ e.description | truncate(60) }}</td>
<td class="ts-cell" title="{{ e.first_seen }}">
<span class="event-age" data-ts="{{ e.first_seen }}">{{ e.first_seen }}</span>
</td>
<td class="ts-cell" title="{{ e.last_seen }}">
<span class="event-age" data-ts="{{ e.last_seen }}">{{ e.last_seen }}</span>
</td>
<td>{{ e.consecutive_failures }}</td>
<td>
{% if e.ticket_id %}
<a href="{{ config.ticket_api.web_url }}{{ e.ticket_id }}" target="_blank"
class="ticket-link">#{{ e.ticket_id }}</a>
{% else %}–{% endif %}
</td>
<td>
<button class="lt-btn lt-btn-ghost lt-btn-sm btn-suppress"
data-sup-type="{{ 'unifi_device' if e.event_type == 'unifi_device_offline' else 'interface' if e.event_type == 'interface_down' else 'host' }}"
data-sup-name="{{ e.target_name }}"
data-sup-detail="{{ e.target_detail or '' }}"
title="Suppress" aria-label="Suppress alert for {{ e.target_name }}">🔕</button>
</td>
</tr>
{% endif %}
{% else %}
<tr><td colspan="10">
<div class="lt-empty-state lt-empty-state--sm">
<div class="lt-empty-state-icon">✔</div>
<div class="lt-empty-state-title">No active alerts</div>
</div>
</td></tr>
{% endfor %}
</tbody>
</table>
</div>
{% else %}
<div class="lt-empty-state lt-empty-state--sm">
<div class="lt-empty-state-icon">✔</div>
<div class="lt-empty-state-title">No active alerts</div>
</div>
{% endif %}
</div><!-- /#events-table-wrap -->
</div><!-- /.lt-frame -->
</section>
<!-- ── Network topology + host grid ───────────────────────────────── -->
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">Network Hosts</h2>
<button type="button" class="topo-collapse-btn" id="topo-toggle-btn"
aria-expanded="true" aria-controls="topo-collapsible-wrap">&#x25B4; Collapse</button>
</div>
<div class="topo-collapsible" id="topo-collapsible-wrap">
<div class="topology" id="topology-diagram">
<div class="topo-v2">
{%- set topo_h = snapshot.hosts if snapshot.hosts else {} -%}
<!-- ══════════════════════════════════════════════════════════════
TIER 1: Internet (WAN edge)
══════════════════════════════════════════════════════════ -->
<div class="topo-tier">
<div class="topo-v2-node topo-v2-internet">
<span class="topo-v2-icon">◈</span>
<span class="topo-v2-label">INTERNET</span>
<span class="topo-v2-sub">WAN uplink</span>
</div>
</div>
<!-- WAN wire: cyan → WAN gradient -->
<div class="topo-vc">
<div class="topo-vc-wire topo-vc-wire--wan"></div>
<span class="topo-vc-label">WAN · 10G SFP+</span>
</div>
<!-- ══════════════════════════════════════════════════════════════
TIER 2: Router – UDM-Pro
══════════════════════════════════════════════════════════ -->
<div class="topo-tier">
<div class="topo-v2-node topo-v2-router">
<span class="topo-v2-icon">⬡</span>
<span class="topo-v2-label">UDM-Pro</span>
<span class="topo-v2-sub">Dream Machine Pro</span>
<span class="topo-v2-sub">RU24</span>
</div>
</div>
<!-- UDM-Pro → USW-Agg (10G SFP+) -->
<div class="topo-vc">
<div class="topo-vc-wire topo-vc-wire--10g"></div>
<span class="topo-vc-label">10G SFP+</span>
</div>
<!-- ══════════════════════════════════════════════════════════════
TIER 3: USW-Aggregation
══════════════════════════════════════════════════════════ -->
<div class="topo-tier">
<div class="topo-v2-node topo-v2-switch" id="topo-switch-agg">
<span class="topo-v2-icon">⬡</span>
<span class="topo-v2-label">USW-Agg</span>
<span class="topo-v2-sub">Aggregation · RU22</span>
<span class="topo-v2-sub">8 × 10G SFP+</span>
<span class="topo-v2-vlan">VLAN90 · 10.10.90.x/24</span>
</div>
</div>
<!-- USW-Agg → Pro 24 PoE (10G trunk) -->
<div class="topo-vc">
<div class="topo-vc-wire topo-vc-wire--10g"></div>
<span class="topo-vc-label">10G trunk</span>
</div>
<!-- ══════════════════════════════════════════════════════════════
TIER 4: Pro 24 PoE
══════════════════════════════════════════════════════════ -->
<div class="topo-tier">
<div class="topo-v2-node topo-v2-switch" id="topo-switch-poe">
<span class="topo-v2-icon">⬡</span>
<span class="topo-v2-label">Pro 24 PoE</span>
<span class="topo-v2-sub">24-Port · RU23</span>
<span class="topo-v2-sub">24 × 1G PoE</span>
<span class="topo-v2-vlan">DHCP · mgmt</span>
</div>
</div>
<!-- Pro 24 PoE → host bus section -->
<div class="topo-vc">
<div class="topo-vc-wire topo-vc-wire--mgmt"></div>
</div>
<!-- ══════════════════════════════════════════════════════════════
TIER 4 connecting bus – two rails (10G green + 1G amber dashed)
showing dual-homing for all 6 servers
══════════════════════════════════════════════════════════ -->
<div class="topo-bus-section">
<!-- 10G storage bus (Agg → VLAN90) -->
<div class="topo-bus-10g">
<span class="topo-bus-10g-label">← USW-Agg · 10G SFP+ · VLAN90 →</span>
<div class="topo-bus-10g-line"></div>
</div>
<!-- 1G management bus (PoE → DHCP) -->
<div class="topo-bus-1g">
<span class="topo-bus-1g-label">← Pro 24 PoE · 1G · DHCP mgmt →</span>
<div class="topo-bus-1g-line"></div>
</div>
<!-- ── Host nodes with drop wires ── -->
<div class="topo-v2-hosts">
{%- set all_defs = [
('compute-storage-gpu-01', 'csg-01', 'RU4–12', 'Ceph · VLAN90', False),
('compute-storage-01', 'cs-01', 'RU14–17', 'Ceph · VLAN90', False),
('storage-01', 'sto-01', 'rack', 'Ceph · VLAN90', False),
('monitor-01', 'mon-01', 'ZimaBoard', 'mgmt', False),
('monitor-02', 'mon-02', 'ZimaBoard', 'mgmt', False),
('large1', 'large1', 'off-rack', 'table', True),
] -%}
{%- for hname, hlabel, hsub, hvlan, off_rack in all_defs -%}
{%- set st = topo_h[hname].status if hname in topo_h else 'unknown' -%}
<div class="topo-v2-host-wrap">
<!-- dual-homing wires: 10G solid green + 1G dashed amber -->
<div class="topo-v2-host-wires">
<div class="topo-v2-wire-10g" data-host="{{ hname }}" title="10G SFP+ → USW-Agg"></div>
<div class="topo-v2-wire-1g" data-host="{{ hname }}" title="1G → Pro 24 PoE"></div>
</div>
<!-- host box -->
<div class="topo-v2-node topo-v2-host topo-host topo-v2-status-{{ st }}{{ ' topo-v2-offrack' if off_rack else '' }} topo-v2-host--bus"
data-host="{{ hname }}">
<span class="topo-v2-icon">▣</span>
<span class="topo-v2-label">{{ hlabel }}</span>
<span class="topo-v2-sub">{{ hsub }}</span>
<span class="topo-v2-vlan">{{ hvlan }}</span>
<span class="topo-badge topo-badge-{{ st }}">{{ st if st != 'unknown' else '–' }}</span>
</div>
</div>
{%- endfor -%}
</div>
</div><!-- /topo-bus-section -->
<!-- ── Legend ── -->
<div class="topo-legend">
<div class="topo-legend-item"><span class="topo-legend-line-wan"></span> WAN / uplink</div>
<div class="topo-legend-item"><span class="topo-legend-line-10g"></span> 10G SFP+ (Ceph / VLAN90)</div>
<div class="topo-legend-item"><span class="topo-legend-line-1g"></span> 1G DHCP (mgmt)</div>
<div class="topo-legend-item topo-legend-item--offrack">dashed border = off-rack</div>
</div>
</div><!-- /topo-v2 -->
</div>
<!-- Host cards -->
<div class="lt-toolbar" id="host-toolbar">
<div class="lt-toolbar-left">
<div class="lt-search">
<input type="search" class="lt-input lt-search-input lt-search-input--sm" id="host-search"
placeholder="Filter hosts…" autocomplete="off" aria-label="Filter hosts">
</div>
</div>
</div>
<div class="host-grid" id="host-grid">
{%- set has_global_sup = suppressions | selectattr('target_type', 'equalto', 'all') | list | length > 0 -%}
{% for name, host in snapshot.hosts.items() %}
{% set suppressed = suppressions | selectattr('target_name', 'equalto', name) | list %}
<div class="host-card host-card-{{ host.status }}" data-host="{{ name }}">
<div class="host-card-header">
<div class="host-name-row">
<span class="host-status-dot dot-{{ host.status }}"></span>
<span class="host-name">{{ name }}</span>
{% if suppressed or has_global_sup %}
<span class="badge-suppressed" title="Suppressed">🔕</span>
{% endif %}
</div>
<div class="host-meta">
<span class="host-ip">{{ host.ip }}</span>
<span class="host-source source-{{ host.source }}">{{ host.source }}</span>
</div>
</div>
{% if host.interfaces %}
<div class="iface-list">
{% for iface, state in host.interfaces.items() | sort %}
<div class="iface-row">
<span class="iface-dot dot-{{ state }}"></span>
<span class="iface-name">{{ iface }}</span>
<span class="iface-state state-{{ state }}">{{ state }}</span>
</div>
{% endfor %}
</div>
{% else %}
<div class="host-ping-note">ping-only / no node_exporter</div>
{% endif %}
<div class="host-actions">
<button class="lt-btn lt-btn-ghost lt-btn-sm btn-suppress"
data-sup-type="host"
data-sup-name="{{ name }}"
data-sup-detail=""
aria-label="Suppress alerts for {{ name }}">
🔕 Suppress
</button>
<a href="{{ url_for('links_page') }}#{{ name }}"
class="lt-btn lt-btn-secondary lt-btn-sm">
↗ Links
</a>
</div>
</div>
{% else %}
<div class="lt-empty-state lt-empty-state--sm">
<div class="lt-empty-state-icon">⌛</div>
<div class="lt-empty-state-title">No host data yet</div>
<div class="lt-empty-state-body">The monitor daemon may still be starting up.</div>
</div>
{% endfor %}
</div>
</div><!-- /#topo-collapsible-wrap -->
</section>
<!-- ── UniFi devices ────────────────────────────────────────────────── -->
{% if snapshot.unifi %}
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">UniFi Devices</h2>
</div>
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
<span class="lt-frame-br">&#x255D;</span>
<div class="lt-section-header">Device Inventory</div>
<div class="lt-table-wrap">
<table class="lt-table" id="unifi-table">
<caption class="lt-sr-only">UniFi network devices</caption>
<thead>
<tr>
<th>Status</th>
<th>Name</th>
<th>Type</th>
<th>Model</th>
<th>IP</th>
<th>Actions</th>
</tr>
</thead>
<tbody>
{% for d in snapshot.unifi %}
<tr class="{% if not d.connected %}row-critical{% endif %}">
<td>
<span class="dot-{{ 'up' if d.connected else 'down' }}"></span>
{{ 'ONLINE' if d.connected else 'OFFLINE' }}
</td>
<td><strong>{{ d.name }}</strong></td>
<td>{{ d.type }}</td>
<td>{{ d.model }}</td>
<td>{{ d.ip }}</td>
<td>
{% if not d.connected %}
<button class="lt-btn lt-btn-ghost lt-btn-sm btn-suppress"
data-sup-type="unifi_device"
data-sup-name="{{ d.name }}"
data-sup-detail=""
aria-label="Suppress alerts for {{ d.name }}">
🔕 Suppress
</button>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
</section>
{% endif %}
<!-- ── Recently Resolved (last 24h) ───────────────────────────────── -->
{% if recent_resolved %}
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">Recently Resolved</h2>
<span class="g-section-badge g-section-badge-resolved">{{ recent_resolved | length }} in last 24h</span>
</div>
<div class="lt-timeline">
{% for e in recent_resolved %}
{%- set dot_cls = 'lt-timeline-item--green' if e.severity == 'info' else 'lt-timeline-item--dim' -%}
<div class="lt-timeline-item {{ dot_cls }}">
<div class="lt-timeline-meta">
<strong class="lt-timeline-actor">{{ e.target_name }}</strong>
{% if e.target_detail %}<span>· {{ e.target_detail }}</span>{% endif %}
<span class="lt-timeline-time event-age" data-ts="{{ e.resolved_at }}">{{ e.resolved_at }}</span>
</div>
<div class="lt-timeline-body">
{{ e.event_type | replace('_', ' ') }}
&nbsp;·&nbsp;
<span class="lt-badge badge-resolved">{{ e.severity }}</span>
&nbsp;·&nbsp; duration
<span class="event-duration" data-first="{{ e.first_seen }}" data-resolved="{{ e.resolved_at }}">–</span>
</div>
</div>
{% endfor %}
</div>
</section>
{% endif %}
{% endblock %}
{% block scripts %}
<script>
// Start auto-refresh using saved settings interval (default 30 s)
const _savedInterval = window.gandalfSettings?.refreshInterval ?? 30;
if (_savedInterval > 0) lt.autoRefresh.start(refreshAll, _savedInterval * 1000);
// When settings change, restart auto-refresh with new interval
window.onGandalfSettingsChanged = function(s) {
lt.autoRefresh.stop();
if (s.refreshInterval > 0) lt.autoRefresh.start(refreshAll, s.refreshInterval * 1000);
};
// ── Topology collapse toggle ───────────────────────────────────
(function() {
const LS_KEY = 'gandalf_topo_collapsed';
const btn = document.getElementById('topo-toggle-btn');
const wrap = document.getElementById('topo-collapsible-wrap');
if (!btn || !wrap) return;
function setCollapsed(v) {
wrap.classList.toggle('is-collapsed', v);
wrap.setAttribute('aria-hidden', v ? 'true' : 'false');
btn.setAttribute('aria-expanded', v ? 'false' : 'true');
btn.textContent = v ? '▾ Expand' : '▴ Collapse';
try { localStorage.setItem(LS_KEY, v ? '1' : '0'); } catch(_) {}
}
let saved = false;
try { saved = localStorage.getItem(LS_KEY) === '1'; } catch(_) {}
setCollapsed(saved);
btn.addEventListener('click', function() {
setCollapsed(!wrap.classList.contains('is-collapsed'));
});
})();
function updateEventAges() {
document.querySelectorAll('.event-age[data-ts]').forEach(el => {
el.textContent = lt.time.ago(_toIso(el.dataset.ts));
});
}
updateEventAges();
setInterval(updateEventAges, 60000);
// ── Event duration (resolved_at - first_seen) ──────────────────
function fmtDuration(firstTs, resolvedTs) {
if (!firstTs || !resolvedTs) return '–';
const secs = Math.floor((new Date(_toIso(resolvedTs)) - new Date(_toIso(firstTs))) / 1000);
if (secs < 0) return '–';
if (secs < 60) return `${secs}s`;
if (secs < 3600) return `${Math.floor(secs/60)}m`;
if (secs < 86400) return `${Math.floor(secs/3600)}h ${Math.floor((secs%3600)/60)}m`;
return `${Math.floor(secs/86400)}d`;
}
document.querySelectorAll('.event-duration[data-first][data-resolved]').forEach(el => {
el.textContent = fmtDuration(el.dataset.first, el.dataset.resolved);
});
// ── Events table filter ────────────────────────────────────────
let _filterSev = '';
function applyEventsFilter() {
const q = (document.getElementById('events-search')?.value || '').toLowerCase();
const tbody = document.querySelector('#events-table tbody');
if (!tbody) return;
tbody.querySelectorAll('tr').forEach(row => {
if (row.children.length < 3) { row.style.display = ''; return; }
const sevMatch = !_filterSev || row.classList.contains(`row-${_filterSev}`);
const textMatch = !q || row.textContent.toLowerCase().includes(q);
row.style.display = (sevMatch && textMatch) ? '' : 'none';
});
}
document.getElementById('events-search')?.addEventListener('input', applyEventsFilter);
document.querySelector('.sev-pills')?.addEventListener('click', e => {
const pill = e.target.closest('.pill[data-sev]');
if (!pill) return;
document.querySelectorAll('.sev-pills .pill').forEach(p => {
p.classList.remove('active');
p.setAttribute('aria-pressed', 'false');
});
pill.classList.add('active');
pill.setAttribute('aria-pressed', 'true');
_filterSev = pill.dataset.sev;
applyEventsFilter();
});
// Re-apply filter after dynamic table updates
new MutationObserver(applyEventsFilter)
.observe(document.getElementById('events-table-wrap'), { childList: true, subtree: true });
// Host grid search filter
document.getElementById('host-search')?.addEventListener('input', function() {
const q = this.value.trim().toLowerCase();
document.querySelectorAll('#host-grid .host-card').forEach(card => {
const name = (card.dataset.host || '').toLowerCase();
card.style.display = (!q || name.includes(q)) ? '' : 'none';
});
});
// Stat card clicks — filter events table by severity
document.querySelectorAll('.lt-stat-card[data-stat-filter]').forEach(card => {
card.addEventListener('click', () => {
const sev = card.dataset.statFilter;
document.querySelectorAll('.sev-pills .pill').forEach(p => {
p.classList.remove('active');
p.setAttribute('aria-pressed', 'false');
});
const matchPill = document.querySelector(`.sev-pills .pill[data-sev="${sev}"]`);
if (matchPill) { matchPill.classList.add('active'); matchPill.setAttribute('aria-pressed', 'true'); }
_filterSev = sev;
applyEventsFilter();
document.getElementById('events-table-wrap')?.scrollIntoView({ behavior: 'smooth', block: 'start' });
});
card.addEventListener('keydown', e => { if (e.key === 'Enter' || e.key === ' ') { e.preventDefault(); card.click(); } });
});
</script>
{% endblock %}
+823
View File
@@ -0,0 +1,823 @@
{% extends "base.html" %}
{% block title %}Inspector – GANDALF{% endblock %}
{% block content %}
<div class="lt-page-header">
<div>
<h1 class="lt-page-title">Network Inspector</h1>
<p class="g-page-sub">
Visual switch chassis diagrams. Click a port to see detailed stats and LLDP path debug.
<span id="inspector-updated" class="g-page-sub-aside"></span>
</p>
</div>
</div>
<div class="inspector-layout">
<div class="inspector-main" id="inspector-main" role="region" aria-label="Switch chassis diagrams">
<div class="link-loading">Loading inspector data</div>
</div>
<div class="inspector-panel" id="inspector-panel" role="complementary" aria-label="Port detail panel">
<div class="inspector-panel-inner" id="inspector-panel-inner"></div>
</div>
</div>
{% endblock %}
{% block scripts %}
<script>
const escHtml = s => lt.escHtml(s);
// ── Switch layout config ─────────────────────────────────────────────────
// keys match the model field returned by the UniFi API
// rows: array of rows, each row is an array of port_idx values
// sfp_ports: port_idx values that are SFP cages (in rows, rendered differently)
// sfp_section: port_idx values rendered as a separate SFP bank below the rows
const SWITCH_LAYOUTS = {
'USF5P': { rows: [[1,2,3,4]], sfp_section: [5] },
'USL8A': { rows: [[1,2,3,4],[5,6,7,8]], sfp_ports: [1,2,3,4,5,6,7,8] },
'US24PRO': {
rows: [
[1,3,5,7,9,11,13,15,17,19,21,23],
[2,4,6,8,10,12,14,16,18,20,22,24],
],
sfp_section: [25,26],
},
'USPPDUP': { rows: [[1]] },
'USMINI': { rows: [[1,2,3,4,5]] },
};
// ── Formatting helpers ───────────────────────────────────────────────────
function fmtSpeed(mbps) {
if (!mbps) return '–';
if (mbps >= 1000) return (mbps / 1000).toFixed(0) + 'G';
return mbps + 'M';
}
function fmtRate(bytesPerSec) {
if (bytesPerSec === null || bytesPerSec === undefined) return '–';
const bps = bytesPerSec * 8;
if (bps < 1e3) return bps.toFixed(0) + ' bps';
if (bps < 1e6) return (bps / 1e3).toFixed(1) + ' Kbps';
if (bps < 1e9) return (bps / 1e6).toFixed(2) + ' Mbps';
return (bps / 1e9).toFixed(3) + ' Gbps';
}
function fmtErrors(rate) {
if (rate === null || rate === undefined) return '–';
if (rate < 0.001) return '<span class="val-good">0 /s</span>';
return `<span class="val-crit">${rate.toFixed(3)} /s</span>`;
}
// ── Build port_idx → port data map ──────────────────────────────────────
function buildPortIdxMap(ports) {
const map = {};
for (const [pname, d] of Object.entries(ports)) {
if (d.port_idx != null) {
map[d.port_idx] = Object.assign({ name: pname }, d);
}
}
return map;
}
// ── Determine port block CSS state class ────────────────────────────────
function portBlockState(d) {
if (!d || !d.up) return 'down';
if (d.is_uplink) return 'uplink';
if (d.poe_power != null && d.poe_power > 0) return 'poe-active';
return 'up';
}
// ── Speed label helper ───────────────────────────────────────────────────
function portSpeedLabel(port) {
if (!port || !port.up) return '';
const spd = port.speed; // speed in Mbps from UniFi API
if (!spd) return '';
if (spd >= 10000) return '10G';
if (spd >= 1000) return '1G';
if (spd >= 100) return '100M';
return spd + 'M';
}
// ── Render a single port block element ──────────────────────────────────
function portBlockHtml(idx, port, swName, sfpBlock) {
const state = portBlockState(port);
const numLabel = sfpBlock ? 'SFP' : idx;
const title = port ? escHtml(port.name) : `Port ${idx}`;
const sfpCls = sfpBlock ? ' sfp-block' : '';
const speedTxt = portSpeedLabel(port);
// LLDP neighbor: first 6 chars of hostname
const lldpName = (port && port.lldp && (port.lldp.system_name || port.lldp.chassis_id))
? escHtml((port.lldp.system_name || port.lldp.chassis_id || '').slice(0, 6))
: '';
const lldpHtml = lldpName ? `<span class="port-lldp">${lldpName}</span>` : '';
const speedHtml = speedTxt ? `<span class="port-speed">${speedTxt}</span>` : '';
return `<div class="switch-port-block ${state}${sfpCls}"
data-switch="${escHtml(swName)}" data-port-idx="${idx}"
title="${title}" aria-label="${title}"
data-action="select-port"><span class="port-num">${numLabel}</span>${speedHtml}${lldpHtml}</div>`;
}
// ── Chassis legend HTML ──────────────────────────────────────────────────
function chassisLegendHtml() {
return `<div class="chassis-legend">
<div class="chassis-legend-item"><span class="chassis-legend-swatch cls-down"></span>down</div>
<div class="chassis-legend-item"><span class="chassis-legend-swatch cls-up"></span>up</div>
<div class="chassis-legend-item"><span class="chassis-legend-swatch cls-poe"></span>poe active</div>
<div class="chassis-legend-item"><span class="chassis-legend-swatch cls-uplink"></span>uplink</div>
</div>`;
}
// ── Render one switch chassis ────────────────────────────────────────────
function renderChassis(swName, sw) {
const model = sw.model || '';
const layout = SWITCH_LAYOUTS[model] || null;
const portMap = buildPortIdxMap(sw.ports || {});
const upCount = Object.values(sw.ports || {}).filter(p => p.up).length;
const totCount = Object.keys(sw.ports || {}).length;
const downCount = totCount - upCount;
const meta = [model, `${upCount}/${totCount} up`, downCount ? `${downCount} down` : ''].filter(Boolean).join(' · ');
// Is this a US24PRO? Used to add group-separator class
const isUs24Pro = (model === 'US24PRO');
let chassisHtml = '';
if (layout) {
const sfpPortSet = new Set(layout.sfp_ports || []);
const sfpSectionSet = new Set(layout.sfp_section || []);
// Main port rows
chassisHtml += '<div class="chassis-rows">';
for (const row of layout.rows) {
const rowCls = isUs24Pro ? ' us24pro-row' : '';
chassisHtml += `<div class="chassis-row${rowCls}">`;
for (const idx of row) {
const port = portMap[idx];
const isSfp = sfpPortSet.has(idx);
const sfpCls = isSfp ? ' sfp-port' : '';
const state = portBlockState(port);
const title = port ? escHtml(port.name) : `Port ${idx}`;
const speedTxt = portSpeedLabel(port);
const lldpName = (port && port.lldp && (port.lldp.system_name || port.lldp.chassis_id))
? escHtml((port.lldp.system_name || port.lldp.chassis_id || '').slice(0, 6))
: '';
const speedHtml = speedTxt ? `<span class="port-speed">${speedTxt}</span>` : '';
const lldpHtml = lldpName ? `<span class="port-lldp">${lldpName}</span>` : '';
chassisHtml += `<div class="switch-port-block ${state}${sfpCls}"
data-switch="${escHtml(swName)}" data-port-idx="${idx}"
title="${title}" aria-label="${title}"
data-action="select-port"><span class="port-num">${idx}</span>${speedHtml}${lldpHtml}</div>`;
}
chassisHtml += '</div>';
}
chassisHtml += '</div>';
// Separate SFP section (if present)
if (sfpSectionSet.size) {
chassisHtml += '<div class="chassis-sfp-section">';
for (const idx of layout.sfp_section) {
chassisHtml += portBlockHtml(idx, portMap[idx], swName, true);
}
chassisHtml += '</div>';
}
} else {
// Fallback: render all ports sorted by idx
const allPorts = Object.entries(sw.ports || {})
.sort(([, a], [, b]) => (a.port_idx || 0) - (b.port_idx || 0));
chassisHtml += '<div class="chassis-rows"><div class="chassis-row">';
for (const [pname, d] of allPorts) {
chassisHtml += portBlockHtml(d.port_idx || 0, Object.assign({ name: pname }, d), swName, false);
}
chassisHtml += '</div></div>';
}
return `
<div class="inspector-chassis" id="chassis-${escHtml(swName)}">
<div class="chassis-header">
<span class="chassis-name">${escHtml(swName)}</span>
${sw.ip ? `<span class="chassis-ip">${escHtml(sw.ip)}</span>` : ''}
<span class="chassis-meta">${escHtml(meta)}</span>
</div>
<div class="chassis-body">
<div class="chassis-ear-l"></div>
<div class="chassis-ear-r"></div>
${chassisHtml}
</div>
${chassisLegendHtml()}
</div>`;
}
// ── State ────────────────────────────────────────────────────────────────
let _selectedSwitch = null;
let _selectedIdx = null;
let _apiData = null;
// ── Port selection ───────────────────────────────────────────────────────
function selectPort(el) {
const swName = el.dataset.switch;
const idx = parseInt(el.dataset.portIdx, 10);
if (_diagPollTimer) { clearInterval(_diagPollTimer); _diagPollTimer = null; }
document.querySelectorAll('.switch-port-block.selected')
.forEach(e => e.classList.remove('selected'));
el.classList.add('selected');
_selectedSwitch = swName;
_selectedIdx = idx;
renderPanel(swName, idx);
}
function closePanel() {
if (_diagPollTimer) { clearInterval(_diagPollTimer); _diagPollTimer = null; }
document.getElementById('inspector-panel').classList.remove('open');
document.querySelectorAll('.switch-port-block.selected')
.forEach(el => el.classList.remove('selected'));
_selectedSwitch = null;
_selectedIdx = null;
}
// ── Render detail panel ──────────────────────────────────────────────────
function renderPanel(swName, idx) {
if (!_apiData) return;
const sw = _apiData.unifi_switches && _apiData.unifi_switches[swName];
if (!sw) return;
const portMap = buildPortIdxMap(sw.ports || {});
const d = portMap[idx];
if (!d) return;
const upStr = d.up ? '<span class="val-good">UP</span>' : '<span class="val-crit">DOWN</span>';
const speedStr = d.speed_mbps ? `<span class="val-cyan">${fmtSpeed(d.speed_mbps)}bps</span>` : '–';
const duplexCls = d.full_duplex ? 'val-good' : (d.up ? 'val-warn' : 'val-neutral');
const duplexStr = d.up ? `<span class="${duplexCls}">${d.full_duplex ? 'Full' : 'Half'}</span>` : '–';
const autoneg = d.autoneg ? 'On' : 'Off';
const mediaStr = d.media || '–';
const isUplinkBadge = d.is_uplink ? ' <span class="port-badge port-badge-uplink">UPLINK</span>' : '';
// PoE section
let poeHtml = '';
if (d.poe_class != null) {
const poeMaxStr = d.poe_max_power != null ? ` / max ${d.poe_max_power.toFixed(1)}W` : '';
const poeCurStr = (d.poe_power != null && d.poe_power > 0) ? ` / draw <span class="val-amber">${d.poe_power.toFixed(1)}W</span>` : '';
poeHtml = `
<div class="lt-divider"><span class="lt-divider-label">PoE</span></div>
<div class="panel-row"><span class="panel-label">Class</span><span class="panel-val">class ${escHtml(String(d.poe_class))}${poeMaxStr}</span></div>
${d.poe_power != null ? `<div class="panel-row"><span class="panel-label">Draw</span><span class="panel-val">${d.poe_power > 0 ? `<span class="val-amber">${d.poe_power.toFixed(1)}W</span>` : '0W'}</span></div>` : ''}
${d.poe_mode ? `<div class="panel-row"><span class="panel-label">Mode</span><span class="panel-val">${escHtml(d.poe_mode)}</span></div>` : ''}`;
}
// Traffic section
let trafficHtml = '';
if (d.tx_bytes_rate != null || d.rx_bytes_rate != null) {
trafficHtml = `
<div class="lt-divider"><span class="lt-divider-label">Traffic</span></div>
<div class="panel-row"><span class="panel-label" data-tooltip="Transmit — outgoing from this port">TX</span><span class="panel-val">${fmtRate(d.tx_bytes_rate)}</span></div>
<div class="panel-row"><span class="panel-label" data-tooltip="Receive — incoming to this port">RX</span><span class="panel-val">${fmtRate(d.rx_bytes_rate)}</span></div>`;
}
// Errors / drops section
let errHtml = '';
if (d.tx_errs_rate != null || d.rx_errs_rate != null) {
errHtml = `
<div class="lt-divider"><span class="lt-divider-label">Errors / Drops</span></div>
<div class="panel-row"><span class="panel-label">TX Err</span><span class="panel-val">${fmtErrors(d.tx_errs_rate)}</span></div>
<div class="panel-row"><span class="panel-label">RX Err</span><span class="panel-val">${fmtErrors(d.rx_errs_rate)}</span></div>
<div class="panel-row"><span class="panel-label">TX Drop</span><span class="panel-val">${fmtErrors(d.tx_drops_rate)}</span></div>
<div class="panel-row"><span class="panel-label">RX Drop</span><span class="panel-val">${fmtErrors(d.rx_drops_rate)}</span></div>`;
}
// LLDP + path debug
let lldpHtml = '';
let pathHtml = '';
if (d.lldp && d.lldp.system_name) {
const l = d.lldp;
lldpHtml = `
<div class="lt-divider"><span class="lt-divider-label">LLDP Neighbor</span></div>
<div class="panel-row"><span class="panel-label">System</span><span class="panel-val val-cyan">${escHtml(l.system_name)}</span></div>
${l.port_id ? `<div class="panel-row"><span class="panel-label">Port</span><span class="panel-val">${escHtml(l.port_id)}</span></div>` : ''}
${l.port_desc ? `<div class="panel-row"><span class="panel-label">Port Desc</span><span class="panel-val">${escHtml(l.port_desc)}</span></div>` : ''}
${l.chassis_id ? `<div class="panel-row"><span class="panel-label">Chassis</span><span class="panel-val">${escHtml(l.chassis_id)}</span></div>` : ''}
${l.mgmt_ips && l.mgmt_ips.length ? `<div class="panel-row"><span class="panel-label">Mgmt IP</span><span class="panel-val">${escHtml(l.mgmt_ips.join(', '))}</span></div>` : ''}`;
// Path debug: look for matching server interface
const hosts = _apiData.hosts || {};
const serverIfaces = hosts[l.system_name];
if (serverIfaces) {
let matchedIface = l.port_id && serverIfaces[l.port_id] ? l.port_id : null;
if (!matchedIface && l.port_id) {
// fuzzy match
matchedIface = Object.keys(serverIfaces).find(k => l.port_id.includes(k) || k.includes(l.port_id)) || null;
}
if (matchedIface) {
pathHtml = buildPathDebug(swName, d, l.system_name, matchedIface, serverIfaces[matchedIface]);
}
}
}
// Diagnose button (only when LLDP has an identified neighbor we can map)
const hasDiagTarget = !!(d.lldp && d.lldp.system_name &&
_apiData.hosts && _apiData.hosts[d.lldp.system_name]);
const diagHtml = hasDiagTarget ? `
<div class="diag-bar">
<button class="btn-diag lt-btn lt-btn-secondary lt-btn-sm" data-action="run-diagnostic"
data-sw="${escHtml(swName)}" data-idx="${idx}"
aria-label="Run link diagnostics for port ${idx} on ${escHtml(swName)}">Run Diagnostics</button>
<span class="diag-status" id="diag-status"></span>
</div>
<div class="diag-results" id="diag-results"></div>` : '';
const inner = document.getElementById('inspector-panel-inner');
inner.innerHTML = `
<div class="panel-header">
<div>
<span class="panel-port-name">${escHtml(d.name)}</span>${isUplinkBadge}
<div class="panel-meta">${escHtml(swName)} · port #${idx}</div>
</div>
<button class="panel-close" data-action="close-panel" aria-label="Close panel">✕</button>
</div>
<div class="lt-divider"><span class="lt-divider-label">Link</span></div>
<div class="panel-row"><span class="panel-label">Status</span><span class="panel-val">${upStr}</span></div>
<div class="panel-row"><span class="panel-label">Speed</span><span class="panel-val">${speedStr}</span></div>
<div class="panel-row"><span class="panel-label" data-tooltip="Full = simultaneous send/receive. Half = one direction at a time.">Duplex</span><span class="panel-val">${duplexStr}</span></div>
<div class="panel-row"><span class="panel-label" data-tooltip="Autonegotiation: NIC and switch automatically agree speed and duplex.">Auto-neg</span><span class="panel-val val-neutral">${autoneg}</span></div>
<div class="panel-row"><span class="panel-label">Media</span><span class="panel-val">${escHtml(mediaStr)}</span></div>
${poeHtml}
${trafficHtml}
${errHtml}
${lldpHtml}
${pathHtml}
${diagHtml}
`;
document.getElementById('inspector-panel').classList.add('open');
}
// ── Build path debug two-column section ─────────────────────────────────
function buildPathDebug(swName, swPort, serverName, ifaceName, svrData) {
const isFiber = (swPort.media || '').toLowerCase().includes('sfp') ||
(svrData.port_type || '').toLowerCase().includes('fibre') ||
(svrData.port_type || '').toLowerCase().includes('fiber') ||
!!svrData.sfp;
const connType = isFiber ? 'SFP / Fiber' : 'Copper';
let sfpDomHtml = '';
if (svrData.sfp && Object.keys(svrData.sfp).length) {
const sfp = svrData.sfp;
sfpDomHtml = '<div class="path-dom">';
if (sfp.vendor) sfpDomHtml += `<div class="path-dom-row"><span>Vendor</span><span>${escHtml(sfp.vendor)}${sfp.part_no ? ' / ' + escHtml(sfp.part_no) : ''}</span></div>`;
if (sfp.temp_c != null) sfpDomHtml += `<div class="path-dom-row"><span>Temp</span><span>${sfp.temp_c.toFixed(1)}°C</span></div>`;
if (sfp.tx_power_dbm != null) sfpDomHtml += `<div class="path-dom-row"><span>TX</span><span>${sfp.tx_power_dbm.toFixed(2)} dBm</span></div>`;
if (sfp.rx_power_dbm != null) sfpDomHtml += `<div class="path-dom-row"><span>RX</span><span>${sfp.rx_power_dbm.toFixed(2)} dBm</span></div>`;
sfpDomHtml += '</div>';
}
const svrErrTx = (svrData.tx_errs_rate > 0.001) ? 'val-crit' : 'val-good';
const svrErrRx = (svrData.rx_errs_rate > 0.001) ? 'val-crit' : 'val-good';
const swErrTx = (swPort.tx_errs_rate > 0.001) ? 'val-crit' : 'val-good';
const swErrRx = (swPort.rx_errs_rate > 0.001) ? 'val-crit' : 'val-good';
// Detect duplex mismatch (switch full_duplex vs server duplex string)
const swFull = swPort.full_duplex;
const svrFull = (svrData.duplex || '').toLowerCase().includes('full');
const duplexMismatch = swPort.up && svrData.duplex &&
((swFull && !svrFull) || (!swFull && svrFull));
const duplexWarnHtml = duplexMismatch
? `<div class="path-mismatch-alert">⚠ DUPLEX MISMATCH — Switch: ${swFull ? 'Full' : 'Half'} · Server: ${escHtml(svrData.duplex)}</div>`
: '';
// Detect speed mismatch
const swSpd = swPort.speed_mbps, svrSpd = svrData.speed_mbps;
const speedMismatch = swSpd && svrSpd && swSpd > 0 && svrSpd > 0 && swSpd !== svrSpd;
const speedWarnHtml = speedMismatch
? `<div class="path-mismatch-alert">⚠ SPEED MISMATCH — Switch: ${fmtSpeed(swSpd)} · Server: ${fmtSpeed(svrSpd)}</div>`
: '';
return `
<div class="lt-divider"><span class="lt-divider-label">Path Debug · ${escHtml(connType)}</span></div>
${duplexWarnHtml}${speedWarnHtml}
<div class="path-debug-cols">
<div class="path-col">
<div class="path-col-header">Switch</div>
<div class="path-row"><span>Port</span><span>${escHtml(swPort.name)}</span></div>
<div class="path-row"><span>Speed</span><span>${fmtSpeed(swPort.speed_mbps)}bps</span></div>
<div class="path-row"><span>TX</span><span>${fmtRate(swPort.tx_bytes_rate)}</span></div>
<div class="path-row"><span>RX</span><span>${fmtRate(swPort.rx_bytes_rate)}</span></div>
<div class="path-row"><span>TX Err</span><span class="${swErrTx}">${fmtErrors(swPort.tx_errs_rate)}</span></div>
<div class="path-row"><span>RX Err</span><span class="${swErrRx}">${fmtErrors(swPort.rx_errs_rate)}</span></div>
${(swPort.poe_power != null && swPort.poe_power > 0) ? `<div class="path-row"><span>PoE</span><span class="val-amber">${swPort.poe_power.toFixed(1)}W</span></div>` : ''}
</div>
<div class="path-col">
<div class="path-col-header">Server: ${escHtml(serverName)}</div>
<div class="path-row"><span>Iface</span><span>${escHtml(ifaceName)}</span></div>
<div class="path-row"><span>Speed</span><span>${svrData.speed_mbps ? fmtSpeed(svrData.speed_mbps) + 'bps' : '–'}</span></div>
<div class="path-row"><span>TX</span><span>${fmtRate(svrData.tx_bytes_rate)}</span></div>
<div class="path-row"><span>RX</span><span>${fmtRate(svrData.rx_bytes_rate)}</span></div>
<div class="path-row"><span>TX Err</span><span class="${svrErrTx}">${fmtErrors(svrData.tx_errs_rate)}</span></div>
<div class="path-row"><span>RX Err</span><span class="${svrErrRx}">${fmtErrors(svrData.rx_errs_rate)}</span></div>
${svrData.carrier_changes != null ? `<div class="path-row"><span>Carrier Chg</span><span class="${(svrData.carrier_changes||0)>10?'val-crit':(svrData.carrier_changes||0)>2?'val-warn':'val-good'}">${svrData.carrier_changes}</span></div>` : ''}
${sfpDomHtml}
</div>
</div>`;
}
// ── Render all switches ──────────────────────────────────────────────────
function renderInspector(data) {
_apiData = data;
const main = document.getElementById('inspector-main');
const switches = data.unifi_switches || {};
const updEl = document.getElementById('inspector-updated');
if (updEl && data.updated) {
const updMs = new Date(_toIso(data.updated));
const ageMin = (Date.now() - updMs) / 60000;
const timeStr = updMs.toLocaleTimeString();
if (ageMin > 15) {
updEl.innerHTML = `<span class="g-stale-warn" title="Data is ${Math.floor(ageMin)} minutes old — monitor may be down">⚠ Stale: ${timeStr}</span>`;
} else {
updEl.textContent = 'Updated: ' + timeStr;
}
}
if (!Object.keys(switches).length) {
main.innerHTML = '<div class="lt-empty-state lt-empty-state--sm"><div class="lt-empty-state-icon">⬡</div><div class="lt-empty-state-title">No switch data available</div><div class="lt-empty-state-body">Monitor may still be initialising.</div></div>';
return;
}
main.innerHTML = Object.entries(switches)
.map(([swName, sw]) => renderChassis(swName, sw))
.join('');
// Re-apply selection highlight after re-render (dataset compare — no CSS escaping)
if (_selectedSwitch && _selectedIdx !== null) {
const block = Array.from(document.querySelectorAll('.switch-port-block')).find(
el => el.dataset.switch === _selectedSwitch && parseInt(el.dataset.portIdx, 10) === _selectedIdx
);
if (block) {
block.classList.add('selected');
renderPanel(_selectedSwitch, _selectedIdx);
}
}
}
// ── Fetch and render ─────────────────────────────────────────────────────
async function loadInspector() {
try {
const data = await lt.api.get('/api/links');
renderInspector(data);
} catch (e) {
document.getElementById('inspector-main').innerHTML =
'<div class="lt-empty-state lt-empty-state--sm"><div class="lt-empty-state-icon">✕</div><div class="lt-empty-state-title">Failed to load inspector data</div></div>';
lt.toast.error('Failed to load inspector data');
}
}
loadInspector();
const _inspInterval = window.gandalfSettings?.refreshInterval ?? 60;
if (_inspInterval > 0) lt.autoRefresh.start(loadInspector, Math.max(_inspInterval, 15) * 1000);
window.onGandalfSettingsChanged = function(s) {
lt.autoRefresh.stop();
if (s.refreshInterval > 0) lt.autoRefresh.start(loadInspector, Math.max(s.refreshInterval, 15) * 1000);
};
lt.keys.on('Escape', () => {
if (document.getElementById('inspector-panel').classList.contains('open')) closePanel();
});
document.addEventListener('click', e => {
const portBlock = e.target.closest('[data-action="select-port"]');
if (portBlock) { selectPort(portBlock); return; }
const closeBtn = e.target.closest('[data-action="close-panel"]');
if (closeBtn) { closePanel(); return; }
const diagBtn = e.target.closest('[data-action="run-diagnostic"]');
if (diagBtn) { runDiagnostic(diagBtn.dataset.sw, parseInt(diagBtn.dataset.idx, 10)); return; }
const toggleDiag = e.target.closest('[data-action="toggle-diag"]');
if (toggleDiag) {
const section = toggleDiag.parentElement;
const nowOpen = section.classList.toggle('diag-open');
const hint = toggleDiag.querySelector('.diag-toggle-hint');
if (hint) hint.textContent = nowOpen ? '[collapse]' : '[expand]';
return;
}
});
// ── Link Diagnostics ─────────────────────────────────────────────────
let _diagPollTimer = null;
function runDiagnostic(swName, portIdx) {
const statusEl = document.getElementById('diag-status');
const resultsEl = document.getElementById('diag-results');
if (!statusEl || !resultsEl) return;
// Clear any previous poll
if (_diagPollTimer) { clearInterval(_diagPollTimer); _diagPollTimer = null; }
statusEl.textContent = 'Submitting to Pulse...';
resultsEl.innerHTML = '';
lt.api.post('/api/diagnose', {switch_name: swName, port_idx: portIdx})
.then(resp => {
statusEl.textContent = 'Collecting diagnostics via Pulse...';
pollDiagnostic(resp.job_id, statusEl, resultsEl);
})
.catch(e => {
const msg = (e && e.status === 429)
? 'Rate limit reached — max 5 diagnostics per minute. Please wait.'
: 'Error: ' + (e && e.message || 'Request failed');
statusEl.textContent = msg;
});
}
function pollDiagnostic(jobId, statusEl, resultsEl) {
let attempts = 0;
_diagPollTimer = setInterval(() => {
attempts++;
if (attempts > 120) { // 2min timeout
clearInterval(_diagPollTimer);
_diagPollTimer = null;
statusEl.innerHTML = 'Timed out waiting for results. '
+ '<button class="lt-btn lt-btn-ghost lt-btn-sm" id="diag-retry-btn">Retry</button>';
document.getElementById('diag-retry-btn')?.addEventListener('click', () => {
const sel = document.querySelector('.switch-port-block.selected');
if (sel) runDiagnostic(sel.dataset.switch, parseInt(sel.dataset.portIdx));
});
return;
}
lt.api.get(`/api/diagnose/${jobId}`)
.then(resp => {
if (resp.status === 'done') {
clearInterval(_diagPollTimer);
_diagPollTimer = null;
statusEl.textContent = '';
renderDiagnosticResults(resp.result, resultsEl);
}
})
.catch(() => {
clearInterval(_diagPollTimer);
_diagPollTimer = null;
statusEl.innerHTML = 'Error: lost connection while collecting diagnostics. '
+ '<button class="lt-btn lt-btn-ghost lt-btn-sm" id="diag-retry-btn">Retry</button>';
document.getElementById('diag-retry-btn')?.addEventListener('click', () => {
const sel = document.querySelector('.switch-port-block.selected');
if (sel) runDiagnostic(sel.dataset.switch, parseInt(sel.dataset.portIdx));
});
});
}, 2000);
}
function renderDiagnosticResults(d, container) {
if (!d || d.status === 'error') {
container.innerHTML = `<div class="diag-error">Diagnostic error: ${escHtml((d && d.error) || 'unknown')}</div>`;
return;
}
const health = d.health || {};
const issues = health.issues || [];
const warns = health.warnings || [];
const infoArr = health.info || [];
const secs = d.sections || {};
const eth = secs.ethtool || {};
const drv = secs.ethtool_driver || {};
const pause = secs.ethtool_pause || {};
const ring = secs.ethtool_ring || {};
const dom = secs.ethtool_dom || {};
const sysfs = secs.sysfs_stats || {};
const dmesg = secs.dmesg || [];
const lldpctl = secs.lldpctl || {};
const nicStats = secs.ethtool_stats || {};
const swPort = d.switch_port || {};
// ── Health banner ──
let bannerHtml = '';
if (issues.length === 0 && warns.length === 0) {
bannerHtml = '<div class="diag-health-banner"><span class="diag-health-ok">ALL OK</span></div>';
} else {
const parts = [];
if (issues.length) parts.push(`<span class="diag-health-critical">${issues.length} CRITICAL</span>`);
if (warns.length) parts.push(`<span class="diag-health-warning">${warns.length} WARNING</span>`);
bannerHtml = `<div class="diag-health-banner">${parts.join(' ')}</div>`;
}
const issueRows = [...issues, ...warns, ...infoArr].map(item => {
const cls = issues.includes(item) ? 'diag-val-bad' : warns.includes(item) ? 'diag-val-warn' : 'diag-val-good';
const label = issues.includes(item) ? 'CRIT' : warns.includes(item) ? 'WARN' : 'INFO';
return `<div class="diag-issue-row"><span class="${cls}">[${label}]</span> <span class="diag-code">${escHtml(item.code)}</span> — ${escHtml(item.message)}</div>`;
}).join('');
// ── Physical layer ──
const carrierVal = secs.carrier === '1' ? '<span class="diag-val-good">YES</span>' :
secs.carrier === '0' ? '<span class="diag-val-bad">NO</span>' : '–';
const operstateVal = (secs.operstate || '?').toUpperCase();
const opstateCls = secs.operstate === 'up' ? 'diag-val-good' : secs.operstate === 'down' ? 'diag-val-bad' : 'diag-val-warn';
const speedVal = eth.speed_mbps ? `<span class="diag-val-good">${fmtSpeed(eth.speed_mbps)}bps</span>` : '<span class="diag-val-warn">–</span>';
const duplexVal = eth.duplex === 'full' ? '<span class="diag-val-good">Full</span>' :
eth.duplex === 'half' ? '<span class="diag-val-bad">Half</span>' : '–';
const linkDetVal = eth.link_detected === true ? '<span class="diag-val-good">Yes</span>' :
eth.link_detected === false ? '<span class="diag-val-bad">No</span>' : '–';
const autonegVal = eth.auto_neg === true ? '<span class="diag-val-good">On</span>' :
eth.auto_neg === false ? '<span class="diag-val-warn">Off</span>' : '–';
const physHtml = `
<div class="diag-section">
<div class="diag-section-header">Physical Layer</div>
<table class="diag-table">
<tr><td>Carrier</td><td>${carrierVal}</td></tr>
<tr><td>Oper State</td><td><span class="${opstateCls}">${escHtml(operstateVal)}</span></td></tr>
<tr><td>Speed</td><td>${speedVal}</td></tr>
<tr><td>Duplex</td><td>${duplexVal}</td></tr>
<tr><td>Link Detected</td><td>${linkDetVal}</td></tr>
<tr><td>Auto-neg</td><td>${autonegVal}</td></tr>
${secs.carrier_changes != null ? `<tr><td>Carrier Changes</td><td><span class="${secs.carrier_changes > 20 ? 'diag-val-warn' : 'diag-val-good'}">${secs.carrier_changes}</span></td></tr>` : ''}
</table>
</div>`;
// ── SFP / DOM ──
let domHtml = '';
if (dom && Object.keys(dom).length > 0) {
const rxBar = dom.rx_power_dbm != null ? renderPowerBar(dom.rx_power_dbm, -18, -25) : '';
const txBar = dom.tx_power_dbm != null ? renderPowerBar(dom.tx_power_dbm, -10, -13) : '';
domHtml = `
<div class="diag-section">
<div class="diag-section-header">SFP / DOM</div>
<table class="diag-table">
${dom.vendor ? `<tr><td>Vendor</td><td>${escHtml(dom.vendor)}${dom.part_no ? ' / ' + escHtml(dom.part_no) : ''}</td></tr>` : ''}
${dom.sfp_type ? `<tr><td>Type</td><td>${escHtml(dom.sfp_type)}</td></tr>` : ''}
${dom.connector ? `<tr><td>Connector</td><td>${escHtml(dom.connector)}</td></tr>` : ''}
${dom.wavelength_nm != null ? `<tr><td>Wavelength</td><td>${dom.wavelength_nm} nm</td></tr>` : ''}
${dom.temp_c != null ? `<tr><td>Temperature</td><td>${dom.temp_c.toFixed(1)} °C</td></tr>` : ''}
${dom.voltage_v != null ? `<tr><td>Voltage</td><td>${dom.voltage_v.toFixed(4)} V</td></tr>` : ''}
${dom.bias_ma != null ? `<tr><td>Bias Current</td><td>${dom.bias_ma.toFixed(3)} mA</td></tr>` : ''}
${dom.tx_power_dbm != null ? `<tr><td>TX Power</td><td>${dom.tx_power_dbm.toFixed(2)} dBm ${txBar}</td></tr>` : ''}
${dom.rx_power_dbm != null ? `<tr><td>RX Power</td><td>${dom.rx_power_dbm.toFixed(2)} dBm ${rxBar}</td></tr>` : ''}
</table>
</div>`;
}
// ── NIC Error Counters ──
const errCounters = ['rx_crc_errors','rx_frame_errors','collisions','tx_carrier_errors','rx_missed_errors','rx_fifo_errors'];
const nonZeroCounters = errCounters.filter(k => sysfs[k] > 0);
let errCounterHtml = '';
if (nonZeroCounters.length > 0 || secs.carrier_changes > 0) {
const rows = nonZeroCounters.map(k => {
const v = sysfs[k];
const cls = v > 100 ? 'diag-val-bad' : 'diag-val-warn';
return `<tr><td>${escHtml(k)}</td><td class="${cls}">${v.toLocaleString()}</td></tr>`;
}).join('');
errCounterHtml = `
<div class="diag-section">
<div class="diag-section-header">NIC Error Counters</div>
<table class="diag-table">
${rows || '<tr><td colspan="2" class="diag-val-good">All zero</td></tr>'}
</table>
</div>`;
}
// ── ethtool -S (collapsible) ──
let nicStatHtml = '';
if (Object.keys(nicStats).length > 0) {
const _ERR_KEYS = /err|drop|miss|crc|frame|fifo|abort|carrier|collision|fault|discard|overflow|reset/i;
const rows = Object.entries(nicStats).map(([k, v]) => {
const cls = _ERR_KEYS.test(k) && v > 0 ? ' class="diag-stat-nonzero-warn"' : '';
return `<tr${cls}><td>${escHtml(k)}</td><td>${v.toLocaleString()}</td></tr>`;
}).join('');
nicStatHtml = `
<div class="diag-section diag-collapsible">
<div class="diag-section-header diag-toggle" data-action="toggle-diag">
ethtool -S (NIC stats) <span class="diag-toggle-hint">[expand]</span>
</div>
<div class="diag-section-body">
<table class="diag-stat-table">${rows}</table>
</div>
</div>`;
}
// ── Flow Control + Ring Buffers ──
let flowRingHtml = '';
const hasPause = Object.keys(pause).length > 0;
const hasRing = Object.keys(ring).length > 0;
if (hasPause || hasRing) {
flowRingHtml = `
<div class="diag-section">
<div class="diag-section-header">Flow Control &amp; Ring Buffers</div>
<table class="diag-table">
${hasPause ? `
<tr><td>RX Pause</td><td>${pause.rx_pause ? '<span class="diag-val-good">On</span>' : 'Off'}</td></tr>
<tr><td>TX Pause</td><td>${pause.tx_pause ? '<span class="diag-val-good">On</span>' : 'Off'}</td></tr>` : ''}
${hasRing ? `
<tr><td>RX Ring</td><td>${ring.rx_current != null ? ring.rx_current : '–'} / ${ring.rx_max != null ? ring.rx_max : '–'} max</td></tr>
<tr><td>TX Ring</td><td>${ring.tx_current != null ? ring.tx_current : '–'} / ${ring.tx_max != null ? ring.tx_max : '–'} max</td></tr>` : ''}
</table>
</div>`;
}
// ── Driver Info ──
let drvHtml = '';
if (Object.keys(drv).length > 0) {
drvHtml = `
<div class="diag-section">
<div class="diag-section-header">Driver Info</div>
<table class="diag-table">
${drv.driver ? `<tr><td>Driver</td><td>${escHtml(drv.driver)}</td></tr>` : ''}
${drv.version ? `<tr><td>Version</td><td>${escHtml(drv.version)}</td></tr>` : ''}
${drv.firmware_version ? `<tr><td>Firmware</td><td>${escHtml(drv.firmware_version)}</td></tr>` : ''}
${drv.bus_info ? `<tr><td>Bus</td><td>${escHtml(drv.bus_info)}</td></tr>` : ''}
</table>
</div>`;
}
// ── LLDP Validation ──
let lldpValHtml = '';
const swLldp = swPort.lldp || {};
lldpValHtml = `
<div class="diag-section">
<div class="diag-section-header">LLDP Validation</div>
<div class="path-debug-cols">
<div class="path-col">
<div class="path-col-header">Switch sees</div>
<div class="path-row"><span>System</span><span>${escHtml(swLldp.system_name || '–')}</span></div>
<div class="path-row"><span>Port</span><span>${escHtml(swLldp.port_id || '–')}</span></div>
<div class="path-row"><span>Chassis</span><span>${escHtml(swLldp.chassis_id || '–')}</span></div>
</div>
<div class="path-col">
<div class="path-col-header">Server lldpctl</div>
${lldpctl.available
? `<div class="path-row"><span>Neighbor</span><span>${escHtml(lldpctl.neighbor_system || '–')}</span></div>
<div class="path-row"><span>Port</span><span>${escHtml(lldpctl.neighbor_port || '–')}</span></div>`
: '<div class="path-row"><span class="diag-val-warn">lldpd not running</span></div>'}
</div>
</div>
</div>`;
// ── dmesg ──
let dmesgHtml = '';
if (dmesg.length > 0) {
const dlines = dmesg.map(e => {
const cls = e.severity === 'error' ? ' diag-dmesg-err' : e.severity === 'warn' ? ' diag-dmesg-warn' : '';
const ts = e.timestamp ? `[${e.timestamp}] ` : '';
return `<div class="diag-dmesg-line${cls}">${escHtml(ts + e.msg)}</div>`;
}).join('');
dmesgHtml = `
<div class="diag-section diag-collapsible">
<div class="diag-section-header diag-toggle" data-action="toggle-diag">
Kernel Events (dmesg) <span class="diag-toggle-hint">[expand]</span>
</div>
<div class="diag-section-body">
<div class="diag-dmesg-wrap">${dlines}</div>
</div>
</div>`;
}
// ── Switch Port Summary ──
const swSummaryHtml = `
<div class="diag-section">
<div class="diag-section-header">Switch Port Summary</div>
<table class="diag-table">
<tr><td>Status</td><td>${swPort.up ? '<span class="diag-val-good">UP</span>' : '<span class="diag-val-bad">DOWN</span>'}</td></tr>
<tr><td>Speed</td><td>${swPort.speed_mbps ? fmtSpeed(swPort.speed_mbps) + 'bps' : '–'}</td></tr>
<tr><td>Duplex</td><td>${swPort.full_duplex ? 'Full' : (swPort.up ? '<span class="diag-val-bad">Half</span>' : '–')}</td></tr>
<tr><td>TX Err</td><td>${fmtErrors(swPort.tx_errs_rate)}</td></tr>
<tr><td>RX Err</td><td>${fmtErrors(swPort.rx_errs_rate)}</td></tr>
${swPort.poe_power != null ? `<tr><td>PoE</td><td><span class="val-amber">${swPort.poe_power.toFixed(1)}W</span></td></tr>` : ''}
</table>
</div>`;
// ── Pulse link ──
const pulseLink = d.pulse_url
? `<div class="diag-pulse-link"><a href="${escHtml(d.pulse_url)}" target="_blank" rel="noopener">View raw output in Pulse ↗</a></div>`
: '';
container.innerHTML = `
<div class="diag-results-inner">
${bannerHtml}
<div class="diag-issue-list">${issueRows}</div>
${physHtml}
${domHtml}
${errCounterHtml}
${nicStatHtml}
${flowRingHtml}
${drvHtml}
${lldpValHtml}
${dmesgHtml}
${swSummaryHtml}
${pulseLink}
</div>`;
}
// SFP power bar: range is 0 dBm (best) to -35 dBm (worst)
function renderPowerBar(dbm, warnThreshold, critThreshold) {
const minDbm = -35, maxDbm = 0;
const pct = Math.max(0, Math.min(100, ((dbm - minDbm) / (maxDbm - minDbm)) * 100));
const warnPct = ((warnThreshold - minDbm) / (maxDbm - minDbm)) * 100;
const critPct = ((critThreshold - minDbm) / (maxDbm - minDbm)) * 100;
const barCls = dbm < critThreshold ? 'diag-val-bad' : dbm < warnThreshold ? 'diag-val-warn' : 'diag-val-good';
return `<span class="diag-power-bar-wrap">
<span class="diag-power-bar ${barCls}" style="width:${pct.toFixed(1)}%"></span>
<span class="diag-power-zone-warn" style="left:${warnPct.toFixed(1)}%"></span>
<span class="diag-power-zone-crit" style="left:${critPct.toFixed(1)}%"></span>
</span>`;
}
</script>
{% endblock %}
+598
View File
@@ -0,0 +1,598 @@
{% extends "base.html" %}
{% block title %}Link Debug – GANDALF{% endblock %}
{% block content %}
<div class="lt-page-header">
<div>
<h1 class="lt-page-title">Link Debug</h1>
<p class="g-page-sub">
Per-interface stats: speed, duplex, SFP optical levels, TX/RX rates, errors, and carrier changes.
<span id="links-updated" class="g-page-sub-aside"></span>
</p>
</div>
</div>
<div class="lt-toolbar" id="links-toolbar" style="display:none">
<div class="lt-toolbar-left">
<div class="lt-search">
<input type="search" class="lt-input lt-search-input" id="links-search"
placeholder="Filter by host or switch name…" autocomplete="off"
aria-label="Filter by host or switch name">
</div>
</div>
<div class="lt-toolbar-right">
<button class="lt-btn lt-btn-ghost lt-btn-sm" data-action="collapse-all">Collapse All</button>
<button class="lt-btn lt-btn-ghost lt-btn-sm" data-action="expand-all">Expand All</button>
</div>
</div>
<div id="links-container">
<div class="link-loading">Loading link statistics</div>
</div>
{% endblock %}
{% block scripts %}
<script>
const escHtml = s => lt.escHtml(s);
// ── Formatting helpers ────────────────────────────────────────────
function fmtRate(bytesPerSec) {
if (bytesPerSec === null || bytesPerSec === undefined) return '–';
const bps = bytesPerSec * 8;
if (bps < 1e3) return bps.toFixed(0) + ' bps';
if (bps < 1e6) return (bps/1e3).toFixed(1) + ' Kbps';
if (bps < 1e9) return (bps/1e6).toFixed(2) + ' Mbps';
return (bps/1e9).toFixed(3) + ' Gbps';
}
function fmtRateBar(bytesPerSec, linkSpeedMbps) {
if (!linkSpeedMbps || linkSpeedMbps <= 0) return 0;
const mbps = (bytesPerSec * 8) / 1e6;
return Math.min(100, (mbps / linkSpeedMbps) * 100);
}
function trafficBarClass(pct, isTx) {
if (pct > 85) return 'lt-progress--red';
if (pct > 65) return 'lt-progress--amber';
return isTx ? '' : 'lt-progress--cyan';
}
function fmtSpeed(mbps) {
if (mbps === null || mbps === undefined) return '–';
if (mbps >= 1000) return (mbps/1000).toFixed(0) + ' Gbps';
return mbps + ' Mbps';
}
function fmtDuplex(d) {
if (!d) return '–';
return d.charAt(0).toUpperCase() + d.slice(1);
}
function fmtTemp(c) {
if (c === null || c === undefined) return '–';
return c.toFixed(1) + ' °C';
}
function fmtVoltage(v) {
if (v === null || v === undefined) return '–';
return v.toFixed(2) + ' V';
}
function fmtPower(dbm) {
if (dbm === null || dbm === undefined) return '–';
return dbm.toFixed(2) + ' dBm';
}
function fmtBias(ma) {
if (ma === null || ma === undefined) return '–';
return ma.toFixed(2) + ' mA';
}
function fmtErrors(rate) {
if (rate === null || rate === undefined) return '–';
if (rate < 0.001) return '<span class="val-good">0 /s</span>';
return `<span class="val-crit">${rate.toFixed(3)} /s</span>`;
}
function fmtCarrier(n) {
if (n === null || n === undefined) return '–';
if (n === 0) return '<span class="counter-zero">0</span>';
return `<span class="counter-nonzero">${n}</span>`;
}
// ── SFP/DOM value classification ─────────────────────────────────
function rxPowerClass(dbm) {
if (dbm === null || dbm === undefined) return 'val-neutral';
if (dbm < -15) return 'val-crit';
if (dbm < -10) return 'val-warn';
return 'val-good';
}
function txPowerClass(dbm) {
if (dbm === null || dbm === undefined) return 'val-neutral';
if (dbm < -5) return 'val-crit';
return 'val-good';
}
function tempClass(c) {
if (c === null || c === undefined) return 'val-neutral';
if (c > 80) return 'val-crit';
if (c > 70) return 'val-warn';
return 'val-good';
}
function voltageClass(v) {
if (v === null || v === undefined) return 'val-neutral';
if (v < 3.0 || v > 3.6) return 'val-crit';
if (v < 3.1 || v > 3.5) return 'val-warn';
return 'val-good';
}
function errorBadges(d) {
const badges = [];
if ((d.tx_errs_rate || 0) > 0.001 || (d.rx_errs_rate || 0) > 0.001)
badges.push('<span class="link-alert-badge">ERR</span>');
if ((d.tx_drops_rate || 0) > 0.001 || (d.rx_drops_rate || 0) > 0.001)
badges.push('<span class="link-alert-badge link-alert-amber">DROP</span>');
if ((d.carrier_changes || 0) > 3)
badges.push('<span class="link-alert-badge link-alert-amber">FLAP</span>');
return badges.join('');
}
// ── Render a single server interface card ─────────────────────────
function renderIfaceCard(ifaceName, d) {
const isDown = d.link_detected === false;
const pt = (d.port_type || '').toUpperCase();
const mediaTag = pt === 'FIBRE' || pt === 'SFP' || pt.includes('FIBRE') ? 'type-fibre'
: pt === 'DA' ? 'type-da'
: 'type-copper';
const mediaLabel = d.port_type || '–';
const speedStr = d.speed_mbps ? fmtSpeed(d.speed_mbps) : '–';
const txPct = fmtRateBar(d.tx_bytes_rate, d.speed_mbps);
const rxPct = fmtRateBar(d.rx_bytes_rate, d.speed_mbps);
let sfpHtml = '';
if (d.sfp && Object.keys(d.sfp).length > 0) {
const s = d.sfp;
const rxClass = rxPowerClass(s.rx_power_dbm);
const txClass = txPowerClass(s.tx_power_dbm);
const tmpClass = tempClass(s.temp_c);
const vClass = voltageClass(s.voltage_v);
const rxPct2 = s.rx_power_dbm != null ? Math.min(100, Math.max(0, (s.rx_power_dbm + 20) / 15 * 100)) : 0;
const txPct2 = s.tx_power_dbm != null ? Math.min(100, Math.max(0, (s.tx_power_dbm + 10) / 8 * 100)) : 0;
sfpHtml = `
<div class="sfp-panel">
<div class="sfp-vendor-row">
${s.vendor ? `<span>${escHtml(s.vendor)}</span>` : ''}
${s.part_no ? ` / <span>${escHtml(s.part_no)}</span>` : ''}
</div>
<div class="sfp-grid">
<div class="sfp-stat">
<span class="sfp-stat-label" data-tooltip="SFP module temperature. Normal: below 70°C. Warn: 70–85°C. Critical: above 85°C.">Temp</span>
<span class="sfp-stat-value ${tmpClass}">${fmtTemp(s.temp_c)}</span>
</div>
<div class="sfp-stat">
<span class="sfp-stat-label" data-tooltip="SFP supply voltage. Normal: 3.1–3.5V.">Voltage</span>
<span class="sfp-stat-value ${vClass}">${fmtVoltage(s.voltage_v)}</span>
</div>
<div class="sfp-stat">
<span class="sfp-stat-label" data-tooltip="Laser bias current in mA. High values may indicate end-of-life laser diode.">Bias</span>
<span class="sfp-stat-value">${fmtBias(s.bias_ma)}</span>
</div>
<div class="sfp-stat">
<span class="sfp-stat-label" data-tooltip="Optical transmit power in dBm. Typical good range: -3 to -9 dBm.">TX Power</span>
<span class="sfp-stat-value ${txClass}">${fmtPower(s.tx_power_dbm)}</span>
<div class="power-row">
<div class="power-track"><div class="power-fill ${txClass === 'val-good' ? 'power-ok' : txClass === 'val-warn' ? 'power-warn' : 'power-crit'}" style="width:${txPct2}%"></div></div>
</div>
</div>
<div class="sfp-stat">
<span class="sfp-stat-label" data-tooltip="Optical receive power in dBm. Typical good range: -3 to -18 dBm. Below -20 dBm may indicate dirty/damaged fiber.">RX Power</span>
<span class="sfp-stat-value ${rxClass}">${fmtPower(s.rx_power_dbm)}</span>
<div class="power-row">
<div class="power-track"><div class="power-fill ${rxClass === 'val-good' ? 'power-ok' : rxClass === 'val-warn' ? 'power-warn' : 'power-crit'}" style="width:${rxPct2}%"></div></div>
</div>
</div>
${s.rx_power_dbm != null && s.tx_power_dbm != null ? `
<div class="sfp-stat">
<span class="sfp-stat-label" data-tooltip="Insertion loss: difference between transmit and receive power. Large negative values indicate fiber loss or connector issues.">RX−TX Δ</span>
<span class="sfp-stat-value">${(s.rx_power_dbm - s.tx_power_dbm).toFixed(2)} dBm</span>
</div>` : ''}
</div>
</div>`;
}
return `
<div class="link-iface-card ${isDown ? 'port-down' : ''}">
<div class="link-iface-header">
<span class="link-iface-name">${escHtml(ifaceName)}</span>
<span class="link-iface-speed">${speedStr}</span>
<span class="link-iface-type ${mediaTag}">${escHtml(mediaLabel)}</span>
${errorBadges(d)}
</div>
<div class="link-stats-grid">
<div class="link-stat">
<span class="link-stat-label">Link</span>
<span class="link-stat-value ${isDown ? 'val-crit' : 'val-good'}">${isDown ? 'DOWN' : 'UP'}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Full = simultaneous send/receive at full speed. Half = one direction at a time, can cause collisions.">Duplex</span>
<span class="link-stat-value">${fmtDuplex(d.duplex)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Autonegotiation: NIC and switch automatically agree on link speed and duplex mode.">Auto-neg</span>
<span class="link-stat-value">${d.auto_neg == null ? '–' : d.auto_neg ? 'On' : 'Off'}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Carrier changes: number of times the link went up or down. High values indicate a flapping or unstable cable/SFP.">Carrier Δ</span>
<span class="link-stat-value">${fmtCarrier(d.carrier_changes)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Transmit errors per second reported by the kernel network driver.">TX Err/s</span>
<span class="link-stat-value">${fmtErrors(d.tx_errs_rate)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Receive errors per second reported by the kernel network driver.">RX Err/s</span>
<span class="link-stat-value">${fmtErrors(d.rx_errs_rate)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Transmit packets dropped per second (ring buffer full or driver overrun).">TX Drop/s</span>
<span class="link-stat-value">${fmtErrors(d.tx_drops_rate)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label" data-tooltip="Receive packets dropped per second (ring buffer full or driver overrun).">RX Drop/s</span>
<span class="link-stat-value">${fmtErrors(d.rx_drops_rate)}</span>
</div>
</div>
<div class="traffic-section">
<div class="traffic-row">
<span class="traffic-label" data-tooltip="Transmit — outgoing traffic from this server">TX</span>
<div class="lt-progress ${trafficBarClass(txPct, true)}"><div class="lt-progress-bar" style="width:${txPct}%"></div></div>
<span class="traffic-value">${fmtRate(d.tx_bytes_rate)}</span>
</div>
<div class="traffic-row">
<span class="traffic-label" data-tooltip="Receive — incoming traffic to this server">RX</span>
<div class="lt-progress ${trafficBarClass(rxPct, false)}"><div class="lt-progress-bar" style="width:${rxPct}%"></div></div>
<span class="traffic-value">${fmtRate(d.rx_bytes_rate)}</span>
</div>
</div>
${sfpHtml}
</div>`;
}
// ── Render a single UniFi switch port card ────────────────────────
function renderPortCard(portName, d) {
const isDown = !d.up;
const speedStr = d.speed_mbps ? fmtSpeed(d.speed_mbps) : '–';
const txPct = fmtRateBar(d.tx_bytes_rate, d.speed_mbps);
const rxPct = fmtRateBar(d.rx_bytes_rate, d.speed_mbps);
const numBadge = d.port_idx != null ? `<span class="port-badge port-badge-num">#${d.port_idx}</span>` : '';
const uplinkBadge = d.is_uplink ? `<span class="port-badge port-badge-uplink">UPLINK</span>` : '';
const poeBadge = d.poe_power ? `<span class="port-badge port-badge-poe">PoE ${d.poe_power.toFixed(1)}W</span>` : '';
const lldpLine = d.lldp ? `<div class="port-lldp">→ ${escHtml(d.lldp.system_name || '')} (${escHtml(d.lldp.port_id || '')})</div>` : '';
const poeLine = d.poe_class ? `<div class="port-poe-info">PoE ${escHtml(d.poe_class)} · max ${d.poe_max_power != null ? d.poe_max_power.toFixed(1)+'W' : '–'}</div>` : '';
return `
<div class="link-iface-card ${isDown ? 'port-down' : ''}">
<div class="link-iface-header">
<span class="link-iface-name">${numBadge} ${escHtml(portName)}</span>
<span class="link-iface-speed">${speedStr}</span>
${uplinkBadge}${poeBadge}
${errorBadges(d)}
</div>
${lldpLine}${poeLine}
<div class="link-stats-grid">
<div class="link-stat">
<span class="link-stat-label">Link</span>
<span class="link-stat-value ${isDown ? 'val-crit' : 'val-good'}">${isDown ? 'DOWN' : 'UP'}</span>
</div>
<div class="link-stat">
<span class="link-stat-label">Duplex</span>
<span class="link-stat-value">${d.full_duplex == null ? '–' : d.full_duplex ? 'Full' : 'Half'}</span>
</div>
<div class="link-stat">
<span class="link-stat-label">Auto-neg</span>
<span class="link-stat-value">${d.autoneg == null ? '–' : d.autoneg ? 'On' : 'Off'}</span>
</div>
<div class="link-stat">
<span class="link-stat-label">TX Err/s</span>
<span class="link-stat-value">${fmtErrors(d.tx_errs_rate)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label">RX Err/s</span>
<span class="link-stat-value">${fmtErrors(d.rx_errs_rate)}</span>
</div>
<div class="link-stat">
<span class="link-stat-label">TX Drop/s</span>
<span class="link-stat-value">${fmtErrors(d.tx_drops_rate)}</span>
</div>
</div>
<div class="traffic-section">
<div class="traffic-row">
<span class="traffic-label">TX</span>
<div class="lt-progress ${trafficBarClass(txPct, true)}"><div class="lt-progress-bar" style="width:${txPct}%"></div></div>
<span class="traffic-value">${fmtRate(d.tx_bytes_rate)}</span>
</div>
<div class="traffic-row">
<span class="traffic-label">RX</span>
<div class="lt-progress ${trafficBarClass(rxPct, false)}"><div class="lt-progress-bar" style="width:${rxPct}%"></div></div>
<span class="traffic-value">${fmtRate(d.rx_bytes_rate)}</span>
</div>
</div>
</div>`;
}
// ── Render all UniFi switches ─────────────────────────────────────
function renderUnifiSwitches(unifiSwitches, dataUpdated) {
if (!unifiSwitches || !Object.keys(unifiSwitches).length) return '';
const updStr = dataUpdated
? new Date(_toIso(dataUpdated)).toLocaleTimeString()
: '';
const html = Object.entries(unifiSwitches).map(([swName, sw]) => {
const ports = sw.ports || {};
const portValues = Object.values(ports);
const portCards = Object.entries(ports)
.sort(([,a],[,b]) => (a.port_idx||0) - (b.port_idx||0))
.map(([pname, d]) => renderPortCard(pname, d)).join('');
const poe_total_w = portValues.reduce((s, p) => s + (p.poe_power || 0), 0);
const poe_max_w = portValues.reduce((s, p) => s + (p.poe_max_power || 0), 0);
const poeLoad = poe_total_w > 0 ? ` · PoE ${poe_total_w.toFixed(1)}W` : '';
// PoE utilisation bar
let poebar = '';
if (poe_total_w > 0 && poe_max_w > 0) {
const pct = Math.min(100, (poe_total_w / poe_max_w) * 100);
const cls = pct > 80 ? 'poe-bar-crit' : pct > 60 ? 'poe-bar-warn' : 'poe-bar-ok';
poebar = `<div class="poe-bar-track"><div class="poe-bar-fill ${cls}" style="width:${pct}%"></div></div>`;
}
return `
<div class="link-host-panel" id="panel-${CSS.escape(swName)}">
<div class="link-host-title" data-action="toggle-panel" role="button" tabindex="0" aria-expanded="true">
<span class="link-host-name">${escHtml(swName)}</span>
<span class="link-host-ip">${escHtml(sw.ip || '')}</span>
<span class="link-host-upd">${escHtml(sw.model || '')}${updStr ? ' · ' + updStr : ''}${poeLoad}</span>
${poebar}
<span class="panel-toggle">[–]</span>
</div>
<div class="link-ifaces-grid">${portCards}</div>
</div>`;
}).join('');
return `<div class="lt-divider lt-divider--unifi"><span class="lt-divider-label lt-divider-label--unifi">UNIFI SWITCH PORTS</span></div>${html}`;
}
// ── Panel collapse / expand ───────────────────────────────────────
function togglePanel(panel) {
panel.classList.toggle('collapsed');
const isCollapsed = panel.classList.contains('collapsed');
const btn = panel.querySelector('.panel-toggle');
const title = panel.querySelector('.link-host-title');
if (btn) btn.textContent = isCollapsed ? '[+]' : '[–]';
if (title) title.setAttribute('aria-expanded', isCollapsed ? 'false' : 'true');
const id = panel.id;
if (id) {
let collapsed = {};
try { collapsed = JSON.parse(sessionStorage.getItem('linksCollapsed') || '{}'); } catch(_) {}
collapsed[id] = panel.classList.contains('collapsed');
try { sessionStorage.setItem('linksCollapsed', JSON.stringify(collapsed)); } catch(_) {}
}
}
function restoreCollapseState() {
let collapsed = {};
try { collapsed = JSON.parse(sessionStorage.getItem('linksCollapsed') || '{}'); } catch(_) {}
for (const [id, isCollapsed] of Object.entries(collapsed)) {
const panel = document.getElementById(id);
if (!panel) continue;
if (isCollapsed) {
panel.classList.add('collapsed');
const btn = panel.querySelector('.panel-toggle');
const title = panel.querySelector('.link-host-title');
if (btn) btn.textContent = '[+]';
if (title) title.setAttribute('aria-expanded', 'false');
}
}
}
// ── Build summary stats header ────────────────────────────────────
function buildLinkSummary(hosts, unifiSwitches) {
let totalIfaces = 0, downIfaces = 0, errIfaces = 0, totalPoe = 0;
for (const ifaces of Object.values(hosts || {})) {
for (const d of Object.values(ifaces)) {
totalIfaces++;
if (d.link_detected === false) downIfaces++;
if ((d.tx_errs_rate || 0) > 0 || (d.rx_errs_rate || 0) > 0) errIfaces++;
}
}
let swTotal = 0, swDown = 0;
for (const sw of Object.values(unifiSwitches || {})) {
for (const p of Object.values(sw.ports || {})) {
totalPoe += p.poe_power || 0;
swTotal++;
if (!p.up) swDown++;
}
}
const allTotal = totalIfaces + swTotal;
const allDown = downIfaces + swDown;
const downCls = allDown > 0 ? 'lt-text-red' : 'lt-text-green';
const errCls = errIfaces > 0 ? 'lt-text-amber' : 'lt-text-green';
const downCardCls = allDown > 0 ? ' lt-stat-card--alert' : '';
const poeCard = totalPoe > 0 ? `
<div class="lt-stat-card">
<span class="lt-stat-icon lt-text-amber" aria-hidden="true">⚡</span>
<div class="lt-stat-info">
<span class="lt-stat-value lt-text-amber">${totalPoe.toFixed(1)}</span>
<span class="lt-stat-label">PoE Load (W)</span>
</div>
</div>` : '';
return `
<div class="lt-stats-grid lt-stats-grid--mb">
<div class="lt-stat-card">
<span class="lt-stat-icon lt-text-cyan" aria-hidden="true">⬡</span>
<div class="lt-stat-info">
<span class="lt-stat-value lt-text-cyan">${allTotal}</span>
<span class="lt-stat-label">Interfaces</span>
</div>
</div>
<div class="lt-stat-card${downCardCls}">
<span class="lt-stat-icon ${downCls}" aria-hidden="true">●</span>
<div class="lt-stat-info">
<span class="lt-stat-value ${downCls}">${allDown}</span>
<span class="lt-stat-label">Ports Down</span>
</div>
</div>
<div class="lt-stat-card">
<span class="lt-stat-icon ${errCls}" aria-hidden="true">▲</span>
<div class="lt-stat-info">
<span class="lt-stat-value ${errCls}">${errIfaces}</span>
<span class="lt-stat-label">With Errors</span>
</div>
</div>
${poeCard}
</div>`;
}
// ── Main render ───────────────────────────────────────────────────
function renderLinks(data) {
const hosts = data.hosts || {};
const unifiSwitches = data.unifi_switches || {};
const parts = [];
parts.push(buildLinkSummary(hosts, unifiSwitches));
parts.push('<div class="link-host-list">');
for (const [hostname, ifaces] of Object.entries(hosts)) {
const ifaceCards = Object.entries(ifaces)
.sort(([a],[b]) => a.localeCompare(b))
.map(([iname, d]) => renderIfaceCard(iname, d)).join('');
const sample = Object.values(ifaces)[0] || {};
const ip = sample.host_ip || '';
const updStr = data.updated
? new Date(_toIso(data.updated)).toLocaleTimeString()
: '';
parts.push(`
<div class="link-host-panel" id="panel-${CSS.escape(hostname)}">
<div class="link-host-title" data-action="toggle-panel" role="button" tabindex="0" aria-expanded="true">
<span class="link-host-name">${escHtml(hostname)}</span>
<span class="link-host-ip">${escHtml(ip)}</span>
<span class="link-host-upd">${updStr}</span>
<span class="panel-toggle">[–]</span>
</div>
<div class="link-ifaces-grid">${ifaceCards}</div>
</div>`);
}
parts.push(renderUnifiSwitches(unifiSwitches, data.updated));
parts.push('</div>');
document.getElementById('links-container').innerHTML = parts.join('');
restoreCollapseState();
document.getElementById('links-toolbar').style.display = '';
applyLinksSearch();
}
// ── Host/switch search filter ─────────────────────────────────────
function applyLinksSearch() {
const q = (document.getElementById('links-search')?.value || '').trim().toLowerCase();
document.querySelectorAll('.link-host-panel').forEach(panel => {
const text = (panel.querySelector('.link-host-name')?.textContent || '').toLowerCase();
panel.style.display = (!q || text.includes(q)) ? '' : 'none';
});
}
function collapseAll() {
document.querySelectorAll('.link-host-panel').forEach(p => {
p.classList.add('collapsed');
const btn = p.querySelector('.panel-toggle');
const title = p.querySelector('.link-host-title');
if (btn) btn.textContent = '[+]';
if (title) title.setAttribute('aria-expanded', 'false');
});
try {
sessionStorage.setItem('linksCollapsed', JSON.stringify(
Object.fromEntries([...document.querySelectorAll('.link-host-panel')].map(p => [p.id, true]))
));
} catch(_) {}
}
function expandAll() {
document.querySelectorAll('.link-host-panel').forEach(p => {
p.classList.remove('collapsed');
const btn = p.querySelector('.panel-toggle');
const title = p.querySelector('.link-host-title');
if (btn) btn.textContent = '[–]';
if (title) title.setAttribute('aria-expanded', 'true');
});
try { sessionStorage.setItem('linksCollapsed', '{}'); } catch(_) {}
}
// ── Stale data warning ────────────────────────────────────────────
function checkLinksStale(updatedStr) {
if (!updatedStr) return;
const age = (Date.now() - new Date(_toIso(updatedStr))) / 1000;
let banner = document.getElementById('links-stale-banner');
if (age > 120) {
if (!banner) {
banner = document.createElement('div');
banner.id = 'links-stale-banner';
banner.className = 'lt-alert lt-alert--warning';
banner.innerHTML = '<span class="lt-alert-icon">⚠</span><div class="lt-alert-body"><div class="lt-alert-msg"></div></div>';
document.getElementById('links-container').prepend(banner);
}
banner.querySelector('.lt-alert-msg').textContent =
`Link data may be stale — last updated ${Math.floor(age/60)}m ago.`;
banner.style.display = '';
} else if (banner) {
banner.style.display = 'none';
}
}
// ── Fetch + render ────────────────────────────────────────────────
async function loadLinks() {
try {
const data = await lt.api.get('/api/links');
if ((!data.hosts || !Object.keys(data.hosts).length) && (!data.unifi_switches || !Object.keys(data.unifi_switches).length)) {
document.getElementById('links-container').innerHTML =
'<div class="link-no-data">No link data yet — monitor has not completed a full cycle.</div>';
return;
}
const updEl = document.getElementById('links-updated');
if (updEl && data.updated) {
updEl.textContent = 'Updated: ' + new Date(_toIso(data.updated)).toLocaleTimeString();
}
renderLinks(data);
checkLinksStale(data.updated);
} catch (e) {
document.getElementById('links-container').innerHTML =
'<div class="error-state">Network error loading link statistics.</div>';
lt.toast.error('Failed to load link statistics');
}
}
loadLinks();
const _linksInterval = window.gandalfSettings?.refreshInterval ?? 60;
if (_linksInterval > 0) lt.autoRefresh.start(loadLinks, Math.max(_linksInterval, 15) * 1000);
window.onGandalfSettingsChanged = function(s) {
lt.autoRefresh.stop();
if (s.refreshInterval > 0) lt.autoRefresh.start(loadLinks, Math.max(s.refreshInterval, 15) * 1000);
};
document.addEventListener('click', e => {
const toggleTitle = e.target.closest('[data-action="toggle-panel"]');
if (toggleTitle) { togglePanel(toggleTitle.closest('.link-host-panel')); return; }
if (e.target.closest('[data-action="collapse-all"]')) { collapseAll(); return; }
if (e.target.closest('[data-action="expand-all"]')) { expandAll(); return; }
});
document.addEventListener('keydown', e => {
if (e.key !== 'Enter' && e.key !== ' ') return;
const toggleTitle = e.target.closest('[data-action="toggle-panel"]');
if (toggleTitle) { e.preventDefault(); togglePanel(toggleTitle.closest('.link-host-panel')); }
});
document.getElementById('links-search')?.addEventListener('input', applyLinksSearch);
</script>
{% endblock %}
+340
View File
@@ -0,0 +1,340 @@
{% extends "base.html" %}
{% block title %}Suppressions – GANDALF{% endblock %}
{% block content %}
<div class="lt-page-header">
<div>
<h1 class="lt-page-title">Alert Suppressions</h1>
<p class="g-page-sub">Manage maintenance windows and per-target alert suppression rules.</p>
</div>
</div>
<!-- ── Create suppression ─────────────────────────────────────────── -->
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">Create Suppression</h2>
</div>
<div class="lt-card">
<div class="lt-card-body">
<form id="create-suppression-form">
<div class="form-row">
<div class="lt-form-group">
<label class="lt-label" for="s-type">Target Type <span class="required">*</span></label>
<select class="lt-select" id="s-type" name="target_type">
<option value="host">Host (all interfaces)</option>
<option value="interface">Specific Interface</option>
<option value="unifi_device">UniFi Device</option>
<option value="all">Global (suppress everything)</option>
</select>
</div>
<div class="lt-form-group" id="name-group">
<label class="lt-label" for="s-name">Target Name <span class="required">*</span></label>
<input type="text" class="lt-input" id="s-name" name="target_name"
placeholder="hostname or device name" autocomplete="off"
required aria-required="true" list="target-name-list">
<datalist id="target-name-list">
{% for name in snapshot.hosts.keys() | sort %}
<option value="{{ name }}">
{% endfor %}
</datalist>
</div>
<div class="lt-form-group" id="detail-group" style="display:none">
<label class="lt-label" for="s-detail">Interface Name</label>
<input type="text" class="lt-input" id="s-detail" name="target_detail"
placeholder="e.g. enp35s0 or bond0" autocomplete="off">
</div>
</div>
<div class="form-row">
<div class="lt-form-group form-group-wide">
<label class="lt-label" for="s-reason">Reason <span class="required">*</span></label>
<input type="text" class="lt-input" id="s-reason" name="reason"
placeholder="e.g. Planned switch maintenance, replacing SFP on large1/enp43s0"
required aria-required="true">
</div>
</div>
<div class="form-row form-row-align">
<div class="lt-form-group">
<label class="lt-label">Duration</label>
<div class="duration-pills" role="group" aria-label="Select suppression duration">
<button type="button" class="pill" data-duration="30" aria-pressed="false" aria-label="30 minutes">30 min</button>
<button type="button" class="pill" data-duration="60" aria-pressed="false" aria-label="1 hour">1 hr</button>
<button type="button" class="pill" data-duration="240" aria-pressed="false" aria-label="4 hours">4 hr</button>
<button type="button" class="pill" data-duration="480" aria-pressed="false" aria-label="8 hours">8 hr</button>
<button type="button" class="pill pill-manual active" data-duration="" aria-pressed="true" aria-label="Manual, no expiry">Manual ∞</button>
</div>
<input type="hidden" id="s-expires" name="expires_minutes" value="">
<div class="lt-field-hint" id="s-dur-hint">Persists until manually removed.</div>
</div>
<div class="lt-form-group form-group-submit">
<button type="submit" class="lt-btn lt-btn-primary lt-btn-lg">🔕 Apply Suppression</button>
</div>
</div>
</form>
</div>
</div>
</section>
<!-- ── Active suppressions ────────────────────────────────────────── -->
<section class="g-section" id="active-sup-section">
<div class="g-section-header">
<h2 class="g-section-title">Active Suppressions</h2>
<span class="g-section-badge" id="active-sup-badge">{{ active | length }}</span>
</div>
<div id="active-sup-wrap">
{% if active %}
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
<span class="lt-frame-br">&#x255D;</span>
<div class="lt-section-header">Active Rules</div>
<div class="lt-table-wrap">
<table class="lt-table" id="active-sup-table">
<caption class="lt-sr-only">Active suppression rules</caption>
<thead>
<tr>
<th>Type</th><th>Target</th><th>Detail</th><th>Reason</th>
<th>By</th><th>Created</th><th>Expires</th><th>Actions</th>
</tr>
</thead>
<tbody>
{% for s in active %}
<tr id="sup-row-{{ s.id }}">
{%- set _sup_badge = {'host':'badge-warning','interface':'badge-info','unifi_device':'badge-purple','all':'badge-critical'} -%}
<td><span class="lt-badge {{ _sup_badge.get(s.target_type, 'badge-neutral') }}">{{ s.target_type }}</span></td>
<td>{{ s.target_name or 'all' }}</td>
<td>{{ s.target_detail or '–' }}</td>
<td>{{ s.reason }}</td>
<td>{{ s.suppressed_by }}</td>
<td class="ts-cell">{{ s.created_at }}</td>
<td class="ts-cell">{% if s.expires_at %}{{ s.expires_at }}{% else %}<em>manual</em>{% endif %}</td>
<td>
<button class="lt-btn lt-btn-danger lt-btn-sm" data-action="remove-sup" data-sup-id="{{ s.id }}"
aria-label="Remove suppression for {{ s.target_name or 'global' }}">Remove</button>
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
{% else %}
<div class="lt-empty-state lt-empty-state--sm" id="no-active-msg">
<div class="lt-empty-state-icon">🔕</div>
<div class="lt-empty-state-title">No active suppressions</div>
<div class="lt-empty-state-body">All alerts are active. Use the form above to silence a host or interface.</div>
</div>
{% endif %}
</div>
</section>
<!-- ── Suppression history ────────────────────────────────────────── -->
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">History</h2>
<span class="g-section-badge">{{ history | length }}</span>
</div>
{% if history %}
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
<span class="lt-frame-br">&#x255D;</span>
<div class="lt-section-header">Suppression Log</div>
<div class="lt-table-wrap">
<table class="lt-table lt-table-sm">
<caption class="lt-sr-only">Suppression history</caption>
<thead>
<tr>
<th>Type</th><th>Target</th><th>Detail</th><th>Reason</th>
<th>By</th><th>Created</th><th>Expires</th><th>Active</th>
</tr>
</thead>
<tbody>
{% for s in history %}
<tr class="{% if not s.active %}row-resolved{% endif %}">
<td>{{ s.target_type }}</td>
<td>{{ s.target_name or 'all' }}</td>
<td>{{ s.target_detail or '–' }}</td>
<td>{{ s.reason }}</td>
<td>{{ s.suppressed_by }}</td>
<td class="ts-cell">{{ s.created_at }}</td>
<td class="ts-cell">{% if s.expires_at %}{{ s.expires_at }}{% else %}<em>manual</em>{% endif %}</td>
<td>
{% if s.active %}
<span class="lt-badge badge-ok">Yes</span>
{% else %}
<span class="lt-badge badge-neutral">No</span>
{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>
</div>
{% else %}
<div class="lt-empty-state lt-empty-state--sm">
<div class="lt-empty-state-icon">📋</div>
<div class="lt-empty-state-title">No suppression history yet</div>
</div>
{% endif %}
</section>
<!-- ── Available targets reference ───────────────────────────────── -->
<section class="g-section">
<div class="g-section-header">
<h2 class="g-section-title">Available Targets</h2>
</div>
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
<span class="lt-frame-br">&#x255D;</span>
<div class="lt-section-header">Host &amp; Interface Reference</div>
<div class="lt-section-body">
<div class="targets-grid">
{% for name, host in snapshot.hosts.items() %}
<div class="target-card">
<div class="target-name">{{ name }}</div>
<div class="target-type">{{ 'Proxmox Host (prometheus)' if host.source == 'prometheus' else 'Ping-only host' }}</div>
{% if host.interfaces %}
<div class="target-ifaces">
{% for iface in host.interfaces.keys() | sort %}
<code class="iface-chip">{{ iface }}</code>
{% endfor %}
</div>
{% endif %}
</div>
{% endfor %}
</div>
</div>
</div>
</section>
{% endblock %}
{% block scripts %}
<script>
function onTypeChange() {
const t = document.getElementById('s-type').value;
document.getElementById('name-group').style.display = (t==='all') ? 'none' : '';
document.getElementById('detail-group').style.display = (t==='interface') ? '' : 'none';
const nameInput = document.getElementById('s-name');
if (nameInput) {
const req = (t !== 'all');
nameInput.required = req;
nameInput.setAttribute('aria-required', String(req));
}
}
function setDur(mins, el) {
setDuration(mins, el, { expiresId: 's-expires', pillSel: '#create-suppression-form .pill', hintId: 's-dur-hint' });
}
function renderActiveRows(rows) {
const wrap = document.getElementById('active-sup-wrap');
const badge = document.getElementById('active-sup-badge');
if (!rows || !rows.length) {
wrap.innerHTML = '<div class="lt-empty-state lt-empty-state--sm" id="no-active-msg"><div class="lt-empty-state-icon">🔕</div><div class="lt-empty-state-title">No active suppressions</div><div class="lt-empty-state-body">All alerts are active. Use the form above to silence a host or interface.</div></div>';
if (badge) badge.textContent = '0';
return;
}
if (badge) badge.textContent = rows.length;
const SUP_BADGE = {host:'badge-warning', interface:'badge-info', unifi_device:'badge-purple', all:'badge-critical'};
const tbody = rows.map(s => `
<tr id="sup-row-${s.id}">
<td><span class="lt-badge ${SUP_BADGE[s.target_type] || 'badge-neutral'}">${lt.escHtml(s.target_type)}</span></td>
<td>${lt.escHtml(s.target_name || 'all')}</td>
<td>${lt.escHtml(s.target_detail || '–')}</td>
<td>${lt.escHtml(s.reason)}</td>
<td>${lt.escHtml(s.suppressed_by)}</td>
<td class="ts-cell">${lt.escHtml(s.created_at || '')}</td>
<td class="ts-cell">${s.expires_at ? lt.escHtml(s.expires_at) : '<em>manual</em>'}</td>
<td><button class="lt-btn lt-btn-danger lt-btn-sm" data-action="remove-sup" data-sup-id="${s.id}"
aria-label="Remove suppression for ${lt.escHtml(s.target_name || 'global')}">Remove</button></td>
</tr>`).join('');
wrap.innerHTML = `
<div class="lt-frame">
<span class="lt-frame-bl">&#x255A;</span>
<span class="lt-frame-br">&#x255D;</span>
<div class="lt-section-header">Active Rules</div>
<div class="lt-table-wrap">
<table class="lt-table" id="active-sup-table">
<caption class="lt-sr-only">Active suppression rules</caption>
<thead><tr>
<th>Type</th><th>Target</th><th>Detail</th><th>Reason</th>
<th>By</th><th>Created</th><th>Expires</th><th>Actions</th>
</tr></thead>
<tbody>${tbody}</tbody>
</table>
</div>
</div>`;
}
async function refreshActive() {
try {
const rows = await lt.api.get('/api/suppressions');
renderActiveRows(rows);
} catch (err) {
showToast('Failed to refresh suppressions', 'warning');
}
}
async function createSuppression(e) {
e.preventDefault();
const form = e.target;
const btn = form.querySelector('[type="submit"]');
btn.classList.add('is-loading');
const payload = {
target_type: form.target_type.value,
target_name: form.target_name ? form.target_name.value : '',
target_detail: document.getElementById('s-detail').value,
reason: form.reason.value,
expires_minutes: form.expires_minutes.value ? parseInt(form.expires_minutes.value) : null,
};
try {
await lt.api.post('/api/suppressions', payload);
showToast('Suppression applied', 'success');
form.reset();
onTypeChange();
setDur(null, document.querySelector('#create-suppression-form .pill-manual'));
await refreshActive();
} catch (err) {
showToast(err.message || 'Error', 'error');
} finally {
btn.classList.remove('is-loading');
}
}
async function removeSuppression(id) {
if (!confirm('Remove this suppression?')) return;
try {
await lt.api.delete(`/api/suppressions/${id}`);
document.getElementById(`sup-row-${id}`)?.remove();
const badge = document.getElementById('active-sup-badge');
if (badge) badge.textContent = Math.max(0, parseInt(badge.textContent || '0') - 1);
const tbody = document.querySelector('#active-sup-table tbody');
if (tbody && !tbody.children.length) {
document.getElementById('active-sup-wrap').innerHTML =
'<div class="lt-empty-state lt-empty-state--sm" id="no-active-msg"><div class="lt-empty-state-icon">🔕</div><div class="lt-empty-state-title">No active suppressions</div><div class="lt-empty-state-body">All alerts are active. Use the form above to silence a host or interface.</div></div>';
if (badge) badge.textContent = '0';
}
showToast('Suppression removed', 'success');
} catch (err) {
showToast(err.message || 'Failed to remove suppression', 'error');
}
}
document.getElementById('s-type')?.addEventListener('change', onTypeChange);
document.getElementById('create-suppression-form')?.addEventListener('submit', createSuppression);
document.addEventListener('click', e => {
const pill = e.target.closest('#create-suppression-form .pill[data-duration]');
if (pill) {
const val = pill.dataset.duration;
setDur(val ? parseInt(val) : null, pill);
return;
}
const removeBtn = e.target.closest('[data-action="remove-sup"]');
if (removeBtn) removeSuppression(parseInt(removeBtn.dataset.supId));
});
</script>
{% endblock %}
+485
View File
@@ -0,0 +1,485 @@
"""Tests for gandalf.diagnose — all pure static methods, no external deps."""
import sys
import os
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
from diagnose import DiagnosticsRunner # noqa: E402
# ── build_ssh_command ────────────────────────────────────────────────────────
class TestBuildSshCommand:
def test_contains_stricthostkeychecking_accept_new(self):
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', 'eth0')
assert 'StrictHostKeyChecking=accept-new' in cmd
def test_contains_host_ip(self):
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', 'eth0')
assert 'root@10.0.0.1' in cmd
def test_contains_interface_name(self):
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', 'eth0')
assert 'eth0' in cmd
def test_shell_quotes_special_chars_in_iface(self):
# shlex.quote wraps the whole iface string in single quotes so
# shell metacharacters are not interpreted as commands
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', "eth0; rm -rf /")
# The iface must appear inside single quotes
assert "'eth0; rm -rf /'" in cmd
def test_contains_sysfs_stats_section(self):
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', 'eth0')
assert 'sysfs_stats' in cmd
def test_contains_ethtool_section(self):
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', 'eth0')
assert 'ethtool' in cmd
def test_dmesg_uses_fixed_string_grep(self):
# grep -F prevents iface names with dots (e.g. eth0.1) being treated as
# regex wildcards; -- prevents leading - from being parsed as a flag
cmd = DiagnosticsRunner.build_ssh_command('10.0.0.1', 'eth0')
assert 'grep -F --' in cmd
# ── parse_output ─────────────────────────────────────────────────────────────
SAMPLE_OUTPUT = """\
=== carrier ===
1
=== operstate ===
up
=== sysfs_stats ===
rx_bytes:12345
tx_bytes:67890
rx_errors:0
tx_errors:0
rx_dropped:0
tx_dropped:0
rx_crc_errors:0
rx_frame_errors:0
rx_fifo_errors:0
tx_carrier_errors:0
collisions:0
rx_missed_errors:0
=== carrier_changes ===
3
=== ethtool ===
Speed: 1000Mb/s
Duplex: Full
Link detected: yes
Auto-negotiation: on
=== ethtool_driver ===
driver: igb
version: 5.6.0-k
firmware-version: 1.67, 0x80000d38
bus-info: 0000:03:00.0
=== ethtool_pause ===
RX: on
TX: off
=== ethtool_ring ===
Pre-set maximums:
RX: 4096
TX: 4096
Current hardware settings:
RX: 256
TX: 256
=== ethtool_stats ===
rx_packets: 1000
tx_packets: 2000
=== ethtool_dom ===
=== ip_link ===
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
=== ip_addr ===
inet 10.0.0.1/24
=== ip_route ===
default via 10.0.0.254
=== dmesg ===
eth0: renamed from ens3
=== lldpctl ===
lldpd not running
=== end ===
"""
class TestParseOutput:
def setup_method(self):
self.parsed = DiagnosticsRunner.parse_output(SAMPLE_OUTPUT)
def test_carrier_parsed(self):
assert self.parsed['carrier'] == '1'
def test_operstate_parsed(self):
assert self.parsed['operstate'] == 'up'
def test_carrier_changes_is_int(self):
assert self.parsed['carrier_changes'] == 3
def test_sysfs_stats_rx_bytes(self):
assert self.parsed['sysfs_stats']['rx_bytes'] == 12345
def test_sysfs_stats_tx_bytes(self):
assert self.parsed['sysfs_stats']['tx_bytes'] == 67890
def test_ethtool_speed(self):
assert self.parsed['ethtool']['speed_mbps'] == 1000
def test_ethtool_link_detected(self):
assert self.parsed['ethtool']['link_detected'] is True
def test_ethtool_duplex(self):
assert self.parsed['ethtool']['duplex'] == 'full'
def test_ethtool_auto_neg(self):
assert self.parsed['ethtool']['auto_neg'] is True
def test_ethtool_driver(self):
assert self.parsed['ethtool_driver']['driver'] == 'igb'
def test_ethtool_pause_rx_on(self):
assert self.parsed['ethtool_pause']['rx_pause'] is True
def test_ethtool_pause_tx_off(self):
assert self.parsed['ethtool_pause']['tx_pause'] is False
def test_ethtool_ring_current(self):
assert self.parsed['ethtool_ring']['rx_current'] == 256
assert self.parsed['ethtool_ring']['tx_current'] == 256
def test_ethtool_ring_max(self):
assert self.parsed['ethtool_ring']['rx_max'] == 4096
def test_ip_addr_present(self):
assert '10.0.0.1/24' in self.parsed['ip_addr']
def test_ip_route_present(self):
assert 'default via 10.0.0.254' in self.parsed['ip_route']
class TestParseOutputEdgeCases:
def test_empty_string_returns_dict(self):
result = DiagnosticsRunner.parse_output('')
assert isinstance(result, dict)
assert result['carrier'] == '?'
assert result['operstate'] == '?'
assert result['carrier_changes'] == 0
def test_no_end_sentinel_still_parses(self):
output = "=== carrier ===\n1\n=== operstate ===\nup\n"
result = DiagnosticsRunner.parse_output(output)
assert result['carrier'] == '1'
# ── parse_sysfs_stats ────────────────────────────────────────────────────────
class TestParseSysfsStats:
def test_parses_known_keys(self):
text = "rx_bytes:100\ntx_bytes:200\nrx_errors:0\n"
result = DiagnosticsRunner.parse_sysfs_stats(text)
assert result['rx_bytes'] == 100
assert result['tx_bytes'] == 200
assert result['rx_errors'] == 0
def test_ignores_unknown_keys(self):
text = "unknown_counter:999\nrx_bytes:50\n"
result = DiagnosticsRunner.parse_sysfs_stats(text)
assert 'unknown_counter' not in result
assert result['rx_bytes'] == 50
def test_non_numeric_value_defaults_to_zero(self):
text = "rx_bytes:not_a_number\n"
result = DiagnosticsRunner.parse_sysfs_stats(text)
assert result['rx_bytes'] == 0
def test_empty_text_returns_empty_dict(self):
assert DiagnosticsRunner.parse_sysfs_stats('') == {}
# ── parse_ethtool ─────────────────────────────────────────────────────────────
class TestParseEthtool:
def test_speed_parsed(self):
text = "Speed: 10000Mb/s\n"
result = DiagnosticsRunner.parse_ethtool(text)
assert result['speed_mbps'] == 10000
def test_unknown_speed(self):
text = "Speed: Unknown! (0)\n"
result = DiagnosticsRunner.parse_ethtool(text)
assert result['speed_mbps'] is None
def test_link_detected_no(self):
text = "Link detected: no\n"
result = DiagnosticsRunner.parse_ethtool(text)
assert result['link_detected'] is False
def test_auto_neg_off(self):
text = "Auto-negotiation: off\n"
result = DiagnosticsRunner.parse_ethtool(text)
assert result['auto_neg'] is False
def test_empty_returns_empty(self):
assert DiagnosticsRunner.parse_ethtool('') == {}
# ── parse_ethtool_driver ──────────────────────────────────────────────────────
class TestParseEthtoolDriver:
def test_driver_name(self):
text = "driver: igb\nversion: 5.6.0-k\nfirmware-version: 1.67\nbus-info: 0000:03:00.0\n"
result = DiagnosticsRunner.parse_ethtool_driver(text)
assert result['driver'] == 'igb'
def test_version(self):
text = "driver: igb\nversion: 5.6.0-k\n"
result = DiagnosticsRunner.parse_ethtool_driver(text)
assert result['version'] == '5.6.0-k'
def test_firmware_version(self):
text = "firmware-version: 1.67, 0x80000d38\n"
result = DiagnosticsRunner.parse_ethtool_driver(text)
assert result['firmware_version'] == '1.67, 0x80000d38'
def test_bus_info(self):
text = "bus-info: 0000:03:00.0\n"
result = DiagnosticsRunner.parse_ethtool_driver(text)
assert result['bus_info'] == '0000:03:00.0'
def test_empty_returns_empty(self):
assert DiagnosticsRunner.parse_ethtool_driver('') == {}
# ── parse_nic_stats ───────────────────────────────────────────────────────────
class TestParseNicStats:
def test_parses_integer_values(self):
text = "rx_packets: 1234\ntx_packets: 5678\n"
result = DiagnosticsRunner.parse_nic_stats(text)
assert result['rx_packets'] == 1234
assert result['tx_packets'] == 5678
def test_non_numeric_skipped(self):
text = "rx_packets: 100\nbad_stat: N/A\n"
result = DiagnosticsRunner.parse_nic_stats(text)
assert 'bad_stat' not in result
assert result['rx_packets'] == 100
def test_empty_returns_empty(self):
assert DiagnosticsRunner.parse_nic_stats('') == {}
# ── parse_ethtool_dom ─────────────────────────────────────────────────────────
class TestParseEthtoolDom:
def test_unsupported_returns_empty(self):
assert DiagnosticsRunner.parse_ethtool_dom('Cannot get module EEPROM information') == {}
def test_empty_returns_empty(self):
assert DiagnosticsRunner.parse_ethtool_dom('') == {}
def test_vendor_name(self):
text = "Vendor name : CISCO-FINISAR\n"
result = DiagnosticsRunner.parse_ethtool_dom(text)
assert result['vendor'] == 'CISCO-FINISAR'
def test_wavelength(self):
text = "Laser wavelength : 1310 nm\n"
result = DiagnosticsRunner.parse_ethtool_dom(text)
assert result['wavelength_nm'] == 1310
def test_tx_power_dbm(self):
text = "Laser output power : 0.3000 mW / -5.23 dBm\n"
result = DiagnosticsRunner.parse_ethtool_dom(text)
assert abs(result['tx_power_dbm'] - (-5.23)) < 0.01
def test_rx_power_dbm(self):
text = "Receiver signal average optical power : 0.1000 mW / -10.00 dBm\n"
result = DiagnosticsRunner.parse_ethtool_dom(text)
assert abs(result['rx_power_dbm'] - (-10.00)) < 0.01
# ── parse_ip_link ─────────────────────────────────────────────────────────────
class TestParseIpLink:
IP_LINK_SAMPLE = """\
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether aa:bb:cc:dd:ee:ff brd ff:ff:ff:ff:ff:ff
RX: bytes packets errors dropped missed mcast
123456789 1000000 0 5 0 0
TX: bytes packets errors dropped carrier collsns
98765432 900000 0 0 0 0
"""
def test_mtu_parsed(self):
result = DiagnosticsRunner.parse_ip_link(self.IP_LINK_SAMPLE)
assert result['mtu'] == 1500
def test_state_parsed(self):
result = DiagnosticsRunner.parse_ip_link(self.IP_LINK_SAMPLE)
assert result['state'] == 'up'
def test_rx_bytes(self):
result = DiagnosticsRunner.parse_ip_link(self.IP_LINK_SAMPLE)
assert result['ip_rx_bytes'] == 123456789
def test_tx_bytes(self):
result = DiagnosticsRunner.parse_ip_link(self.IP_LINK_SAMPLE)
assert result['ip_tx_bytes'] == 98765432
def test_rx_dropped(self):
result = DiagnosticsRunner.parse_ip_link(self.IP_LINK_SAMPLE)
assert result['ip_rx_dropped'] == 5
def test_empty_returns_empty(self):
assert DiagnosticsRunner.parse_ip_link('') == {}
# ── parse_dmesg ───────────────────────────────────────────────────────────────
class TestParseDmesg:
def test_error_severity(self):
text = "[ 1.234567] eth0: Link failure detected\n"
events = DiagnosticsRunner.parse_dmesg(text)
assert len(events) == 1
assert events[0]['severity'] == 'error'
def test_warn_severity(self):
text = "[ 2.000000] eth0: dropped packet\n"
events = DiagnosticsRunner.parse_dmesg(text)
assert events[0]['severity'] == 'warn'
def test_info_severity(self):
text = "[ 3.000000] eth0: NIC Link is Up\n"
events = DiagnosticsRunner.parse_dmesg(text)
assert events[0]['severity'] == 'info'
def test_timestamp_extracted(self):
text = "[ 5.678900] eth0: reset\n"
events = DiagnosticsRunner.parse_dmesg(text)
assert events[0]['timestamp'] == '5.678900'
def test_no_timestamp(self):
text = "eth0: some timeout event\n"
events = DiagnosticsRunner.parse_dmesg(text)
assert events[0]['timestamp'] == ''
assert events[0]['severity'] == 'error'
def test_empty_returns_empty_list(self):
assert DiagnosticsRunner.parse_dmesg('') == []
# ── parse_lldpctl ─────────────────────────────────────────────────────────────
class TestParseLldpctl:
def test_unavailable_when_not_running(self):
result = DiagnosticsRunner.parse_lldpctl('lldpd not running')
assert result == {'available': False}
def test_unavailable_on_empty(self):
result = DiagnosticsRunner.parse_lldpctl('')
assert result == {'available': False}
def test_neighbor_system_parsed(self):
text = " SysName: core-sw-01\n PortID: Gi1/0/5\n ChassisID: aa:bb:cc:dd:ee:ff\n"
result = DiagnosticsRunner.parse_lldpctl(text)
assert result['available'] is True
assert result['neighbor_system'] == 'core-sw-01'
def test_neighbor_port_parsed(self):
text = " SysName: core-sw-01\n PortID: Gi1/0/5\n"
result = DiagnosticsRunner.parse_lldpctl(text)
assert result['neighbor_port'] == 'Gi1/0/5'
def test_chassis_id_parsed(self):
text = " ChassisID: aa:bb:cc:dd:ee:ff\n"
result = DiagnosticsRunner.parse_lldpctl(text)
assert result['neighbor_chassis_id'] == 'aa:bb:cc:dd:ee:ff'
# ── analyze ───────────────────────────────────────────────────────────────────
class TestAnalyze:
def _sections(self, **overrides):
base = {
'carrier': '1',
'operstate': 'up',
'ethtool': {'link_detected': True, 'duplex': 'full', 'speed_mbps': 1000},
'sysfs_stats': {'rx_crc_errors': 0},
'ethtool_dom': {},
'dmesg': [],
'lldpctl': {'available': False},
'carrier_changes': 0,
}
base.update(overrides)
return base
def test_no_carrier_is_issue(self):
result = DiagnosticsRunner.analyze(self._sections(carrier='0'), {})
codes = [i['code'] for i in result['issues']]
assert 'NO_CARRIER' in codes
def test_half_duplex_is_issue(self):
sections = self._sections(ethtool={'duplex': 'half', 'link_detected': True, 'speed_mbps': 1000})
result = DiagnosticsRunner.analyze(sections, {})
codes = [i['code'] for i in result['issues']]
assert 'HALF_DUPLEX' in codes
def test_speed_mismatch_is_warning(self):
result = DiagnosticsRunner.analyze(
self._sections(ethtool={'duplex': 'full', 'link_detected': True, 'speed_mbps': 100}),
{'speed_mbps': 1000}
)
codes = [w['code'] for w in result['warnings']]
assert 'SPEED_MISMATCH' in codes
def test_sfp_rx_critical_low(self):
sections = self._sections(ethtool_dom={'rx_power_dbm': -30.0})
result = DiagnosticsRunner.analyze(sections, {})
codes = [i['code'] for i in result['issues']]
assert 'SFP_RX_CRITICAL' in codes
def test_sfp_rx_low_is_warning(self):
sections = self._sections(ethtool_dom={'rx_power_dbm': -20.0})
result = DiagnosticsRunner.analyze(sections, {})
codes = [w['code'] for w in result['warnings']]
assert 'SFP_RX_LOW' in codes
def test_high_crc_is_issue(self):
sections = self._sections(sysfs_stats={'rx_crc_errors': 200})
result = DiagnosticsRunner.analyze(sections, {})
codes = [i['code'] for i in result['issues']]
assert 'CRC_ERRORS_HIGH' in codes
def test_low_crc_is_warning(self):
sections = self._sections(sysfs_stats={'rx_crc_errors': 50})
result = DiagnosticsRunner.analyze(sections, {})
codes = [w['code'] for w in result['warnings']]
assert 'CRC_ERRORS_LOW' in codes
def test_carrier_flapping_issue(self):
result = DiagnosticsRunner.analyze(self._sections(carrier_changes=150), {})
codes = [i['code'] for i in result['issues']]
assert 'CARRIER_FLAPPING' in codes
def test_carrier_flaps_warning(self):
result = DiagnosticsRunner.analyze(self._sections(carrier_changes=25), {})
codes = [w['code'] for w in result['warnings']]
assert 'CARRIER_FLAPS' in codes
def test_lldp_missing_is_info(self):
result = DiagnosticsRunner.analyze(self._sections(lldpctl={'available': False}), {})
codes = [i['code'] for i in result['info']]
assert 'LLDP_MISSING' in codes
def test_lldp_mismatch_is_warning(self):
sections = self._sections(lldpctl={'available': True, 'neighbor_system': 'wrong-switch'})
switch_data = {'speed_mbps': 1000, 'lldp': {'system_name': 'core-sw-01'}}
result = DiagnosticsRunner.analyze(sections, switch_data)
codes = [w['code'] for w in result['warnings']]
assert 'LLDP_MISMATCH' in codes
def test_healthy_link_no_issues(self):
result = DiagnosticsRunner.analyze(self._sections(), {})
assert result['issues'] == []
assert result['warnings'] == []