Monitor: keep checking interfaces whose link-down alert is still open #31

Merged

jared merged 1 commits from stale-interface-alerts into main 2026-11-11 01:06:38 -05:00

Conversation 0 Commits 1 Files Changed 2

+25

jared commented 2026-11-11 01:03:09 -05:00

Owner

Copy Link

Copy Source

Why

micro1's enp3s0 link-down alert has been open since March, frozen at 1 failed check. It never ticketed, updated or resolved.

After the interface baseline is reset, a port that's already down counts as "first seen down". The monitor neither alerts nor resolves on a port in that state. Any alert opened before the reset was left stuck.

What changes

An interface with an open interface_down alert is treated as previously up, whatever the baseline now says. Its alert keeps counting failed checks, opens a ticket at the threshold, and resolves when the link returns.

Ports first seen down with no alert still stay quiet, as before.

After deploy

micro1 enp3s0 will open a P2 ticket within about two checks. The port is meant to be up and is waiting for a longer cable, so the ticket works as the reminder. When the cable is in and the port comes up, the alert resolves and the ticket gets a recovery note.

Testing

373 tests pass (3 new):

• an open alert survives a baseline reset and tickets
• it resolves when the link returns
• a port first seen down with no alert stays quiet

flake8 is clean.

🤖 Generated with Claude Code

## Why micro1's `enp3s0` link-down alert has been open since March, frozen at 1 failed check. It never ticketed, updated or resolved. After the interface baseline is reset, a port that's already down counts as "first seen down". The monitor neither alerts nor resolves on a port in that state. Any alert opened before the reset was left stuck. ## What changes An interface with an open `interface_down` alert is treated as previously up, whatever the baseline now says. Its alert keeps counting failed checks, opens a ticket at the threshold, and resolves when the link returns. Ports first seen down with no alert still stay quiet, as before. ## After deploy micro1 `enp3s0` will open a P2 ticket within about two checks. The port is meant to be up and is waiting for a longer cable, so the ticket works as the reminder. When the cable is in and the port comes up, the alert resolves and the ticket gets a recovery note. ## Testing 373 tests pass (3 new): - an open alert survives a baseline reset and tickets - it resolves when the link returns - a port first seen down with no alert stays quiet flake8 is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jared added 1 commit 2026-11-11 01:03:09 -05:00

Monitor: keep checking interfaces whose link-down alert is still open ... d07787ee38

After the interface baseline is reset, a port that is already down reads as
"first seen down", which neither alerts nor resolves. An alert opened before
the reset was then stuck open without updates: micro1's enp3s0 had sat at
one failed check since March with no ticket. An interface with an open
interface_down alert is now treated as previously up, so the alert keeps
counting, tickets and resolves as normal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

jared merged commit a0e062493d into main 2026-11-11 01:06:38 -05:00

jared referenced this issue from a commit 2026-11-11 01:11:05 -05:00

Revert #31: open link-down alerts from a monitor glitch were revived and ticketed

jared referenced this pull request 2026-11-11 01:11:11 -05:00

Revert #31: revived glitch alerts opened tickets #33

jared referenced this issue from a commit 2026-11-11 01:17:50 -05:00

Merge pull request 'Revert #31: revived glitch alerts opened tickets' (#33) from revert-31 into main

jared referenced this pull request 2026-11-11 17:20:07 -05:00

Mark host interfaces as unused #35

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

cole (Cole Keller) jared (Jared Vititoe) nathan (Nathan Vititoe)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: LotusGuild/gandalf#31