Recurring hwmonDaemon alerts are silently swallowed when the matching ticket is Closed and can't be auto-reopened #115

New Issue

2026-11-03T01:22:51-05:00

jared commented

2026-11-03 01:22:51 -05:00

Summary

When create_ticket_api.php matches an incoming automated report to an existing ticket by dedup hash, and that ticket is Closed, it tries to reopen it through the Workflow Designer. If no admin-free, comment-free transition out of Closed is configured (the current setup), the API leaves the ticket Closed, posts an "Issue recurred, but the ticket could not be reopened automatically" comment, and returns success: true.

As a result the alert disappears: closed tickets don't show in any open queue, and hwmonDaemon logs Ticket created successfully for every run.

Real-world impact

LXC 118 on compute-storage-gpu-01 reached 90.5% root disk usage (critical threshold). hwmonDaemon reported it every hour, but each report matched #147937387 (an LXC 118 storage ticket from June, closed 2026-07-15). That closed ticket now has 786 comments, mostly the same "could not be reopened" message posted about once an hour, and nobody saw the alert.

In the same 01:01 run, LXC 107 (81.5%) had no prior ticket, so it got a fresh, visible ticket (571365260). Anyone looking at the ticket queue would conclude 107 was the problem and 118 was fine.

Every recurring automated alert whose ticket has been closed behaves this way, so other alerts may be hidden too.

Fix

In the closed-ticket branch, when no unattended reopen transition exists:

Release the dedup hash from the closed ticket (hash = NULL). The column is UNIQUE, so the new ticket can't take it otherwise.
Fall through to the normal create path: insert a new ticket carrying the hash, audit-log it and send the usual notification.
Link the new ticket to the old one (relates_to) and comment on both, so the history stays connected.

Further recurrences then dedup onto the new open ticket in the usual way. When a Closed→Open transition is allowed, behaviour is unchanged and the old ticket is reopened.

Related (hwmonDaemon, not fixed here)

hwmonDaemon logs Ticket created successfully whenever the API returns success: true, even when action is deduplicated, updated or recurrence_noted. Logging the action would have made this visible in the journal.

## Summary When `create_ticket_api.php` matches an incoming automated report to an existing ticket by dedup hash, and that ticket is **Closed**, it tries to reopen it through the Workflow Designer. If no admin-free, comment-free transition out of Closed is configured (the current setup), the API leaves the ticket Closed, posts an "Issue recurred, but the ticket could not be reopened automatically" comment, and returns `success: true`. As a result the alert disappears: closed tickets don't show in any open queue, and hwmonDaemon logs `Ticket created successfully` for every run. ## Real-world impact LXC 118 on compute-storage-gpu-01 reached **90.5%** root disk usage (critical threshold). hwmonDaemon reported it every hour, but each report matched #147937387 (an LXC 118 storage ticket from June, closed 2026-07-15). That closed ticket now has **786 comments**, mostly the same "could not be reopened" message posted about once an hour, and nobody saw the alert. In the same 01:01 run, LXC 107 (81.5%) had no prior ticket, so it got a fresh, visible ticket (571365260). Anyone looking at the ticket queue would conclude 107 was the problem and 118 was fine. Every recurring automated alert whose ticket has been closed behaves this way, so other alerts may be hidden too. ## Fix In the closed-ticket branch, when no unattended reopen transition exists: 1. Release the dedup hash from the closed ticket (`hash = NULL`). The column is `UNIQUE`, so the new ticket can't take it otherwise. 2. Fall through to the normal create path: insert a new ticket carrying the hash, audit-log it and send the usual notification. 3. Link the new ticket to the old one (`relates_to`) and comment on both, so the history stays connected. Further recurrences then dedup onto the new open ticket in the usual way. When a Closed→Open transition *is* allowed, behaviour is unchanged and the old ticket is reopened. ## Related (hwmonDaemon, not fixed here) hwmonDaemon logs `Ticket created successfully` whenever the API returns `success: true`, even when `action` is `deduplicated`, `updated` or `recurrence_noted`. Logging the `action` would have made this visible in the journal.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: LotusGuild/tinker_tickets#115