Compare commits

...

28 Commits

Author SHA1 Message Date
jared 53b61249b0 ci: add notify-failure, deploy tagging, and jest coverage
Lint / JS (eslint) (push) Successful in 11s
Test / JS Tests (jest) (push) Successful in 10s
Lint / Notify on failure (push) Has been skipped
Lint / Deploy (push) Successful in 2s
Security / JS Security (npm audit) (push) Successful in 10s
- lint.yml: add notify-failure Matrix alert job; add Tag deployed commit
  step to deploy job with deploy-YYYY.MM.DD-N tagging via Gitea API
- package.json: add --coverage flag to jest test script

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 16:25:12 -04:00
jared 162ca5f7a7 Add CI badges and CI/CD section to README
Lint / JS (eslint) (push) Successful in 10s
Security / JS Security (npm audit) (push) Successful in 14s
Test / JS Tests (jest) (push) Successful in 13s
Lint / Deploy (push) Successful in 3s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 12:53:48 -04:00
jared f4e44b67a9 Fix ESLint errors in test files and npm vulnerabilities
Lint / JS (eslint) (push) Successful in 11s
Security / JS Security (npm audit) (push) Successful in 10s
Test / JS Tests (jest) (push) Successful in 10s
Lint / Deploy (push) Successful in 3s
- Add tests/.eslintrc.json to declare jest globals (describe/test/expect)
- Fix no-useless-escape in lib/utils.js regex character class
- Run npm audit fix: updated path-to-regexp and qs (1 high, 1 moderate fixed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 12:41:09 -04:00
jared 6e5f18ea58 Add jest test suite, extract pure utils module, fix cron-parser v5 API
Lint / JS (eslint) (push) Failing after 12s
Security / JS Security (npm audit) (push) Failing after 13s
Test / JS Tests (jest) (push) Successful in 13s
Lint / Deploy (push) Has been skipped
- Extract validateWebhookUrl, applyParams, evalCondition, calculateNextRun
  to lib/utils.js so they can be tested without DB connection
- Fix cron-parser v5 API: parseExpression → CronExpressionParser.parse
- Add 31 jest tests covering all four utility functions
- Add test.yml CI workflow running jest on every push/PR
- Add jest devDependency

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 12:24:30 -04:00
jared 0a677d69a8 Add npm audit security scanning workflow
Lint / JS (eslint) (push) Successful in 8s
Security / JS Security (npm audit) (push) Failing after 7s
Lint / Deploy (push) Successful in 2s
Scans npm dependencies weekly and on every push/PR for high+ severity issues.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 11:26:10 -04:00
jared 1110804662 Add deploy gating to CI pipeline
Lint / JS (eslint) (push) Successful in 9s
Lint / Deploy (push) Successful in 4s
- Add deploy job gated on js-lint passing
- Deploy triggers pulse-deploy webhook on main branch only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 10:14:35 -04:00
jared 1b0ea8b648 ci: use --ext .js . instead of explicit file paths
Lint / JS (eslint) (push) Successful in 8s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 00:18:32 -04:00
jared 1d33e261dc ci: use explicit file paths in ESLint command instead of directory globs
Lint / JS (eslint) (push) Failing after 15s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 00:13:11 -04:00
jared d523b6d02a ci: add ESLint lint workflow with per-directory configs
Lint / JS (eslint) (push) Failing after 8s
Node.js env for server.js/worker/, browser env for public/.
All errors downgraded to warnings (empty blocks, inner declarations,
loose equality, useless escape, constant condition) for practical CI.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 23:46:46 -04:00
jared 0990e5b807 Add parse/route step types, wider modal, automated link troubleshooter
- Add 'parse' step type: reads KEY=VALUE lines from last command output into execution state
- Add 'route' step type: evaluates JS conditions list, auto-jumps to first match with no user input
- Support condition field on parse steps (skip when conditional execute was also skipped)
- Widen execution detail modal to min(1100px, 96vw) to eliminate horizontal scrolling
- Show last command output in prompt boxes so user has context before clicking
- Add formatLogEntry support for parse_complete and route_taken log actions
- Apply applyParams() substitution to prompt messages ({{server_name}}, {{iface}}, etc.)
- Fix prompt button onclick using data-opt attribute to avoid JSON double-quote breakage

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 19:29:28 -04:00
jared 2290d52f8b Fix prompt buttons, add command output to prompt steps, add worker to repo
- Fix onclick buttons broken by JSON.stringify double-quotes inside HTML
  attributes — use data-opt attribute + this.dataset.opt instead
- Track last command stdout in execution state when command_result arrives
- executePromptStep: include last command output in log entry and broadcast
  so users can review results alongside the question in the same view
- GET /api/executions/🆔 propagate output field to pending prompt response
- Add .prompt-output CSS class for scrollable terminal-style output block
- Fix MariaDB CAST(? AS JSON) → JSON_EXTRACT(?, '$') (MariaDB 10.11 compat)
- Add worker/worker.js to repo (deployed on pulse-worker-01 / LXC 153)
  Fix: worker was not echoing command_id back in result — resolvers always
  got undefined, causing every workflow step to timeout and fail

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-17 19:06:02 -04:00
jared 3f6e04d1ab Apply LotusGuild design system convergence (aesthetic_diff.md)
- §5: Section headers now ╠═══ TITLE ═══╣ (was ═══ TITLE ═══)
- §8+§18: Replace inline-style showTerminalNotification() with lt.toast.*
  delegate wrapper; load base.js from /base.js
- §12: Fix --text-muted #008822→#00bb33 (WCAG AA contrast)

base.js symlinked from web_template into public/ so lt.* is available.
showTerminalNotification() is kept as a thin wrapper so all existing
call sites continue to work unchanged.

README: Remove completed pending items (toast, text-muted, position)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-14 21:40:36 -04:00
jared 18a6b642d1 Fix all-workers-offline silent success bug, noisy logging, UX polish
server.js:
- Fix bug: when all targeted workers disconnect before step runs, results[] was empty
  and results.every() returned true vacuously (silent false success). Now tracks sentCount
  and fails with 'no_workers' log if nothing was actually dispatched
- Remove per-message console.log on every WebSocket message (high noise)
- Only log a warning for failed commands (not every success)

index.html:
- loadSchedules() catch now shows error message in scheduleList (was silent)
- abortExecution() shows server's error message from JSON body instead of generic string
  (e.g. "Execution is not running" instead of "Failed to abort execution")

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 13:48:35 -04:00
jared 8ff3700601 Fix XSS, add per-step timeout/retry to workflow engine
index.html:
- Escape started_by in execution list, execution cards, and execution detail modal
- Escape schedule name in schedules list

server.js:
- Per-step timeout: step.timeout (seconds) overrides global COMMAND_TIMEOUT_MS (5s-600s range)
- Per-step retry: step.retries (max 5) with step.retryDelayMs (max 30s) re-sends command on failure
  with command_retry log entries showing attempt/max_retries progress

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 17:36:35 -04:00
jared ba5ba6f899 Security and reliability fixes: XSS, race conditions, logging
- Fix XSS in workflow list: escape name/description/created_by with escapeHtml()
- Fix broadcast() race: snapshot browserClients Set before iterating
- Fix waitForCommandResult: use position-based matching (worker omits command_id in result)
- Fix scheduler duplicate-run race: atomic UPDATE with affectedRows check
- Suppress toast alerts for automated executions (gandalf:/scheduler: prefix)
- Add waiting_for_input + prompt fields to GET /executions/:id
- Add GET /api/workflows/:id endpoint
- Add interactive prompt UI: respondToPrompt(), WebSocket execution_prompt handler
- Add workflow editor modal (admin-only)
- Remove sensitive console.log calls (WS payload, command output)
- Show error messages in list containers on load failure
- evalCondition: log warning instead of silently swallowing errors
- Worker reconnect: clean up stale entries for same dbWorkerId
- Null-check execState before continuing workflow steps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 17:30:32 -04:00
jared a596c6075c Fix ESC handler, form reset, and scheduler next-run countdown
- Fix ESC key handler to use .modal.show class selector instead of style.display check
- Reset create workflow form fields when opening the create modal
- Show relative countdown (e.g. "in 5m") alongside next run timestamp in scheduler list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 11:26:48 -04:00
jared 95f6554cc2 Frontend improvements: safety, UX, and WebSocket handling
- Guard all new Date().toLocaleString() calls with safeDate() to prevent 'Invalid Date'
- Guard escapeHtml() for null/undefined input
- Guard getTimeAgo() for null/NaN dates; add safeDate() and formatElapsed() helpers
- Show elapsed time for running executions in the execution list
- Add status-running CSS pulse animation class to running execution items
- Add explicit executions_bulk_deleted WebSocket handler
- clearCompletedExecutions() uses new bulk DELETE endpoint instead of N individual requests
- switchTab() persists active tab to localStorage; init restores it on load
- refreshData() updates lastRefreshed timestamp in header
- Add Ctrl+Enter shortcut for quick command form
- Wrap rerunCommand worker_id with escapeHtml() to prevent XSS

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 11:24:34 -04:00
jared ba75a61bd2 Security hardening, bug fixes, and backend improvements
Security:
- validateWebhookUrl() rejects non-http/https and private/internal IPs
- Validate webhook URL on workflow create and update
- Replace error.message in all HTTP 500 responses with 'Internal server error'
- Add requireJSON middleware (HTTP 415 if Content-Type wrong) on POST/PUT routes
- Reject missing API keys in worker heartbeat (not just wrong ones)
- Validate prompt response against allowed options before accepting

Bugs fixed:
- goto infinite loop protection: stepVisits[] counter, fails at GOTO_MAX_VISITS (100)
- wait step: validate duration (no NaN/negative), cap at WAIT_STEP_MAX_MS (24h)
- _executionPrompts now stores {resolve, options} for option validation
- JSON.parse wrapped in try/catch: workflows/:id, executions/:id, internal/executions/:id, POST /api/executions, scheduler worker_ids
- pong handler uses ws.dbWorkerId (set on connect) not message.worker_id
- Worker disconnect now marks worker offline in DB and broadcasts update
- command validation (type + empty check) on POST /api/workers/:id/command
- workflow_id required check on POST /api/executions

Performance & reliability:
- markStaleWorkersOffline() runs every 60s, marks workers without recent heartbeat offline
- Named constants: PROMPT_TIMEOUT_MS, COMMAND_TIMEOUT_MS, QUICK_CMD_TIMEOUT_MS,
  WEBHOOK_TIMEOUT_MS, WAIT_STEP_MAX_MS, GOTO_MAX_VISITS, WORKER_STALE_MINUTES

New features:
- GET /api/health (auth required): version, uptime, worker counts
- DELETE /api/executions/completed: bulk delete finished executions (admin)
- schedule_value positive-integer validation for interval/hourly schedule types
- Request logging middleware: [HTTP] METHOD /path STATUS Xms

Code quality:
- All console.log on error paths changed to console.error
- Removed stray debug console.log in POST /api/workflows

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 10:59:07 -04:00
jared 658caa9f7e Fix bracket wrapping to new lines via inline-flex on buttons
The CSS ::before/::after pseudo-elements for [ ] brackets were rendering
on separate lines because button HTML had multiline whitespace (newlines +
indentation) inside the tag, causing:
  [
    Button Text
  ]

Fix: set button display to inline-flex with align-items:center so
pseudo-elements become flex children that stay on the same line
as button content regardless of internal whitespace. Also add
white-space:nowrap and flex-shrink:0 on pseudo-elements.

Also fix compareBtn.style.display to use inline-flex to avoid
reverting to block-level display that would re-introduce wrapping.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 23:27:00 -04:00
jared af63fbb1de Fix remaining [[ ]] double-bracket bugs on Templates, History, and sub-tab buttons
- Remove manual [ ] from sub-tab buttons (Manual Runs, Automated)
- Add CSS to suppress button::before/after pseudo-elements on .tab
  buttons and border:none inline-styled buttons so they don't get
  double-bracketed
- Prevents [[ text ]] from appearing on Templates/History/sub-tabs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 23:23:58 -04:00
jared d7b26c2b70 Fix [[ ]] visual bug and add missing log type handlers
Root cause: CSS button::before/after adds [ ] universally, but many
buttons had hardcoded [ text ] content, producing [[ text ]].

- Strip manual [ ] wrappers from all button text in HTML and JS
- Fix JS textContent assignments for compare mode buttons
- Fix dynamic button HTML strings in execution details panel
- Add formatLogEntry handlers for previously unhandled action types:
  dry_run_skipped, execution_timeout, goto_error, step_error,
  workflow_result, params, server_restart_recovery
- Unknown log actions now show action name instead of raw JSON
- Add cron schedule type display in schedules list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 23:18:18 -04:00
jared 2d6a0f1054 Add rate limiting, cron scheduling, webhooks, dry-run, execution filtering, and UX improvements
- Rate limiting: 300 req/15min general, 20 req/min on POST /api/executions
- Cron schedule type support using cron-parser for full cron expressions
- Webhook notifications: POST to workflow webhook_url on execution complete/failed
- Dry-run mode: simulate workflow execution without running any commands
- Global execution timeout via EXECUTION_MAX_MINUTES env var (default 60min)
- Execution filtering: status, workflow_id, started_by, after, before, search
- Event-driven command result delivery (replaces 500ms DB polling)
- Atomic log appends via JSON_ARRAY_APPEND (no read-modify-write race)
- Separate browserClients/workerClients sets (workers no longer receive broadcasts)
- Stale execution cleanup on startup (mark running→failed after crash)
- Scheduler overlap prevention (skip if same workflow already running)
- Frontend: webhook_url field in create/edit workflow modals
- Frontend: dry-run checkbox in workflow param modal
- Frontend: ESC closes modals, ws.onerror handler added
- Frontend: selectedExecutions changed from Array to Set (O(1) ops)
- Frontend: XSS fixes via escapeHtml() on all user-controlled innerHTML
- Frontend: param modal keydown listener deduplication fix
- Remove unused npm packages (bcryptjs, body-parser, cors, js-yaml, jsonwebtoken)
- Add express-rate-limit and cron-parser dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 23:06:09 -04:00
jared 58c172e131 Security hardening, bug fixes, and performance improvements
Security fixes:
- Replace new Function() condition eval with vm.runInNewContext() (RCE fix)
- Add admin checks to DELETE executions, all scheduled-commands endpoints
- Remove api_key from GET /api/workers response (was exposed to all employees)
- Separate browserClients/workerClients sets; broadcast() now sends to browsers only
- Add worker WebSocket auth: reject if api_key provided but invalid
- Fix XSS: escapeHtml() on step_name, duration, worker_id, user info, execution_id

Bug fixes:
- Replace DB-polling waitForCommandResult with event-driven _commandResolvers Map
- Replace non-atomic addExecutionLog with JSON_ARRAY_APPEND (fixes concurrent write race)
- Add stale execution recovery on startup: running→failed with log entry
- Fix calculateNextRun returning null for unknown types (now throws)
- Fix scheduler overlap: skip if previous execution still running
- Fix JSON double-parse on worker_ids column
- Fix switchTab() bare event.target reference
- Fix selectedExecutions Array→Set (O(1) lookups, fixes performance regression)
- Fix param modal event listener leak (delegated handler, removes before re-adding)
- Add ws.onerror handler (was silently swallowing WebSocket errors)
- Move misplaced routes to before server.listen()

Performance/cleanup:
- DB connection pool 10→50
- EXECUTION_RETENTION_DAYS default 1→30 (matches docs)
- Remove unused packages: bcryptjs, body-parser, cors, js-yaml, jsonwebtoken
- Remove generateUUID() wrapper, use crypto.randomUUID() directly
- Remove dead example workflow constants
- Add ESC key handler to close modals
- Fix clearCompletedExecutions limit 1000→9999
- Add security notice to README.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 22:53:25 -04:00
jared 0fee118d1d Suppress toast notifications for automated executions
Automated executions (started_by gandalf: or scheduler:) no longer
trigger success/failure toast alerts for connected browser users.
Server now includes is_automated flag in command_result broadcasts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 16:26:18 -05:00
jared 937bddbe2f Fix waitForCommandResult: match by position, not command_id
Worker does not echo command_id back in command_result message.
Previously this caused all workflow steps to time out after 120s.
Now: find the command_sent entry for the commandId, then take the
next command_result after it — safe since steps run sequentially.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 11:21:30 -05:00
jared bf9b14bc96 Add interactive workflow system with prompt steps, workflow editor
- executeWorkflowSteps: rewritten with step-id/goto branching support
- executePromptStep: async pause via _executionPrompts Map, 60min timeout
- POST /api/executions/:id/respond: resolves pending prompt from browser
- PUT /api/workflows/🆔 admin-only workflow editing, broadcasts workflow_updated
- GET /api/workflows/🆔 fetch single workflow for edit modal
- GET /api/executions/🆔 now includes waiting_for_input + prompt fields
- index.html: prompt/prompt_response/step_skipped log entry rendering
- index.html: execution_prompt WebSocket handler refreshes open modal
- index.html: workflow_updated WebSocket handler reloads workflow list
- index.html: Edit button + modal for in-browser workflow editing
- index.html: respondToPrompt keeps modal open, refreshes execution view
- Interactive Link Troubleshooter v2 workflow: 45-step wizard with
  copper/fiber branches, clean/swap/reseat actions, re-test loops,
  CRC error path, performance diagnostics, SUCCESS/ESCALATE terminals

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:55:02 -05:00
jared 033237482d feat: workflow param substitution + link troubleshooter support
Adds {{param_name}} template substitution to the workflow execution engine
so workflows can accept user-supplied inputs at run time.

server.js:
- applyParams() helper — substitutes {{name}} in command strings with
  validated values (alphanumeric + safe punctuation only); unsafe values
  throw and fail the execution cleanly
- executeCommandStep() / executeWorkflowSteps() — accept params={} and
  apply substitution before dispatching commands to workers
- POST /api/executions — accepts params:{} from client; validates required
  params against definition.params[]; logs params in initial execution log

index.html:
- loadWorkflows() caches definition in _workflowRegistry keyed by id;
  shows "[N params]" badge on parameterised workflows
- executeWorkflow() checks for definition.params; if present, shows
  param input modal instead of plain confirm()
- showParamModal() — builds labelled input form from param definitions,
  marks required fields, focuses first input, Enter submits
- submitParamForm() — validates required fields, calls startExecution()
- startExecution() — POSTs {workflow_id, params} and switches to executions tab
- Param input modal — terminal-aesthetic overlay, no external dependencies

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:20:05 -05:00
jared 6d945a1913 feat: Gandalf M2M API, manual/automated execution sub-tabs, cleanup tuning
- server.js: add authenticateGandalf middleware (X-Gandalf-API-Key header)
  and two internal endpoints used by Gandalf link diagnostics:
    POST /api/internal/command  — submit SSH command to a worker, returns execution_id
    GET  /api/internal/executions/:id — poll execution status/logs
  Also tag automated executions as started_by 'gandalf:*' / 'scheduler:*';
  add hide_internal query param to GET /api/executions; change cleanup
  from daily/30d to hourly/1d to keep execution history lean
- index.html: add Manual / Automated sub-tabs on Execution History tab so
  Gandalf diagnostic runs don't clutter the manual run view; persists
  selected tab to localStorage; dashboard recent-run strip filters to
  manual runs only; sub-tabs show live counts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 16:04:22 -05:00
15 changed files with 6502 additions and 689 deletions
+13
View File
@@ -0,0 +1,13 @@
{
"env": { "node": true, "es2021": true },
"extends": "eslint:recommended",
"parserOptions": { "ecmaVersion": 2021, "sourceType": "commonjs" },
"rules": {
"no-unused-vars": "warn",
"no-empty": "warn",
"no-constant-condition": "warn",
"no-useless-escape": "warn",
"semi": ["error", "always"],
"eqeqeq": "warn"
}
}
+71
View File
@@ -0,0 +1,71 @@
name: Lint
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
jobs:
js-lint:
name: JS (eslint)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install ESLint
run: npm install --save-dev eslint@8
- name: Run ESLint
run: npx eslint --ext .js .
notify-failure:
name: Notify on failure
runs-on: ubuntu-latest
needs: [js-lint]
if: failure() && github.event_name == 'push'
steps:
- name: Send Matrix alert
env:
MATRIX_WEBHOOK_URL: ${{ secrets.MATRIX_WEBHOOK_URL }}
REPO: ${{ github.repository }}
BRANCH: ${{ github.ref_name }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
if [ -z "$MATRIX_WEBHOOK_URL" ] || [ "$MATRIX_WEBHOOK_URL" = "CONFIGURE_ME" ]; then exit 0; fi
curl -sf -X POST "$MATRIX_WEBHOOK_URL" \
-H "Content-Type: application/json" \
-d "{\"text\":\"CI FAILED: ${REPO} @ ${BRANCH} — ${RUN_URL}\"}"
deploy:
name: Deploy
runs-on: ubuntu-latest
needs: [js-lint]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: write
steps:
- name: Trigger webhook
env:
WEBHOOK_SECRET: ${{ secrets.WEBHOOK_SECRET }}
GIT_REF: ${{ github.ref }}
run: |
PAYLOAD="{\"ref\":\"${GIT_REF}\"}"
SIG=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$WEBHOOK_SECRET" | awk '{print $2}')
curl -sf --connect-timeout 10 \
-X POST \
-H "Content-Type: application/json" \
-H "X-Gitea-Signature: ${SIG}" \
-d "$PAYLOAD" \
"http://10.10.10.65:9000/hooks/pulse-deploy"
- name: Tag deployed commit
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="deploy-$(date -u +%Y.%m.%d)-${{ github.run_number }}"
curl -sf -X POST \
-H "Authorization: token $GITHUB_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"tag_name\":\"${TAG}\",\"target\":\"${{ github.sha }}\",\"message\":\"Deployed to production\"}" \
"https://code.lotusguild.org/api/v1/repos/${{ github.repository }}/tags"
+22
View File
@@ -0,0 +1,22 @@
name: Security
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
schedule:
- cron: '0 6 * * 1'
jobs:
npm-audit:
name: JS Security (npm audit)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: npm install
- name: Run npm audit
run: npm audit --audit-level=high
+20
View File
@@ -0,0 +1,20 @@
name: Test
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
jobs:
jest:
name: JS Tests (jest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: npm install
- name: Run jest
run: npm test
+65 -5
View File
@@ -1,7 +1,29 @@
# PULSE - Pipelined Unified Logic & Server Engine
[![Lint](https://code.lotusguild.org/LotusGuild/pulse/actions/workflows/lint.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/pulse/actions?workflow=lint.yml)
[![Test](https://code.lotusguild.org/LotusGuild/pulse/actions/workflows/test.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/pulse/actions?workflow=test.yml)
[![Security](https://code.lotusguild.org/LotusGuild/pulse/actions/workflows/security.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/pulse/actions?workflow=security.yml)
A distributed workflow orchestration platform for managing and executing complex multi-step operations across server clusters through a retro terminal-themed web interface.
> **Security Notice:** This repository is hosted on Gitea and is version-controlled. **Never commit secrets, credentials, passwords, API keys, or any sensitive information to this repo.** All sensitive configuration belongs exclusively in `.env` files which are listed in `.gitignore` and must never be committed. This includes database passwords, worker API keys, webhook secrets, and internal IP details.
**Design System**: [web_template](https://code.lotusguild.org/LotusGuild/web_template) — shared CSS, JS, and layout patterns for all LotusGuild apps
## Styling & Layout
PULSE uses the **LotusGuild Terminal Design System**. For all styling, component, and layout documentation see:
- [`web_template/README.md`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/README.md) — full component reference, CSS variables, JS API
- [`web_template/base.css`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/base.css) — unified CSS (`.lt-*` classes)
- [`web_template/base.js`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/base.js) — `window.lt` utilities (toast, modal, WebSocket helpers, fetch)
- [`web_template/aesthetic_diff.md`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/aesthetic_diff.md) — cross-app divergence analysis and convergence guide
- [`web_template/node/middleware.js`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/node/middleware.js) — Express auth, CSRF, CSP nonce middleware
**Pending convergence items (see aesthetic_diff.md):**
- Extract inline `<style>` from `public/index.html` into `public/style.css` and extend `base.css`
- Use `lt.autoRefresh.start(refreshData, 30000)` instead of raw `setInterval`
## Overview
PULSE is a centralized workflow execution system designed to orchestrate operations across distributed infrastructure. It provides a powerful web-based interface with a vintage CRT terminal aesthetic for defining, managing, and executing workflows that can span multiple servers, require human interaction, and perform complex automation tasks at scale.
@@ -305,12 +327,34 @@ MAX_CONCURRENT_TASKS=5 # Max parallel tasks (default: 5)
PULSE uses MariaDB with the following tables:
- **users**: User accounts from Authelia SSO
- **workers**: Worker node registry with metadata
- **workflows**: Workflow definitions (JSON)
- **executions**: Execution history with logs
| Table | Purpose |
|-------|---------|
| `users` | User accounts synced from Authelia SSO |
| `workers` | Worker node registry with connection metadata |
| `workflows` | Workflow definitions stored as JSON |
| `executions` | Execution history with logs, status, and timestamps |
See [Claude.md](Claude.md) for complete schema details.
### `executions` Table Key Columns
| Column | Description |
|--------|-------------|
| `id` | Auto-increment primary key |
| `worker_id` | Foreign key to workers |
| `command` | The command that was executed |
| `status` | `running`, `completed`, `failed` |
| `output` | Command output / log (JSON or text) |
| `created_at` | Execution start timestamp |
| `completed_at` | Execution end timestamp |
### `workers` Table Key Columns
| Column | Description |
|--------|-------------|
| `id` | Auto-increment primary key |
| `name` | Worker name (from `WORKER_NAME` env) |
| `last_seen` | Last heartbeat timestamp |
| `status` | `online`, `offline` |
| `metadata` | JSON blob of system info |
## Troubleshooting
@@ -370,6 +414,22 @@ MIT License - See LICENSE file for details
---
## CI / CD
| Workflow | Purpose | Triggers |
|---|---|---|
| `lint.yml` | ESLint on all `.js` files | Every push and PR |
| `test.yml` | Jest unit tests (`lib/utils.js`) | Every push and PR |
| `security.yml` | `npm audit --audit-level=high` | Every push, PR, and weekly Monday 6am |
| `deploy` job in `lint.yml` | Calls the `pulse-deploy` webhook on CT122 (10.10.10.65) to pull + restart | Push to `main` only, after lint passes |
Branch protection is enabled on `main` — the `lint.yml` check must pass before any PR can merge.
Tests live in `tests/utils.test.js` and cover the pure utility functions in `lib/utils.js`:
`validateWebhookUrl`, `applyParams`, `evalCondition`, `calculateNextRun`.
---
**PULSE** - Orchestrating your infrastructure, one heartbeat at a time. ⚡
Built with retro terminal aesthetics 🖥️ | Powered by WebSockets 🔌 | Secured by Authelia 🔐
+90
View File
@@ -0,0 +1,90 @@
'use strict';
/**
* Validate a webhook URL.
* Returns { ok: true, url } or { ok: false, reason }.
*/
function validateWebhookUrl(raw) {
if (!raw) return { ok: true, url: null };
let url;
try { url = new URL(raw); } catch { return { ok: false, reason: 'Invalid URL format' }; }
if (!['http:', 'https:'].includes(url.protocol)) {
return { ok: false, reason: 'Webhook URL must use http or https' };
}
const host = url.hostname.toLowerCase();
if (
host === 'localhost' ||
host === '::1' ||
/^127\./.test(host) ||
/^10\./.test(host) ||
/^192\.168\./.test(host) ||
/^172\.(1[6-9]|2\d|3[01])\./.test(host) ||
/^169\.254\./.test(host) ||
/^fe80:/i.test(host)
) {
return { ok: false, reason: 'Webhook URL must not point to a private/internal address' };
}
return { ok: true, url };
}
/**
* Replace {{param}} placeholders in a command string.
* Throws if a substituted value contains unsafe characters.
*/
function applyParams(command, params) {
return command.replace(/\{\{(\w+)\}\}/g, (match, key) => {
if (!(key in params)) return match;
const val = String(params[key]).trim();
if (!/^[a-zA-Z0-9._:@/-]+$/.test(val)) {
throw new Error(`Unsafe value for workflow parameter "${key}"`);
}
return val;
});
}
/**
* Evaluate a condition expression safely using vm.runInNewContext.
* Returns boolean (false on error).
*/
function evalCondition(condition, state, params) {
const vm = require('vm');
try {
const context = vm.createContext({ state, params, promptResponse: state.promptResponse });
return !!vm.runInNewContext(condition, context, { timeout: 100 });
} catch (e) {
console.warn(`[Workflow] evalCondition error (treated as false): ${e.message} — condition: ${condition}`);
return false;
}
}
/**
* Calculate the next run date for a scheduled command.
*/
function calculateNextRun(scheduleType, scheduleValue) {
const cronParser = require('cron-parser');
const now = new Date();
if (scheduleType === 'interval') {
const minutes = parseInt(scheduleValue);
if (isNaN(minutes) || minutes <= 0) throw new Error(`Invalid interval value: ${scheduleValue}`);
return new Date(now.getTime() + minutes * 60000);
} else if (scheduleType === 'daily') {
const [hours, minutes] = scheduleValue.split(':').map(Number);
if (isNaN(hours) || isNaN(minutes)) throw new Error(`Invalid daily time format: ${scheduleValue}`);
const next = new Date(now);
next.setHours(hours, minutes, 0, 0);
if (next <= now) next.setDate(next.getDate() + 1);
return next;
} else if (scheduleType === 'hourly') {
const hours = parseInt(scheduleValue);
if (isNaN(hours) || hours <= 0) throw new Error(`Invalid hourly value: ${scheduleValue}`);
return new Date(now.getTime() + hours * 3600000);
} else if (scheduleType === 'cron') {
const interval = cronParser.CronExpressionParser.parse(scheduleValue, { currentDate: now });
return interval.next().toDate();
}
throw new Error(`Unknown schedule type: ${scheduleType}`);
}
module.exports = { validateWebhookUrl, applyParams, evalCondition, calculateNextRun };
+4130 -132
View File
File diff suppressed because it is too large Load Diff
+7 -6
View File
@@ -3,21 +3,22 @@
"version": "1.0.0",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
"test": "jest --coverage"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"bcryptjs": "^3.0.3",
"body-parser": "^2.2.1",
"cors": "^2.8.5",
"cron-parser": "^5.5.0",
"dotenv": "^17.2.3",
"express": "^5.1.0",
"js-yaml": "^4.1.1",
"jsonwebtoken": "^9.0.2",
"express-rate-limit": "^8.3.1",
"mysql2": "^3.15.3",
"ws": "^8.18.3"
},
"devDependencies": {
"eslint": "^8.57.1",
"jest": "^29.7.0"
}
}
+11
View File
@@ -0,0 +1,11 @@
{
"env": { "browser": true, "es2021": true },
"parserOptions": { "ecmaVersion": 2021, "sourceType": "script" },
"rules": {
"no-unused-vars": "warn",
"no-empty": "warn",
"no-inner-declarations": "warn",
"semi": ["error", "always"],
"eqeqeq": "warn"
}
}
+1
View File
@@ -0,0 +1 @@
/root/code/web_template/base.js
+745 -223
View File
File diff suppressed because it is too large Load Diff
+900 -323
View File
File diff suppressed because it is too large Load Diff
+7
View File
@@ -0,0 +1,7 @@
{
"env": {
"node": true,
"jest": true,
"es2021": true
}
}
+163
View File
@@ -0,0 +1,163 @@
'use strict';
const { validateWebhookUrl, applyParams, evalCondition, calculateNextRun } = require('../lib/utils');
// ── validateWebhookUrl ──────────────────────────────────────────────────────
describe('validateWebhookUrl', () => {
test('null/empty returns ok with null url', () => {
expect(validateWebhookUrl(null)).toEqual({ ok: true, url: null });
expect(validateWebhookUrl('')).toEqual({ ok: true, url: null });
});
test('valid public https URL is accepted', () => {
const result = validateWebhookUrl('https://hooks.example.com/notify');
expect(result.ok).toBe(true);
expect(result.url).toBeInstanceOf(URL);
});
test('valid public http URL is accepted', () => {
const result = validateWebhookUrl('http://webhook.example.com/event');
expect(result.ok).toBe(true);
});
test('invalid URL format is rejected', () => {
expect(validateWebhookUrl('not a url')).toMatchObject({ ok: false, reason: expect.stringContaining('Invalid URL') });
});
test('non-http protocol is rejected', () => {
expect(validateWebhookUrl('ftp://example.com/hook')).toMatchObject({ ok: false, reason: expect.stringContaining('http') });
});
test('localhost is rejected', () => {
expect(validateWebhookUrl('http://localhost/hook')).toMatchObject({ ok: false });
});
test('10.x.x.x private range is rejected', () => {
expect(validateWebhookUrl('http://10.0.0.1/hook')).toMatchObject({ ok: false });
expect(validateWebhookUrl('http://10.10.10.45/hook')).toMatchObject({ ok: false });
});
test('192.168.x.x private range is rejected', () => {
expect(validateWebhookUrl('http://192.168.1.1/hook')).toMatchObject({ ok: false });
});
test('127.0.0.1 loopback is rejected', () => {
expect(validateWebhookUrl('http://127.0.0.1/hook')).toMatchObject({ ok: false });
});
test('172.16.x.x 172.31.x.x private range is rejected', () => {
expect(validateWebhookUrl('http://172.16.0.1/hook')).toMatchObject({ ok: false });
expect(validateWebhookUrl('http://172.31.255.255/hook')).toMatchObject({ ok: false });
});
test('172.32.x.x (outside private range) is accepted', () => {
const result = validateWebhookUrl('http://172.32.0.1/hook');
expect(result.ok).toBe(true);
});
});
// ── applyParams ─────────────────────────────────────────────────────────────
describe('applyParams', () => {
test('replaces a single placeholder', () => {
expect(applyParams('echo {{msg}}', { msg: 'hello' })).toBe('echo hello');
});
test('replaces multiple placeholders', () => {
expect(applyParams('cp {{src}} {{dst}}', { src: 'a.txt', dst: 'b.txt' })).toBe('cp a.txt b.txt');
});
test('leaves unknown placeholders unchanged', () => {
expect(applyParams('echo {{unknown}}', {})).toBe('echo {{unknown}}');
});
test('throws on unsafe param value with spaces', () => {
expect(() => applyParams('echo {{x}}', { x: 'rm -rf /' })).toThrow('Unsafe value');
});
test('throws on unsafe param value with semicolons', () => {
expect(() => applyParams('echo {{x}}', { x: 'a;b' })).toThrow('Unsafe value');
});
test('allows safe characters: dots, dashes, slashes, colons, @', () => {
expect(applyParams('ssh {{host}}', { host: 'user@10.0.0.1:22' })).toBe('ssh user@10.0.0.1:22');
expect(applyParams('cat {{path}}', { path: '/etc/hosts' })).toBe('cat /etc/hosts');
});
test('trims whitespace from param values', () => {
expect(applyParams('echo {{x}}', { x: ' hello ' })).toBe('echo hello');
});
});
// ── evalCondition ───────────────────────────────────────────────────────────
describe('evalCondition', () => {
test('evaluates truthy expression', () => {
expect(evalCondition('1 === 1', {}, {})).toBe(true);
});
test('evaluates falsy expression', () => {
expect(evalCondition('1 === 2', {}, {})).toBe(false);
});
test('accesses state variables', () => {
expect(evalCondition('state.count > 5', { count: 10 }, {})).toBe(true);
expect(evalCondition('state.count > 5', { count: 3 }, {})).toBe(false);
});
test('accesses params', () => {
expect(evalCondition('params.env === "prod"', {}, { env: 'prod' })).toBe(true);
});
test('returns false on syntax error (does not throw)', () => {
expect(evalCondition('this is not valid js !!!', {}, {})).toBe(false);
});
test('returns false on timeout / infinite loop', () => {
expect(evalCondition('while(true){}', {}, {})).toBe(false);
});
});
// ── calculateNextRun ────────────────────────────────────────────────────────
describe('calculateNextRun', () => {
test('interval: returns a date ~N minutes in the future', () => {
const before = Date.now();
const result = calculateNextRun('interval', '30');
expect(result.getTime()).toBeGreaterThanOrEqual(before + 30 * 60000 - 100);
expect(result.getTime()).toBeLessThanOrEqual(before + 30 * 60000 + 1000);
});
test('interval: throws on non-positive value', () => {
expect(() => calculateNextRun('interval', '0')).toThrow();
expect(() => calculateNextRun('interval', '-5')).toThrow();
expect(() => calculateNextRun('interval', 'abc')).toThrow();
});
test('hourly: returns a date ~N hours in the future', () => {
const before = Date.now();
const result = calculateNextRun('hourly', '2');
expect(result.getTime()).toBeGreaterThanOrEqual(before + 2 * 3600000 - 100);
});
test('daily: returns a Date object', () => {
const result = calculateNextRun('daily', '06:00');
expect(result).toBeInstanceOf(Date);
expect(result.getTime()).toBeGreaterThan(Date.now());
});
test('daily: throws on invalid time format', () => {
expect(() => calculateNextRun('daily', 'noon')).toThrow();
});
test('cron: returns next occurrence after now', () => {
const result = calculateNextRun('cron', '0 6 * * 1');
expect(result).toBeInstanceOf(Date);
expect(result.getTime()).toBeGreaterThan(Date.now());
});
test('unknown type throws', () => {
expect(() => calculateNextRun('weekly', '1')).toThrow('Unknown schedule type');
});
});
+257
View File
@@ -0,0 +1,257 @@
const axios = require('axios');
const WebSocket = require('ws');
const { exec } = require('child_process');
const { promisify } = require('util');
const os = require('os');
const crypto = require('crypto');
require('dotenv').config();
const execAsync = promisify(exec);
class PulseWorker {
constructor() {
this.workerId = crypto.randomUUID();
this.workerName = process.env.WORKER_NAME || os.hostname();
this.serverUrl = process.env.PULSE_SERVER || 'http://localhost:8080';
this.wsUrl = process.env.PULSE_WS || 'ws://localhost:8080';
this.apiKey = process.env.WORKER_API_KEY;
this.heartbeatInterval = parseInt(process.env.HEARTBEAT_INTERVAL || '30') * 1000;
this.maxConcurrentTasks = parseInt(process.env.MAX_CONCURRENT_TASKS || '5');
this.activeTasks = 0;
this.ws = null;
this.heartbeatTimer = null;
}
async start() {
console.log(`[PULSE Worker] Starting worker: ${this.workerName}`);
console.log(`[PULSE Worker] Worker ID: ${this.workerId}`);
console.log(`[PULSE Worker] Server: ${this.serverUrl}`);
// Send initial heartbeat
await this.sendHeartbeat();
// Start heartbeat timer
this.startHeartbeat();
// Connect to WebSocket for real-time commands
this.connectWebSocket();
console.log(`[PULSE Worker] Worker started successfully`);
}
startHeartbeat() {
this.heartbeatTimer = setInterval(async () => {
try {
await this.sendHeartbeat();
} catch (error) {
console.error('[PULSE Worker] Heartbeat failed:', error.message);
}
}, this.heartbeatInterval);
}
async sendHeartbeat() {
const metadata = {
hostname: os.hostname(),
platform: os.platform(),
arch: os.arch(),
cpus: os.cpus().length,
totalMem: os.totalmem(),
freeMem: os.freemem(),
uptime: os.uptime(),
loadavg: os.loadavg(),
activeTasks: this.activeTasks,
maxConcurrentTasks: this.maxConcurrentTasks
};
try {
const response = await axios.post(
`${this.serverUrl}/api/workers/heartbeat`,
{
worker_id: this.workerId,
name: this.workerName,
metadata: metadata
},
{
headers: {
'X-API-Key': this.apiKey,
'Content-Type': 'application/json'
}
}
);
console.log(`[PULSE Worker] Heartbeat sent - Status: online`);
return response.data;
} catch (error) {
console.error('[PULSE Worker] Heartbeat error:', error.message);
throw error;
}
}
connectWebSocket() {
console.log(`[PULSE Worker] Connecting to WebSocket...`);
this.ws = new WebSocket(this.wsUrl);
this.ws.on('open', () => {
console.log('[PULSE Worker] WebSocket connected');
// Identify this worker
this.ws.send(JSON.stringify({
type: 'worker_connect',
worker_id: this.workerId,
worker_name: this.workerName
}));
});
this.ws.on('message', async (data) => {
try {
const message = JSON.parse(data.toString());
await this.handleMessage(message);
} catch (error) {
console.error('[PULSE Worker] Message handling error:', error);
}
});
this.ws.on('close', () => {
console.log('[PULSE Worker] WebSocket disconnected, reconnecting...');
setTimeout(() => this.connectWebSocket(), 5000);
});
this.ws.on('error', (error) => {
console.error('[PULSE Worker] WebSocket error:', error.message);
});
}
async handleMessage(message) {
console.log(`[PULSE Worker] Received message:`, message.type);
switch (message.type) {
case 'execute_command':
await this.executeCommand(message);
break;
case 'execute_workflow':
await this.executeWorkflow(message);
break;
case 'ping':
this.sendPong();
break;
default:
console.log(`[PULSE Worker] Unknown message type: ${message.type}`);
}
}
async executeCommand(message) {
const { command, execution_id, command_id, timeout = 300000 } = message;
if (this.activeTasks >= this.maxConcurrentTasks) {
console.log(`[PULSE Worker] Max concurrent tasks reached, rejecting command`);
return;
}
this.activeTasks++;
console.log(`[PULSE Worker] Executing command (active tasks: ${this.activeTasks})`);
try {
const startTime = Date.now();
const { stdout, stderr } = await execAsync(command, {
timeout: timeout,
maxBuffer: 10 * 1024 * 1024 // 10MB buffer
});
const duration = Date.now() - startTime;
const result = {
type: 'command_result',
execution_id,
worker_id: this.workerId,
command_id,
success: true,
stdout: stdout,
stderr: stderr,
duration: duration,
timestamp: new Date().toISOString()
};
this.sendResult(result);
console.log(`[PULSE Worker] Command completed in ${duration}ms`);
} catch (error) {
const result = {
type: 'command_result',
execution_id,
worker_id: this.workerId,
command_id,
success: false,
error: error.message,
stdout: error.stdout || '',
stderr: error.stderr || '',
timestamp: new Date().toISOString()
};
this.sendResult(result);
console.error(`[PULSE Worker] Command failed:`, error.message);
} finally {
this.activeTasks--;
}
}
async executeWorkflow(message) {
const { workflow, execution_id } = message;
console.log(`[PULSE Worker] Executing workflow: ${workflow.name}`);
// Workflow execution will be implemented in phase 2
// For now, just acknowledge receipt
this.sendResult({
type: 'workflow_result',
execution_id,
worker_id: this.workerId,
success: true,
message: 'Workflow execution not yet implemented',
timestamp: new Date().toISOString()
});
}
sendResult(result) {
if (this.ws && this.ws.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(result));
}
}
sendPong() {
if (this.ws && this.ws.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify({ type: 'pong', worker_id: this.workerId }));
}
}
async stop() {
console.log('[PULSE Worker] Shutting down...');
if (this.heartbeatTimer) {
clearInterval(this.heartbeatTimer);
}
if (this.ws) {
this.ws.close();
}
console.log('[PULSE Worker] Shutdown complete');
}
}
// Start worker
const worker = new PulseWorker();
// Handle graceful shutdown
process.on('SIGTERM', async () => {
await worker.stop();
process.exit(0);
});
process.on('SIGINT', async () => {
await worker.stop();
process.exit(0);
});
// Start the worker
worker.start().catch((error) => {
console.error('[PULSE Worker] Fatal error:', error);
process.exit(1);
});