Compare commits

..

31 Commits

Author SHA1 Message Date
jared 9fb9424cea Add abort execution feature for stuck/running processes 2026-01-08 22:11:59 -05:00
jared 500163d3f8 Fix execution details endpoint - remove reference to deleted activeExecutions 2026-01-08 22:06:25 -05:00
jared 3ea640350c Add missing helper functions for workflow execution (addExecutionLog, updateExecutionStatus) 2026-01-08 22:03:00 -05:00
jared 8721e47a4a Add WebSocket error handling with stack traces
Wrapped ws.onmessage in try-catch to capture full stack trace
when errors occur during message handling. This will help identify
where the 'Cannot read properties of undefined' error is coming from.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:33:07 -05:00
jared 025aa11f34 Add detailed logging to workflow creation endpoint
This will help diagnose the 'Cannot read properties of undefined' error
by logging each step of the workflow creation process.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:30:50 -05:00
jared f8b8427f77 Improve error handling in workflow creation and data loading
- Separated JSON validation from API call error handling
- Changed refreshData() to async with individual try-catch blocks
- Better error messages: "Invalid JSON" vs "Error creating workflow"
- Console.error logging for each data loading function
- Changed success alert to terminal notification
- This will help identify which specific function is failing

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:27:54 -05:00
jared 2f941d0b6f Fix: Remove duplicate old workflow execution code
Removed old/obsolete workflow execution system that was conflicting
with the new executeWorkflowSteps() engine:

Removed:
- activeExecutions Map (old tracking system)
- executeWorkflow() - old workflow executor
- executeNextStep() - old step processor
- executeCommandStep() - old command executor (duplicate)
- handleUserInput() - unimplemented prompt handler
- Duplicate app.post('/api/executions') endpoint
- app.post('/api/executions/:id/respond') endpoint

This was causing "Cannot read properties of undefined (reading 'target')"
error because the old code was being called instead of the new engine.

The new executeWorkflowSteps() engine is now the only workflow system.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:24:42 -05:00
jared 1b7a4db868 Implement Complete Workflow Execution Engine
Added full workflow execution engine that actually runs workflow steps:

Server-Side (server.js):
- executeWorkflowSteps() - Main workflow orchestration function
- executeCommandStep() - Executes commands on target workers
- waitForCommandResult() - Polls for command completion
- Support for step types: execute, wait, prompt (prompt skipped for now)
- Sequential step execution with failure handling
- Worker targeting: "all" or specific worker IDs/names
- Automatic status updates (running -> completed/failed)
- Real-time WebSocket broadcasts for step progress
- Command result tracking with command_id for workflows
- Only updates status for non-workflow quick commands

Client-Side (index.html):
- Enhanced formatLogEntry() with workflow-specific log types
- step_started - Shows step number and name with amber color
- step_completed - Shows completion with green checkmark
- waiting - Displays wait duration
- no_workers - Error when no workers available
- worker_offline - Warning for offline workers
- workflow_error - Critical workflow errors
- Better visual feedback for workflow progress

Workflow Definition Format:
{
  "steps": [
    {
      "name": "Step Name",
      "type": "execute",
      "targets": ["all"] or ["worker-name"],
      "command": "your command here"
    },
    {
      "type": "wait",
      "duration": 5
    }
  ]
}

Features:
- Executes steps sequentially
- Stops on first failure
- Supports multiple workers per step
- Real-time progress updates
- Comprehensive logging
- Terminal-themed workflow logs

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:19:12 -05:00
jared 56f8047322 Phase 10: Command Scheduler
Added comprehensive command scheduling system:

Backend:
- New scheduled_commands database table
- Scheduler processor runs every minute
- Support for three schedule types: interval, hourly, daily
- calculateNextRun() function for intelligent scheduling
- API endpoints: GET, POST, PUT (toggle), DELETE
- Executions automatically created and tracked
- Enable/disable schedules without deleting

Frontend:
- New Scheduler tab in navigation
- Create Schedule modal with worker selection
- Dynamic schedule input based on type
- Schedule list showing status, next/last run times
- Enable/Disable toggle for each schedule
- Delete schedule functionality
- Terminal-themed scheduler UI
- Integration with existing worker and execution systems

Schedule Types:
- Interval: Every X minutes (e.g., 30 for every 30 min)
- Hourly: Every X hours (e.g., 2 for every 2 hours)
- Daily: At specific time (e.g., 03:00 for 3 AM daily)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:13:27 -05:00
jared bf908568e3 Phase 9: Execution Diff View
Added powerful execution comparison and diff view:

- Compare Mode toggle button in executions tab
- Multi-select up to 5 executions for comparison
- Visual selection indicators with checkmarks
- Comparison modal with summary table (status, duration, timestamps)
- Side-by-side output view for all selected executions
- Line-by-line diff analysis for 2-execution comparisons
- Highlights identical vs. different lines
- Shows identical/different line counts
- Color-coded diff (green for exec 1, amber for exec 2)
- Perfect for comparing same command across workers
- Terminal-themed comparison UI

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:08:55 -05:00
jared ffa124ed9d Phase 8: Execution Search & Filtering
Added comprehensive search and filtering for execution history:

- Search bar to filter by command text, execution ID, or workflow name
- Status filter dropdown (All, Running, Completed, Failed, Waiting)
- Real-time client-side filtering as user types
- Filter statistics showing X of Y executions
- Clear Filters button to reset all filters
- Extracts command text from logs for quick command searches
- Maintains all executions in memory for instant filtering
- Terminal-themed filter UI matching existing aesthetic

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:06:43 -05:00
jared 84edea8027 Phase 7: Multi-Worker Command Execution
Added ability to execute commands on multiple workers simultaneously:

- Added execution mode selector (Single/Multiple Workers)
- Multi-worker mode with checkbox list for worker selection
- Helper buttons: Select All, Online Only, Clear All
- Sequential execution across selected workers
- Results summary showing success/fail count per worker
- Updated command history to track multi-worker executions
- Terminal beep feedback based on overall success/failure
- Maintained backward compatibility with single worker mode

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 23:03:45 -05:00
jared 095bfb65ab Fix clearCompletedExecutions for new pagination API format
Changes:
- Handle new pagination response format (data.executions vs data)
- Request up to 1000 executions to ensure all are checked
- Track successful deletions count
- Use terminal notification instead of alert
- Better error handling for individual delete failures

Fixes regression from Phase 5 pagination changes.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:55:13 -05:00
jared c619add705 Phase 6: Terminal aesthetic refinements and notifications
Changes:
- Added blinking terminal cursor animation
- Smooth hover effects for execution/worker/workflow items
- Hover animation: background highlight + border expand + slide
- Loading pulse animation for loading states
- Slide-in animation for log entries
- Terminal beep sound using Web Audio API (different tones for success/error)
- Real-time terminal notifications for command completion
- Toast-style notifications with green glow effects
- Auto-dismiss after 3 seconds with fade-out
- Visual and audio feedback for user actions

Sound features:
- 800Hz tone for success (higher pitch)
- 200Hz tone for errors (lower pitch)
- 440Hz tone for info (standard A note)
- 100ms duration, exponential fade-out
- Graceful fallback if Web Audio API not supported

Notification features:
- Fixed position top-right
- Terminal-themed styling with glow
- Color-coded: green for success, red for errors
- Icons: ✓ success, ✗ error, ℹ info
- Smooth animations (slide-in, fade-out)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:52:51 -05:00
jared e6a6b7e359 Phase 5: Auto-cleanup and pagination for executions
Changes:
Server-side:
- Added automatic cleanup of old executions (runs daily)
- Configurable retention period via EXECUTION_RETENTION_DAYS env var (default: 30 days)
- Cleanup runs on server startup and every 24 hours
- Only cleans completed/failed executions, keeps running ones
- Added pagination support to /api/executions endpoint
- Returns total count, limit, offset, and hasMore flag

Client-side:
- Implemented "Load More" button for execution pagination
- Loads 50 executions at a time
- Appends additional executions when "Load More" clicked
- Shows total execution count info
- Backward compatible with old API format

Benefits:
- Automatic database maintenance
- Prevents execution table from growing indefinitely
- Better performance with large execution histories
- User can browse all executions via pagination
- Configurable retention policy per deployment

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:50:39 -05:00
jared d25ba27f24 Phase 4: Execution detail enhancements with re-run and download
Changes:
- Added "Re-run Command" button to execution details modal
- Added "Download Logs" button to export execution data as JSON
- Re-run automatically switches to Quick Command tab and pre-fills form
- Download includes all execution metadata and logs
- Buttons only show for applicable execution types
- Terminal-themed button styling

Features:
- Re-run: Quickly repeat a previous command on same worker
- Download: Export execution logs for auditing/debugging
- JSON format includes: execution_id, status, timestamps, logs
- Filename includes execution ID and date for easy organization

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:49:20 -05:00
jared 76b0a6d0d3 Phase 3: Quick command enhancements with templates and history
Changes:
- Added command templates modal with 12 common system commands
- Added command history tracking (stored in localStorage)
- History saves last 50 commands with timestamp and worker name
- Template categories: system info, disk/memory, network, Docker, logs
- Click templates to auto-fill command field
- Click history items to reuse previous commands
- Terminal-themed modals with green/amber styling
- History persists across browser sessions

Templates included:
- System: uname, uptime, CPU info, processes
- Resources: df -h, free -h, memory usage
- Network: ip addr, active connections
- Docker: container list
- Logs: syslog tail, who is logged in, last logins

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:45:40 -05:00
jared adbbec2631 Phase 2: Enhanced worker status display with metadata
Changes:
- Show worker system metrics in dashboard and worker list
- Display CPU cores, memory usage, load average, uptime
- Added formatBytes() to display memory in human-readable format
- Added formatUptime() to show uptime as days/hours/minutes
- Added getTimeAgo() to show relative last-seen time
- Improved worker list with detailed metadata panel
- Show active tasks vs max concurrent tasks
- Terminal-themed styling for metadata display
- Amber labels for metadata fields

Benefits:
- See worker health at a glance
- Monitor resource usage (CPU, RAM, load)
- Track worker activity (active tasks)
- Better operational visibility

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:43:13 -05:00
jared 8b8d2c6312 Phase 1: Improve log display formatting
Changes:
- Added formatLogEntry() function to parse and format log entries
- Replaced raw JSON display with readable formatted logs
- Added specific formatting for command_sent and command_result logs
- Show timestamp, status, duration, stdout/stderr in organized layout
- Color-coded success (green) and failure (red) states
- Added scrollable output sections with max-height
- Syntax highlighting for command code blocks
- Terminal-themed styling with green/amber colors

Benefits:
- Much easier to read execution logs
- Clear visual distinction between sent/result logs
- Professional terminal aesthetic maintained
- Better UX for debugging command execution

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:41:29 -05:00
jared 1f5c84f327 Add execution cleanup functionality
Changes:
- Added DELETE /api/executions/:id endpoint
- Added "Clear Completed" button to Executions tab
- Deletes all completed and failed executions
- Broadcasts execution_deleted event to update all clients
- Shows count of deleted executions

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:36:51 -05:00
jared e03f8d6287 Fix worker ID mapping - use database ID for command routing
Problem:
- Workers generate random UUID on startup (runtime ID)
- Database stores workers with persistent IDs (database ID)
- UI sends commands using database ID
- Server couldn't find worker connection (stored by runtime ID)
- Result: 400 Bad Request "Worker not connected"

Solution:
- When worker connects, look up database ID by worker name
- Store WebSocket connection in Map using BOTH IDs:
  * Runtime ID (from worker_connect message)
  * Database ID (from database lookup by name)
- Commands from UI use database ID → finds correct WebSocket
- Cleanup both IDs when worker disconnects

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:29:23 -05:00
jared 2097b73404 Fix worker command execution and execution status updates
Changes:
- Removed duplicate /api/executions/:id endpoint that didn't parse logs
- Added workers Map to track worker_id -> WebSocket connection
- Store worker connections when they send worker_connect message
- Send commands to specific worker instead of broadcasting to all clients
- Clean up workers Map when worker disconnects
- Update execution status to completed/failed when command results arrive
- Add proper error handling when worker is not connected

Fixes:
- execution.logs.forEach is not a function (logs now properly parsed)
- Commands stuck in "running" status (now update to completed/failed)
- Commands not reaching workers (now sent to specific worker WebSocket)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:23:02 -05:00
jared 6d15e4d240 Remove database migrations after direct schema fixes
Changes:
- Removed all migration code from server.js
- Database schema fixed directly via MySQL:
  * Dropped users.role column (SSO only)
  * Dropped users.password column (SSO only)
  * Added executions.started_by column
  * Added workflows.created_by column
  * All tables now match expected schema
- Server startup will be faster without migrations

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:11:07 -05:00
jared 7896b40d91 Remove password column from users table
Changes:
- Drop password column from users table (SSO authentication only)
- PULSE uses Authelia SSO, not password-based authentication
- Fixes 500 error: Field 'password' doesn't have a default value

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 22:01:58 -05:00
jared e2dc371bfe Add last_login column to users table migration
Changes:
- Add last_login TIMESTAMP column to existing users table
- Complete the users table migration with all required columns
- Fixes 500 error: Unknown column 'last_login' in 'INSERT INTO'

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 20:33:04 -05:00
jared df0184facf Add migration to update users table schema
Changes:
- Add display_name, email, and groups columns to existing users table
- Handle MariaDB lack of IF NOT EXISTS in ALTER TABLE
- Gracefully skip columns that already exist
- Fixes 500 error when authenticating users

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 20:28:47 -05:00
jared a8be111e04 Allow NULL workflow_id in executions table for quick commands
Changes:
- Modified executions table schema to allow NULL workflow_id
- Removed foreign key constraint that prevented NULL values
- Added migration to update existing table structure
- Quick commands can now be stored without a workflow reference

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 20:27:02 -05:00
jared b3806545bd Fix quick command executions not appearing in execution tab
Changes:
- Create execution record in database when quick command is sent
- Store initial log entry with command details
- Broadcast execution_started event to update UI
- Display quick commands as "[Quick Command]" in execution list
- Fix worker communication to properly track all executions

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-07 20:24:11 -05:00
jared 2767087e27 Updated websocket handler 2026-01-07 20:20:18 -05:00
jared a1cf8ac90b updates aesthetic 2026-01-07 20:12:16 -05:00
jared 9e842624e1 Claude md file 2026-01-07 19:57:16 -05:00
16 changed files with 2057 additions and 6498 deletions
-13
View File
@@ -1,13 +0,0 @@
{
"env": { "node": true, "es2021": true },
"extends": "eslint:recommended",
"parserOptions": { "ecmaVersion": 2021, "sourceType": "commonjs" },
"rules": {
"no-unused-vars": "warn",
"no-empty": "warn",
"no-constant-condition": "warn",
"no-useless-escape": "warn",
"semi": ["error", "always"],
"eqeqeq": "warn"
}
}
-71
View File
@@ -1,71 +0,0 @@
name: Lint
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
jobs:
js-lint:
name: JS (eslint)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install ESLint
run: npm install --save-dev eslint@8
- name: Run ESLint
run: npx eslint --ext .js .
notify-failure:
name: Notify on failure
runs-on: ubuntu-latest
needs: [js-lint]
if: failure() && github.event_name == 'push'
steps:
- name: Send Matrix alert
env:
MATRIX_WEBHOOK_URL: ${{ secrets.MATRIX_WEBHOOK_URL }}
REPO: ${{ github.repository }}
BRANCH: ${{ github.ref_name }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
if [ -z "$MATRIX_WEBHOOK_URL" ] || [ "$MATRIX_WEBHOOK_URL" = "CONFIGURE_ME" ]; then exit 0; fi
curl -sf -X POST "$MATRIX_WEBHOOK_URL" \
-H "Content-Type: application/json" \
-d "{\"text\":\"CI FAILED: ${REPO} @ ${BRANCH} — ${RUN_URL}\"}"
deploy:
name: Deploy
runs-on: ubuntu-latest
needs: [js-lint]
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
permissions:
contents: write
steps:
- name: Trigger webhook
env:
WEBHOOK_SECRET: ${{ secrets.WEBHOOK_SECRET }}
GIT_REF: ${{ github.ref }}
run: |
PAYLOAD="{\"ref\":\"${GIT_REF}\"}"
SIG=$(echo -n "$PAYLOAD" | openssl dgst -sha256 -hmac "$WEBHOOK_SECRET" | awk '{print $2}')
curl -sf --connect-timeout 10 \
-X POST \
-H "Content-Type: application/json" \
-H "X-Gitea-Signature: ${SIG}" \
-d "$PAYLOAD" \
"http://10.10.10.65:9000/hooks/pulse-deploy"
- name: Tag deployed commit
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
TAG="deploy-$(date -u +%Y.%m.%d)-${{ github.run_number }}"
curl -sf -X POST \
-H "Authorization: token $GITHUB_TOKEN" \
-H "Content-Type: application/json" \
-d "{\"tag_name\":\"${TAG}\",\"target\":\"${{ github.sha }}\",\"message\":\"Deployed to production\"}" \
"https://code.lotusguild.org/api/v1/repos/${{ github.repository }}/tags"
-22
View File
@@ -1,22 +0,0 @@
name: Security
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
schedule:
- cron: '0 6 * * 1'
jobs:
npm-audit:
name: JS Security (npm audit)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: npm install
- name: Run npm audit
run: npm audit --audit-level=high
-20
View File
@@ -1,20 +0,0 @@
name: Test
on:
push:
branches: ["**"]
pull_request:
branches: ["**"]
jobs:
jest:
name: JS Tests (jest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install dependencies
run: npm install
- name: Run jest
run: npm test
+1372
View File
File diff suppressed because it is too large Load Diff
+5 -65
View File
@@ -1,29 +1,7 @@
# PULSE - Pipelined Unified Logic & Server Engine
[![Lint](https://code.lotusguild.org/LotusGuild/pulse/actions/workflows/lint.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/pulse/actions?workflow=lint.yml)
[![Test](https://code.lotusguild.org/LotusGuild/pulse/actions/workflows/test.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/pulse/actions?workflow=test.yml)
[![Security](https://code.lotusguild.org/LotusGuild/pulse/actions/workflows/security.yml/badge.svg)](https://code.lotusguild.org/LotusGuild/pulse/actions?workflow=security.yml)
A distributed workflow orchestration platform for managing and executing complex multi-step operations across server clusters through a retro terminal-themed web interface.
> **Security Notice:** This repository is hosted on Gitea and is version-controlled. **Never commit secrets, credentials, passwords, API keys, or any sensitive information to this repo.** All sensitive configuration belongs exclusively in `.env` files which are listed in `.gitignore` and must never be committed. This includes database passwords, worker API keys, webhook secrets, and internal IP details.
**Design System**: [web_template](https://code.lotusguild.org/LotusGuild/web_template) — shared CSS, JS, and layout patterns for all LotusGuild apps
## Styling & Layout
PULSE uses the **LotusGuild Terminal Design System**. For all styling, component, and layout documentation see:
- [`web_template/README.md`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/README.md) — full component reference, CSS variables, JS API
- [`web_template/base.css`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/base.css) — unified CSS (`.lt-*` classes)
- [`web_template/base.js`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/base.js) — `window.lt` utilities (toast, modal, WebSocket helpers, fetch)
- [`web_template/aesthetic_diff.md`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/aesthetic_diff.md) — cross-app divergence analysis and convergence guide
- [`web_template/node/middleware.js`](https://code.lotusguild.org/LotusGuild/web_template/src/branch/main/node/middleware.js) — Express auth, CSRF, CSP nonce middleware
**Pending convergence items (see aesthetic_diff.md):**
- Extract inline `<style>` from `public/index.html` into `public/style.css` and extend `base.css`
- Use `lt.autoRefresh.start(refreshData, 30000)` instead of raw `setInterval`
## Overview
PULSE is a centralized workflow execution system designed to orchestrate operations across distributed infrastructure. It provides a powerful web-based interface with a vintage CRT terminal aesthetic for defining, managing, and executing workflows that can span multiple servers, require human interaction, and perform complex automation tasks at scale.
@@ -327,34 +305,12 @@ MAX_CONCURRENT_TASKS=5 # Max parallel tasks (default: 5)
PULSE uses MariaDB with the following tables:
| Table | Purpose |
|-------|---------|
| `users` | User accounts synced from Authelia SSO |
| `workers` | Worker node registry with connection metadata |
| `workflows` | Workflow definitions stored as JSON |
| `executions` | Execution history with logs, status, and timestamps |
- **users**: User accounts from Authelia SSO
- **workers**: Worker node registry with metadata
- **workflows**: Workflow definitions (JSON)
- **executions**: Execution history with logs
### `executions` Table Key Columns
| Column | Description |
|--------|-------------|
| `id` | Auto-increment primary key |
| `worker_id` | Foreign key to workers |
| `command` | The command that was executed |
| `status` | `running`, `completed`, `failed` |
| `output` | Command output / log (JSON or text) |
| `created_at` | Execution start timestamp |
| `completed_at` | Execution end timestamp |
### `workers` Table Key Columns
| Column | Description |
|--------|-------------|
| `id` | Auto-increment primary key |
| `name` | Worker name (from `WORKER_NAME` env) |
| `last_seen` | Last heartbeat timestamp |
| `status` | `online`, `offline` |
| `metadata` | JSON blob of system info |
See [Claude.md](Claude.md) for complete schema details.
## Troubleshooting
@@ -414,22 +370,6 @@ MIT License - See LICENSE file for details
---
## CI / CD
| Workflow | Purpose | Triggers |
|---|---|---|
| `lint.yml` | ESLint on all `.js` files | Every push and PR |
| `test.yml` | Jest unit tests (`lib/utils.js`) | Every push and PR |
| `security.yml` | `npm audit --audit-level=high` | Every push, PR, and weekly Monday 6am |
| `deploy` job in `lint.yml` | Calls the `pulse-deploy` webhook on CT122 (10.10.10.65) to pull + restart | Push to `main` only, after lint passes |
Branch protection is enabled on `main` — the `lint.yml` check must pass before any PR can merge.
Tests live in `tests/utils.test.js` and cover the pure utility functions in `lib/utils.js`:
`validateWebhookUrl`, `applyParams`, `evalCondition`, `calculateNextRun`.
---
**PULSE** - Orchestrating your infrastructure, one heartbeat at a time. ⚡
Built with retro terminal aesthetics 🖥️ | Powered by WebSockets 🔌 | Secured by Authelia 🔐
-90
View File
@@ -1,90 +0,0 @@
'use strict';
/**
* Validate a webhook URL.
* Returns { ok: true, url } or { ok: false, reason }.
*/
function validateWebhookUrl(raw) {
if (!raw) return { ok: true, url: null };
let url;
try { url = new URL(raw); } catch { return { ok: false, reason: 'Invalid URL format' }; }
if (!['http:', 'https:'].includes(url.protocol)) {
return { ok: false, reason: 'Webhook URL must use http or https' };
}
const host = url.hostname.toLowerCase();
if (
host === 'localhost' ||
host === '::1' ||
/^127\./.test(host) ||
/^10\./.test(host) ||
/^192\.168\./.test(host) ||
/^172\.(1[6-9]|2\d|3[01])\./.test(host) ||
/^169\.254\./.test(host) ||
/^fe80:/i.test(host)
) {
return { ok: false, reason: 'Webhook URL must not point to a private/internal address' };
}
return { ok: true, url };
}
/**
* Replace {{param}} placeholders in a command string.
* Throws if a substituted value contains unsafe characters.
*/
function applyParams(command, params) {
return command.replace(/\{\{(\w+)\}\}/g, (match, key) => {
if (!(key in params)) return match;
const val = String(params[key]).trim();
if (!/^[a-zA-Z0-9._:@/-]+$/.test(val)) {
throw new Error(`Unsafe value for workflow parameter "${key}"`);
}
return val;
});
}
/**
* Evaluate a condition expression safely using vm.runInNewContext.
* Returns boolean (false on error).
*/
function evalCondition(condition, state, params) {
const vm = require('vm');
try {
const context = vm.createContext({ state, params, promptResponse: state.promptResponse });
return !!vm.runInNewContext(condition, context, { timeout: 100 });
} catch (e) {
console.warn(`[Workflow] evalCondition error (treated as false): ${e.message} — condition: ${condition}`);
return false;
}
}
/**
* Calculate the next run date for a scheduled command.
*/
function calculateNextRun(scheduleType, scheduleValue) {
const cronParser = require('cron-parser');
const now = new Date();
if (scheduleType === 'interval') {
const minutes = parseInt(scheduleValue);
if (isNaN(minutes) || minutes <= 0) throw new Error(`Invalid interval value: ${scheduleValue}`);
return new Date(now.getTime() + minutes * 60000);
} else if (scheduleType === 'daily') {
const [hours, minutes] = scheduleValue.split(':').map(Number);
if (isNaN(hours) || isNaN(minutes)) throw new Error(`Invalid daily time format: ${scheduleValue}`);
const next = new Date(now);
next.setHours(hours, minutes, 0, 0);
if (next <= now) next.setDate(next.getDate() + 1);
return next;
} else if (scheduleType === 'hourly') {
const hours = parseInt(scheduleValue);
if (isNaN(hours) || hours <= 0) throw new Error(`Invalid hourly value: ${scheduleValue}`);
return new Date(now.getTime() + hours * 3600000);
} else if (scheduleType === 'cron') {
const interval = cronParser.CronExpressionParser.parse(scheduleValue, { currentDate: now });
return interval.next().toDate();
}
throw new Error(`Unknown schedule type: ${scheduleType}`);
}
module.exports = { validateWebhookUrl, applyParams, evalCondition, calculateNextRun };
+135 -4133
View File
File diff suppressed because it is too large Load Diff
+6 -7
View File
@@ -3,22 +3,21 @@
"version": "1.0.0",
"main": "index.js",
"scripts": {
"test": "jest --coverage"
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"cron-parser": "^5.5.0",
"bcryptjs": "^3.0.3",
"body-parser": "^2.2.1",
"cors": "^2.8.5",
"dotenv": "^17.2.3",
"express": "^5.1.0",
"express-rate-limit": "^8.3.1",
"js-yaml": "^4.1.1",
"jsonwebtoken": "^9.0.2",
"mysql2": "^3.15.3",
"ws": "^8.18.3"
},
"devDependencies": {
"eslint": "^8.57.1",
"jest": "^29.7.0"
}
}
-11
View File
@@ -1,11 +0,0 @@
{
"env": { "browser": true, "es2021": true },
"parserOptions": { "ecmaVersion": 2021, "sourceType": "script" },
"rules": {
"no-unused-vars": "warn",
"no-empty": "warn",
"no-inner-declarations": "warn",
"semi": ["error", "always"],
"eqeqeq": "warn"
}
}
-1
View File
@@ -1 +0,0 @@
/root/code/web_template/base.js
+223 -745
View File
File diff suppressed because it is too large Load Diff
+316 -893
View File
File diff suppressed because it is too large Load Diff
-7
View File
@@ -1,7 +0,0 @@
{
"env": {
"node": true,
"jest": true,
"es2021": true
}
}
-163
View File
@@ -1,163 +0,0 @@
'use strict';
const { validateWebhookUrl, applyParams, evalCondition, calculateNextRun } = require('../lib/utils');
// ── validateWebhookUrl ──────────────────────────────────────────────────────
describe('validateWebhookUrl', () => {
test('null/empty returns ok with null url', () => {
expect(validateWebhookUrl(null)).toEqual({ ok: true, url: null });
expect(validateWebhookUrl('')).toEqual({ ok: true, url: null });
});
test('valid public https URL is accepted', () => {
const result = validateWebhookUrl('https://hooks.example.com/notify');
expect(result.ok).toBe(true);
expect(result.url).toBeInstanceOf(URL);
});
test('valid public http URL is accepted', () => {
const result = validateWebhookUrl('http://webhook.example.com/event');
expect(result.ok).toBe(true);
});
test('invalid URL format is rejected', () => {
expect(validateWebhookUrl('not a url')).toMatchObject({ ok: false, reason: expect.stringContaining('Invalid URL') });
});
test('non-http protocol is rejected', () => {
expect(validateWebhookUrl('ftp://example.com/hook')).toMatchObject({ ok: false, reason: expect.stringContaining('http') });
});
test('localhost is rejected', () => {
expect(validateWebhookUrl('http://localhost/hook')).toMatchObject({ ok: false });
});
test('10.x.x.x private range is rejected', () => {
expect(validateWebhookUrl('http://10.0.0.1/hook')).toMatchObject({ ok: false });
expect(validateWebhookUrl('http://10.10.10.45/hook')).toMatchObject({ ok: false });
});
test('192.168.x.x private range is rejected', () => {
expect(validateWebhookUrl('http://192.168.1.1/hook')).toMatchObject({ ok: false });
});
test('127.0.0.1 loopback is rejected', () => {
expect(validateWebhookUrl('http://127.0.0.1/hook')).toMatchObject({ ok: false });
});
test('172.16.x.x 172.31.x.x private range is rejected', () => {
expect(validateWebhookUrl('http://172.16.0.1/hook')).toMatchObject({ ok: false });
expect(validateWebhookUrl('http://172.31.255.255/hook')).toMatchObject({ ok: false });
});
test('172.32.x.x (outside private range) is accepted', () => {
const result = validateWebhookUrl('http://172.32.0.1/hook');
expect(result.ok).toBe(true);
});
});
// ── applyParams ─────────────────────────────────────────────────────────────
describe('applyParams', () => {
test('replaces a single placeholder', () => {
expect(applyParams('echo {{msg}}', { msg: 'hello' })).toBe('echo hello');
});
test('replaces multiple placeholders', () => {
expect(applyParams('cp {{src}} {{dst}}', { src: 'a.txt', dst: 'b.txt' })).toBe('cp a.txt b.txt');
});
test('leaves unknown placeholders unchanged', () => {
expect(applyParams('echo {{unknown}}', {})).toBe('echo {{unknown}}');
});
test('throws on unsafe param value with spaces', () => {
expect(() => applyParams('echo {{x}}', { x: 'rm -rf /' })).toThrow('Unsafe value');
});
test('throws on unsafe param value with semicolons', () => {
expect(() => applyParams('echo {{x}}', { x: 'a;b' })).toThrow('Unsafe value');
});
test('allows safe characters: dots, dashes, slashes, colons, @', () => {
expect(applyParams('ssh {{host}}', { host: 'user@10.0.0.1:22' })).toBe('ssh user@10.0.0.1:22');
expect(applyParams('cat {{path}}', { path: '/etc/hosts' })).toBe('cat /etc/hosts');
});
test('trims whitespace from param values', () => {
expect(applyParams('echo {{x}}', { x: ' hello ' })).toBe('echo hello');
});
});
// ── evalCondition ───────────────────────────────────────────────────────────
describe('evalCondition', () => {
test('evaluates truthy expression', () => {
expect(evalCondition('1 === 1', {}, {})).toBe(true);
});
test('evaluates falsy expression', () => {
expect(evalCondition('1 === 2', {}, {})).toBe(false);
});
test('accesses state variables', () => {
expect(evalCondition('state.count > 5', { count: 10 }, {})).toBe(true);
expect(evalCondition('state.count > 5', { count: 3 }, {})).toBe(false);
});
test('accesses params', () => {
expect(evalCondition('params.env === "prod"', {}, { env: 'prod' })).toBe(true);
});
test('returns false on syntax error (does not throw)', () => {
expect(evalCondition('this is not valid js !!!', {}, {})).toBe(false);
});
test('returns false on timeout / infinite loop', () => {
expect(evalCondition('while(true){}', {}, {})).toBe(false);
});
});
// ── calculateNextRun ────────────────────────────────────────────────────────
describe('calculateNextRun', () => {
test('interval: returns a date ~N minutes in the future', () => {
const before = Date.now();
const result = calculateNextRun('interval', '30');
expect(result.getTime()).toBeGreaterThanOrEqual(before + 30 * 60000 - 100);
expect(result.getTime()).toBeLessThanOrEqual(before + 30 * 60000 + 1000);
});
test('interval: throws on non-positive value', () => {
expect(() => calculateNextRun('interval', '0')).toThrow();
expect(() => calculateNextRun('interval', '-5')).toThrow();
expect(() => calculateNextRun('interval', 'abc')).toThrow();
});
test('hourly: returns a date ~N hours in the future', () => {
const before = Date.now();
const result = calculateNextRun('hourly', '2');
expect(result.getTime()).toBeGreaterThanOrEqual(before + 2 * 3600000 - 100);
});
test('daily: returns a Date object', () => {
const result = calculateNextRun('daily', '06:00');
expect(result).toBeInstanceOf(Date);
expect(result.getTime()).toBeGreaterThan(Date.now());
});
test('daily: throws on invalid time format', () => {
expect(() => calculateNextRun('daily', 'noon')).toThrow();
});
test('cron: returns next occurrence after now', () => {
const result = calculateNextRun('cron', '0 6 * * 1');
expect(result).toBeInstanceOf(Date);
expect(result.getTime()).toBeGreaterThan(Date.now());
});
test('unknown type throws', () => {
expect(() => calculateNextRun('weekly', '1')).toThrow('Unknown schedule type');
});
});
-257
View File
@@ -1,257 +0,0 @@
const axios = require('axios');
const WebSocket = require('ws');
const { exec } = require('child_process');
const { promisify } = require('util');
const os = require('os');
const crypto = require('crypto');
require('dotenv').config();
const execAsync = promisify(exec);
class PulseWorker {
constructor() {
this.workerId = crypto.randomUUID();
this.workerName = process.env.WORKER_NAME || os.hostname();
this.serverUrl = process.env.PULSE_SERVER || 'http://localhost:8080';
this.wsUrl = process.env.PULSE_WS || 'ws://localhost:8080';
this.apiKey = process.env.WORKER_API_KEY;
this.heartbeatInterval = parseInt(process.env.HEARTBEAT_INTERVAL || '30') * 1000;
this.maxConcurrentTasks = parseInt(process.env.MAX_CONCURRENT_TASKS || '5');
this.activeTasks = 0;
this.ws = null;
this.heartbeatTimer = null;
}
async start() {
console.log(`[PULSE Worker] Starting worker: ${this.workerName}`);
console.log(`[PULSE Worker] Worker ID: ${this.workerId}`);
console.log(`[PULSE Worker] Server: ${this.serverUrl}`);
// Send initial heartbeat
await this.sendHeartbeat();
// Start heartbeat timer
this.startHeartbeat();
// Connect to WebSocket for real-time commands
this.connectWebSocket();
console.log(`[PULSE Worker] Worker started successfully`);
}
startHeartbeat() {
this.heartbeatTimer = setInterval(async () => {
try {
await this.sendHeartbeat();
} catch (error) {
console.error('[PULSE Worker] Heartbeat failed:', error.message);
}
}, this.heartbeatInterval);
}
async sendHeartbeat() {
const metadata = {
hostname: os.hostname(),
platform: os.platform(),
arch: os.arch(),
cpus: os.cpus().length,
totalMem: os.totalmem(),
freeMem: os.freemem(),
uptime: os.uptime(),
loadavg: os.loadavg(),
activeTasks: this.activeTasks,
maxConcurrentTasks: this.maxConcurrentTasks
};
try {
const response = await axios.post(
`${this.serverUrl}/api/workers/heartbeat`,
{
worker_id: this.workerId,
name: this.workerName,
metadata: metadata
},
{
headers: {
'X-API-Key': this.apiKey,
'Content-Type': 'application/json'
}
}
);
console.log(`[PULSE Worker] Heartbeat sent - Status: online`);
return response.data;
} catch (error) {
console.error('[PULSE Worker] Heartbeat error:', error.message);
throw error;
}
}
connectWebSocket() {
console.log(`[PULSE Worker] Connecting to WebSocket...`);
this.ws = new WebSocket(this.wsUrl);
this.ws.on('open', () => {
console.log('[PULSE Worker] WebSocket connected');
// Identify this worker
this.ws.send(JSON.stringify({
type: 'worker_connect',
worker_id: this.workerId,
worker_name: this.workerName
}));
});
this.ws.on('message', async (data) => {
try {
const message = JSON.parse(data.toString());
await this.handleMessage(message);
} catch (error) {
console.error('[PULSE Worker] Message handling error:', error);
}
});
this.ws.on('close', () => {
console.log('[PULSE Worker] WebSocket disconnected, reconnecting...');
setTimeout(() => this.connectWebSocket(), 5000);
});
this.ws.on('error', (error) => {
console.error('[PULSE Worker] WebSocket error:', error.message);
});
}
async handleMessage(message) {
console.log(`[PULSE Worker] Received message:`, message.type);
switch (message.type) {
case 'execute_command':
await this.executeCommand(message);
break;
case 'execute_workflow':
await this.executeWorkflow(message);
break;
case 'ping':
this.sendPong();
break;
default:
console.log(`[PULSE Worker] Unknown message type: ${message.type}`);
}
}
async executeCommand(message) {
const { command, execution_id, command_id, timeout = 300000 } = message;
if (this.activeTasks >= this.maxConcurrentTasks) {
console.log(`[PULSE Worker] Max concurrent tasks reached, rejecting command`);
return;
}
this.activeTasks++;
console.log(`[PULSE Worker] Executing command (active tasks: ${this.activeTasks})`);
try {
const startTime = Date.now();
const { stdout, stderr } = await execAsync(command, {
timeout: timeout,
maxBuffer: 10 * 1024 * 1024 // 10MB buffer
});
const duration = Date.now() - startTime;
const result = {
type: 'command_result',
execution_id,
worker_id: this.workerId,
command_id,
success: true,
stdout: stdout,
stderr: stderr,
duration: duration,
timestamp: new Date().toISOString()
};
this.sendResult(result);
console.log(`[PULSE Worker] Command completed in ${duration}ms`);
} catch (error) {
const result = {
type: 'command_result',
execution_id,
worker_id: this.workerId,
command_id,
success: false,
error: error.message,
stdout: error.stdout || '',
stderr: error.stderr || '',
timestamp: new Date().toISOString()
};
this.sendResult(result);
console.error(`[PULSE Worker] Command failed:`, error.message);
} finally {
this.activeTasks--;
}
}
async executeWorkflow(message) {
const { workflow, execution_id } = message;
console.log(`[PULSE Worker] Executing workflow: ${workflow.name}`);
// Workflow execution will be implemented in phase 2
// For now, just acknowledge receipt
this.sendResult({
type: 'workflow_result',
execution_id,
worker_id: this.workerId,
success: true,
message: 'Workflow execution not yet implemented',
timestamp: new Date().toISOString()
});
}
sendResult(result) {
if (this.ws && this.ws.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(result));
}
}
sendPong() {
if (this.ws && this.ws.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify({ type: 'pong', worker_id: this.workerId }));
}
}
async stop() {
console.log('[PULSE Worker] Shutting down...');
if (this.heartbeatTimer) {
clearInterval(this.heartbeatTimer);
}
if (this.ws) {
this.ws.close();
}
console.log('[PULSE Worker] Shutdown complete');
}
}
// Start worker
const worker = new PulseWorker();
// Handle graceful shutdown
process.on('SIGTERM', async () => {
await worker.stop();
process.exit(0);
});
process.on('SIGINT', async () => {
await worker.stop();
process.exit(0);
});
// Start the worker
worker.start().catch((error) => {
console.error('[PULSE Worker] Fatal error:', error);
process.exit(1);
});