Jared Vititoe
018752813a
Added full workflow execution engine that actually runs workflow steps:
Server-Side (server.js):
- executeWorkflowSteps() - Main workflow orchestration function
- executeCommandStep() - Executes commands on target workers
- waitForCommandResult() - Polls for command completion
- Support for step types: execute, wait, prompt (prompt skipped for now)
- Sequential step execution with failure handling
- Worker targeting: "all" or specific worker IDs/names
- Automatic status updates (running -> completed/failed)
- Real-time WebSocket broadcasts for step progress
- Command result tracking with command_id for workflows
- Only updates status for non-workflow quick commands
Client-Side (index.html):
- Enhanced formatLogEntry() with workflow-specific log types
- step_started - Shows step number and name with amber color
- step_completed - Shows completion with green checkmark
- waiting - Displays wait duration
- no_workers - Error when no workers available
- worker_offline - Warning for offline workers
- workflow_error - Critical workflow errors
- Better visual feedback for workflow progress
Workflow Definition Format:
{
"steps": [
{
"name": "Step Name",
"type": "execute",
"targets": ["all"] or ["worker-name"],
"command": "your command here"
},
{
"type": "wait",
"duration": 5
}
]
}
Features:
- Executes steps sequentially
- Stops on first failure
- Supports multiple workers per step
- Real-time progress updates
- Comprehensive logging
- Terminal-themed workflow logs
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
PULSE - Pipelined Unified Logic & Server Engine
A distributed workflow orchestration platform for managing and executing complex multi-step operations across server clusters through a retro terminal-themed web interface.
Overview
PULSE is a centralized workflow execution system designed to orchestrate operations across distributed infrastructure. It provides a powerful web-based interface with a vintage CRT terminal aesthetic for defining, managing, and executing workflows that can span multiple servers, require human interaction, and perform complex automation tasks at scale.
Key Features
- 🎨 Retro Terminal Interface: Phosphor green CRT-style interface with scanlines, glow effects, and ASCII art
- ⚡ Quick Command Execution: Instantly execute commands on any worker with built-in templates and command history
- 📊 Real-Time Worker Monitoring: Live system metrics including CPU, memory, load average, and active tasks
- 🔄 Interactive Workflow Management: Define and execute multi-step workflows with conditional logic and user prompts
- 🌐 Distributed Execution: Run commands across multiple worker nodes simultaneously via WebSocket
- 📈 Execution Tracking: Comprehensive logging with formatted output, re-run capabilities, and JSON export
- 🔐 SSO Authentication: Seamless integration with Authelia for enterprise authentication
- 🧹 Auto-Cleanup: Automatic removal of old executions with configurable retention policies
- 🔔 Terminal Notifications: Audio beeps and visual toasts for command completion events
Architecture
PULSE consists of two core components:
PULSE Server
Location: 10.10.10.65 (LXC Container ID: 122)
Directory: /opt/pulse-server
The central orchestration hub that:
- Hosts the retro terminal web interface
- Manages workflow definitions and execution state
- Coordinates task distribution to worker nodes via WebSocket
- Handles user interactions through Authelia SSO
- Provides real-time status updates and logging
- Stores all data in MariaDB database
Technology Stack:
- Node.js 20.x
- Express.js (web framework)
- WebSocket (ws package) for real-time bidirectional communication
- MySQL2 (MariaDB driver)
- Authelia SSO integration
PULSE Worker
Example: 10.10.10.151 (LXC Container ID: 153, hostname: pulse-worker-01)
Directory: /opt/pulse-worker
Lightweight execution agents that:
- Connect to PULSE server via WebSocket with heartbeat monitoring
- Execute shell commands and report results in real-time
- Provide system metrics (CPU, memory, load, uptime)
- Support concurrent task execution with configurable limits
- Automatically reconnect on connection loss
Technology Stack:
- Node.js 20.x
- WebSocket client
- Child process execution
- System metrics collection
┌─────────────────────────────────┐
│ PULSE Server (10.10.10.65) │
│ Terminal Web Interface + API │
│ ┌───────────┐ ┌──────────┐ │
│ │ MariaDB │ │ Authelia │ │
│ │ Database │ │ SSO │ │
│ └───────────┘ └──────────┘ │
└────────────┬────────────────────┘
│ WebSocket
┌────────┴────────┬───────────┐
│ │ │
┌───▼────────┐ ┌───▼────┐ ┌──▼─────┐
│ Worker 1 │ │Worker 2│ │Worker N│
│10.10.10.151│ │ ... │ │ ... │
└────────────┘ └────────┘ └────────┘
LXC Containers in Proxmox with Ceph
Installation
Prerequisites
- Node.js 20.x or higher
- MariaDB 10.x or higher
- Authelia configured for SSO (optional but recommended)
- Network Connectivity between server and workers
PULSE Server Setup
# Clone repository
cd /opt
git clone <your-repo-url> pulse-server
cd pulse-server
# Install dependencies
npm install
# Create .env file with configuration
cat > .env << EOF
# Server Configuration
PORT=8080
SECRET_KEY=your-secret-key-here
# MariaDB Configuration
DB_HOST=10.10.10.50
DB_PORT=3306
DB_NAME=pulse
DB_USER=pulse_user
DB_PASSWORD=your-db-password
# Worker API Key (for worker authentication)
WORKER_API_KEY=your-worker-api-key
# Auto-cleanup configuration (optional)
EXECUTION_RETENTION_DAYS=30
EOF
# Create systemd service
cat > /etc/systemd/system/pulse.service << EOF
[Unit]
Description=PULSE Workflow Orchestration Server
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/pulse-server
ExecStart=/usr/bin/node server.js
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
# Start service
systemctl daemon-reload
systemctl enable pulse.service
systemctl start pulse.service
PULSE Worker Setup
# On each worker node
cd /opt
git clone <your-repo-url> pulse-worker
cd pulse-worker
# Install dependencies
npm install
# Create .env file
cat > .env << EOF
# Worker Configuration
WORKER_NAME=pulse-worker-01
PULSE_SERVER=http://10.10.10.65:8080
PULSE_WS=ws://10.10.10.65:8080
WORKER_API_KEY=your-worker-api-key
# Performance Settings
HEARTBEAT_INTERVAL=30
MAX_CONCURRENT_TASKS=5
EOF
# Create systemd service
cat > /etc/systemd/system/pulse-worker.service << EOF
[Unit]
Description=PULSE Worker Node
After=network.target
[Service]
Type=simple
User=root
WorkingDirectory=/opt/pulse-worker
ExecStart=/usr/bin/node worker.js
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
EOF
# Start service
systemctl daemon-reload
systemctl enable pulse-worker.service
systemctl start pulse-worker.service
Usage
Quick Command Execution
- Access PULSE at
http://your-server:8080 - Navigate to ⚡ Quick Command tab
- Select a worker from the dropdown
- Use Templates for pre-built commands or History for recent commands
- Enter your command and click Execute
- View results in the Executions tab
Built-in Command Templates:
- System Info:
uname -a - Disk Usage:
df -h - Memory Usage:
free -h - CPU Info:
lscpu - Running Processes:
ps aux --sort=-%mem | head -20 - Network Interfaces:
ip addr show - Docker Containers:
docker ps -a - System Logs:
tail -n 50 /var/log/syslog
Worker Monitoring
The Workers tab displays real-time metrics for each worker:
- System information (OS, architecture, CPU cores)
- Memory usage (used/total with percentage)
- Load averages (1m, 5m, 15m)
- System uptime
- Active tasks vs. maximum concurrent capacity
Execution Management
- View Details: Click any execution to see formatted logs with timestamps, status, and output
- Re-run Command: Click "Re-run" button in execution details to repeat a command
- Download Logs: Export execution data as JSON for auditing
- Clear Completed: Bulk delete finished executions
- Auto-Cleanup: Executions older than 30 days are automatically removed
Workflow Creation (Future Feature)
- Navigate to Workflows → Create New
- Define workflow steps using JSON syntax
- Specify target workers
- Add interactive prompts where needed
- Save and execute
Features in Detail
Terminal Aesthetic
- Phosphor green (#00ff41) on black (#0a0a0a) color scheme
- CRT scanline animation effect
- Text glow and shadow effects
- ASCII box-drawing characters for borders
- Boot sequence animation on first load
- Hover effects with smooth transitions
Real-Time Communication
- WebSocket-based bidirectional communication
- Instant command result notifications
- Live worker status updates
- Terminal beep sounds for events
- Toast notifications with visual feedback
Execution Tracking
- Formatted log display (not raw JSON)
- Color-coded success/failure indicators
- Timestamp and duration for each step
- Scrollable output with syntax highlighting
- Persistent history with pagination
- Load More button for large execution lists
Security
- Authelia SSO integration for user authentication
- API key authentication for workers
- User session management
- Admin-only operations (worker deletion, workflow management)
- Audit logging for all executions
Performance
- Automatic cleanup of old executions (configurable retention)
- Pagination for large execution lists (50 at a time)
- Efficient WebSocket connection pooling
- Worker heartbeat monitoring
- Database connection pooling
Configuration
Environment Variables
Server (.env):
PORT=8080 # Server port
SECRET_KEY=<random-string> # Session secret
DB_HOST=10.10.10.50 # MariaDB host
DB_PORT=3306 # MariaDB port
DB_NAME=pulse # Database name
DB_USER=pulse_user # Database user
DB_PASSWORD=<password> # Database password
WORKER_API_KEY=<api-key> # Worker authentication key
EXECUTION_RETENTION_DAYS=30 # Auto-cleanup retention (default: 30)
Worker (.env):
WORKER_NAME=pulse-worker-01 # Unique worker name
PULSE_SERVER=http://10.10.10.65:8080 # Server HTTP URL
PULSE_WS=ws://10.10.10.65:8080 # Server WebSocket URL
WORKER_API_KEY=<api-key> # Must match server key
HEARTBEAT_INTERVAL=30 # Heartbeat seconds (default: 30)
MAX_CONCURRENT_TASKS=5 # Max parallel tasks (default: 5)
Database Schema
PULSE uses MariaDB with the following tables:
- users: User accounts from Authelia SSO
- workers: Worker node registry with metadata
- workflows: Workflow definitions (JSON)
- executions: Execution history with logs
See Claude.md for complete schema details.
Troubleshooting
Worker Not Connecting
# Check worker service status
systemctl status pulse-worker
# Check worker logs
journalctl -u pulse-worker -n 50 -f
# Verify API key matches server
grep WORKER_API_KEY /opt/pulse-worker/.env
Commands Stuck in "Running"
- This was fixed in recent updates - restart the server:
systemctl restart pulse.service
Clear All Executions
Use the database directly if needed:
mysql -h 10.10.10.50 -u pulse_user -p pulse
> DELETE FROM executions WHERE status IN ('completed', 'failed');
Development
Recent Updates
Phase 1-6 Improvements:
- Formatted log display with color-coding
- Worker system metrics monitoring
- Command templates and history
- Re-run and download execution features
- Auto-cleanup and pagination
- Terminal aesthetic refinements
- Audio notifications and visual toasts
See git history for detailed changelog.
Future Enhancements
- Full workflow system implementation
- Multi-worker command execution
- Scheduled/cron job support
- Execution search and filtering
- Dark/light theme toggle
- Mobile-responsive design
- REST API documentation
- Webhook integrations
License
MIT License - See LICENSE file for details
PULSE - Orchestrating your infrastructure, one heartbeat at a time. ⚡
Built with retro terminal aesthetics 🖥️ | Powered by WebSockets 🔌 | Secured by Authelia 🔐