pulse/README.md

# PULSE - Pipelined Unified Logic & Server Engine

A distributed workflow orchestration platform for managing and executing complex multi-step operations across server clusters through an intuitive web interface.

## Overview

PULSE is a centralized workflow execution system designed to orchestrate operations across distributed infrastructure. It provides a powerful web-based interface for defining, managing, and executing workflows that can span multiple servers, require human interaction, and perform complex automation tasks at scale.

### Key Features

- **Interactive Workflow Management**: Define and execute multi-step workflows with conditional logic, user prompts, and decision points
- **Distributed Execution**: Run commands and scripts across multiple worker nodes simultaneously
- **High Availability Architecture**: Deploy redundant worker nodes in LXC containers with Ceph storage for fault tolerance
- **Web-Based Control Center**: Intuitive interface for workflow selection, monitoring, and interactive input
- **Flexible Worker Pool**: Scale horizontally by adding worker nodes as needed
- **Real-Time Monitoring**: Track workflow progress, view logs, and receive notifications

## Architecture

PULSE consists of two core components:

### PULSE Server
The central orchestration hub that:
- Hosts the web interface for workflow management
- Manages workflow definitions and execution state
- Coordinates task distribution to worker nodes
- Handles user interactions and input collection
- Provides real-time status updates and logging

### PULSE Worker
Lightweight execution agents that:
- Connect to the PULSE server and await task assignments
- Execute commands, scripts, and code on target infrastructure
- Report execution status and results back to the server
- Support multiple concurrent workflow executions
- Automatically reconnect and resume on failure
```
┌─────────────────────┐
│   PULSE Server      │
│   (Web Interface)   │
└──────────┬──────────┘
           │
    ┌──────┴───────┬──────────────┬──────────────┐
    │              │              │              │
┌───▼────┐    ┌───▼────┐    ┌───▼────┐    ┌───▼────┐
│ Worker │    │ Worker │    │ Worker │    │ Worker │
│ Node 1 │    │ Node 2 │    │ Node 3 │    │ Node N │
└────────┘    └────────┘    └────────┘    └────────┘
     LXC Containers in Proxmox with Ceph
```

## Deployment

### Prerequisites

- **Proxmox VE Cluster**: Hypervisor environment for container deployment
- **Ceph Storage**: Distributed storage backend for high availability
- **LXC Support**: Container runtime for worker node deployment
- **Network Connectivity**: Communication between server and workers

### Installation

#### PULSE Server
```bash
# Clone the repository
git clone https://github.com/yourusername/pulse.git
cd pulse

# Install dependencies
npm install  # or pip install -r requirements.txt

# Configure server settings
cp config.example.yml config.yml
nano config.yml

# Start the PULSE server
npm start  # or python server.py
```

#### PULSE Worker
```bash
# On each worker node (LXC container)
cd pulse-worker

# Install dependencies
npm install  # or pip install -r requirements.txt

# Configure worker connection
cp worker-config.example.yml worker-config.yml
nano worker-config.yml

# Start the worker daemon
npm start  # or python worker.py
```

### High Availability Setup

Deploy multiple worker nodes across Proxmox hosts:
```bash
# Create LXC template
pct create 1000 local:vztmpl/ubuntu-22.04-standard_amd64.tar.zst \
  --rootfs ceph-pool:8 \
  --memory 2048 \
  --cores 2 \
  --net0 name=eth0,bridge=vmbr0,ip=dhcp

# Clone for additional workers
pct clone 1000 1001 --full --storage ceph-pool
pct clone 1000 1002 --full --storage ceph-pool
pct clone 1000 1003 --full --storage ceph-pool

# Start all workers
for i in {1000..1003}; do pct start $i; done
```

## Usage

### Creating a Workflow

1. Access the PULSE web interface at `http://your-server:8080`
2. Navigate to **Workflows** → **Create New**
3. Define workflow steps using the visual editor or YAML syntax
4. Specify execution targets (specific nodes, groups, or all workers)
5. Add interactive prompts where user input is required
6. Save and activate the workflow

### Example Workflow
```yaml
name: "System Update and Reboot"
description: "Update all servers in the cluster with user confirmation"
steps:
  - name: "Check Current Versions"
    type: "execute"
    targets: ["all"]
    command: "apt list --upgradable"
    
  - name: "User Approval"
    type: "prompt"
    message: "Review available updates. Proceed with installation?"
    options: ["Yes", "No", "Cancel"]
    
  - name: "Install Updates"
    type: "execute"
    targets: ["all"]
    command: "apt-get update && apt-get upgrade -y"
    condition: "prompt_response == 'Yes'"
    
  - name: "Reboot Confirmation"
    type: "prompt"
    message: "Updates complete. Reboot all servers?"
    options: ["Yes", "No"]
    
  - name: "Rolling Reboot"
    type: "execute"
    targets: ["all"]
    command: "reboot"
    strategy: "rolling"
    condition: "prompt_response == 'Yes'"
```

### Running a Workflow

1. Select a workflow from the dashboard
2. Click **Execute**
3. Monitor progress in real-time
4. Respond to interactive prompts as they appear
5. View detailed logs for each execution step

## Configuration

### Server Configuration (`config.yml`)
```yaml
server:
  host: "0.0.0.0"
  port: 8080
  secret_key: "your-secret-key"

database:
  type: "postgresql"
  host: "localhost"
  port: 5432
  name: "pulse"

workers:
  heartbeat_interval: 30
  timeout: 300
  max_concurrent_tasks: 10

security:
  enable_authentication: true
  require_approval: true
```

### Worker Configuration (`worker-config.yml`)
```yaml
worker:
  name: "worker-01"
  server_url: "http://pulse-server:8080"
  api_key: "worker-api-key"
  
resources:
  max_cpu_percent: 80
  max_memory_mb: 1024
  
executor:
  shell: "/bin/bash"
  working_directory: "/tmp/pulse"
  timeout: 3600
```

## Features in Detail

### Interactive Workflows
- Pause execution to collect user input via web forms
- Display intermediate results for review
- Conditional branching based on user decisions
- Multi-choice prompts with validation

### Mass Execution
- Run commands across all workers simultaneously
- Target specific node groups or individual servers
- Rolling execution for zero-downtime updates
- Parallel and sequential execution strategies

### Monitoring & Logging
- Real-time workflow execution dashboard
- Detailed per-step logging and output capture
- Historical execution records and analytics
- Alert notifications for failures or completion

### Security
- Role-based access control (RBAC)
- API key authentication for workers
- Workflow approval requirements
- Audit logging for all actions


---

**PULSE** - Orchestrating your infrastructure, one heartbeat at a time.