Durable Execution Engine - User Manual (Docker Compose & CLI)

---

1. Clone the Repository

git clone https://github.com/SalmaElsoly/durable-execution-engine.git
cd durable-execution-engine

---

2. Prepare Secrets (Production Mode)

Create two files in the project root:

• jwt_secret_access.txt
• jwt_refresh_secret.txt

Each should contain a secure random string (e.g., use openssl rand -hex 32).

3.Prepare environment variables (Development Mode)

You must set JWT_ACCESS_SECRET & JWT_REFRESH_SECRET as environment variables using a secure random string algorithm.

Take care the engine won't run without setting those tokens : access and refresh

---

3. Start the Engine with Docker Compose

docker compose up --build

Take care if production mode , build the frontend container using this command

 docker compose build frontend --build-arg NGINX_CONF=nginx.conf.template --no-cache

This will start all services:

• Backend API (go-app)
• Frontend UI
• CLI container
• etcd (state store)
• Prometheus (metrics)
• Grafana (dashboard)
• Node Exporter (host metrics)

---

4. Accessing the Services

Service	URL/Port	Description
Frontend	http://localhost:5005	Web UI for workflows
Backend API	http://localhost:8080	REST API
Prometheus	http://localhost:9090	Metrics
Grafana	http://localhost:3000	Dashboards
etcd	http://localhost:2379	State store (internal)

---

5. Using the CLI

You can use the CLI either from the container or by building it locally.

5.1. Using the CLI in Docker

Open a shell in the CLI container:

docker compose run --rm cli /bin/bash

Now you can use the CLI as endure-cli.

5.2. Building the CLI Locally

cd cli
go build -o endure-cli main.go
./endure-cli --help

5.3 Register a User

To register a new user, run in a terminal (after buiding cli or using docker):

endure-cli user register --username <username> --email <email>

• You will be prompted to enter and confirm a password.
• Example:

endure-cli user register --username alice --email [email protected]

• To target a different backend host/port (default is localhost:8080):

endure-cli -H localhost:8080 user register --username alice --email [email protected]

5.4. Register a Service

To register a new service (must be a reachable HTTP/HTTPS URL):

endure-cli services register <service-url>

• Example:

endure-cli services register http://my-service:8081

• To target a different backend host/port:

endure-cli -H localhost:8080 services register http://my-service:8081

5.5. Common CLI Flags

• -H, --host <host:port>: Specify the backend API host (default: localhost:8080)
• -v, --verbose: Enable verbose output

---

6. Workflow Usage Flow

To see and interact with workflows in the UI, follow this flow:

1. Ensure a Service is Running with the SDK

   • Your microservice must be running and integrated with the Durable Execution Engine SDK. This allows the engine to discover and interact with your service's workflows. SDK Repo here

2. Register the Service with the Engine

   • Use the CLI to register your service with the engine:

     endure-cli services register <service-url>

   • The <service-url> should be the HTTP/HTTPS endpoint where your service (with the SDK) is accessible.

3. Register a User (First Time Only)

   • Before accessing the UI, you must register a user via the CLI:

     endure-cli user register --username <username> --email <email>

   • You will be prompted to enter and confirm a password.

4. Login to the UI

   • Go to http://localhost:5000.
   • Log in using the credentials you registered with the CLI.

5. Trigger a Workflow Execution (for Testing)

   • To see a workflow instance in the UI, you must first trigger an execution. This is typically done by making a POST request to the backend API:

     curl -X POST http://localhost:8080/api/v1/executions -H "Content-Type: application/json" -d '{"workflow_name": "<workflow>", "input": { ... }}'

   • Replace <workflow> with the workflow name, and provide the required input.

6. View and Manage Workflows in the UI

   • After triggering an execution, you can view workflow runs, details, and status in the UI dashboard.

---

7. Deployment

For deployment , you will need to deploy the docker-stack.yml file on vms and for etcd you will need to create a cluster of one node first then create other nodes and make them join the cluster manually. We added 3 nodes only , each one on a seperate vm. You can find our deployment model in the slides (next section)

---

8. Documentation

Slides