feat: Support Azure ServiceBus PubSub for internalmq

[alexluong]

implements #139

Azure Resource Concepts

For those unfamiliar with Azure's resource hierarchy:

• Tenant: The top-level organizational boundary representing a company or project
• Subscription: A billing account that contains and manages resources within a tenant
• Service Principal: An identity used for programmatic access, similar to an IAM role in other clouds
• Resource Group: A logical container that groups related Azure resources for easier management
• Service Bus Namespace: A messaging container within Azure Service Bus where you create topics and subscriptions for event routing

This implementation leverages Azure Service Bus topics and subscriptions within a namespace to provide reliable message delivery with Outpost's multi-tenant architecture.

User Setup Requirements

Outpost requires specific Azure permissions to manage Service Bus resources on your behalf:

Required Role: Azure Service Bus Data Owner on the Service Bus namespace

Required Permissions:

• Create and delete topics and subscriptions
• Publish messages to topics
• Consume messages from subscriptions

Recommended Setup:

• Optionally create a dedicated resource group to contain Outpost-related resources
• Create a dedicated Service Bus namespace for Outpost
• Assign the Service Principal used by Outpost the Azure Service Bus Data Owner role scoped to the namespace

This scoped permission model ensures Outpost can fully manage message routing within its designated namespace while maintaining security boundaries with your other Azure resources.

Azure Setup Commands

Prerequisites

# Install Azure CLI if not already installed
# https://docs.microsoft.com/en-us/cli/azure/install-azure-cli

# Login to Azure
az login

1. Create Resource Group (Optional but Recommended)

# Set variables
RESOURCE_GROUP="outpost-rg"
LOCATION="eastus"

# Create resource group
az group create --name $RESOURCE_GROUP --location $LOCATION

2. Create Service Bus Namespace

# Generate unique namespace name (must be globally unique)
RANDOM_SUFFIX=$(openssl rand -hex 4)
NAMESPACE_NAME="outpost-servicebus-$RANDOM_SUFFIX"

# Create Service Bus namespace
az servicebus namespace create \
  --resource-group $RESOURCE_GROUP \
  --name $NAMESPACE_NAME \
  --location $LOCATION \
  --sku Standard

3. Create Service Principal

APP_NAME="outpost-service-principal"

# Create service principal and capture output
SP_OUTPUT=$(az ad sp create-for-rbac --name $APP_NAME)
CLIENT_ID=$(echo $SP_OUTPUT | jq -r '.appId')
CLIENT_SECRET=$(echo $SP_OUTPUT | jq -r '.password')
TENANT_ID=$(echo $SP_OUTPUT | jq -r '.tenant')

4. Assign Permissions

# Get the namespace resource ID
NAMESPACE_ID=$(az servicebus namespace show \
  --resource-group $RESOURCE_GROUP \
  --name $NAMESPACE_NAME \
  --query id -o tsv)

# Assign Azure Service Bus Data Owner role
az role assignment create \
  --assignee $CLIENT_ID \
  --role "Azure Service Bus Data Owner" \
  --scope $NAMESPACE_ID

5. Configuration Values for Outpost

# Get subscription ID
SUBSCRIPTION_ID=$(az account show --query id -o tsv)

# Echo all required values
echo "tenant_id: $TENANT_ID"
echo "client_id: $CLIENT_ID"
echo "client_secret: $CLIENT_SECRET"
echo "subscription_id: $SUBSCRIPTION_ID"
echo "resource_group: $RESOURCE_GROUP"
echo "namespace: $NAMESPACE_NAME"

TODOs

• set up emulator for integration testing with Azure
• light load test to make sure the Azure integration (pub, sub) can handle moderate load

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
outpost-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 10, 2025 9:24am
outpost-website	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 10, 2025 9:24am

[leggetter]

@alexluong - following the setup instructions and then running Outpost locally with make up, I get the following in the api service logs:

running...
{"level":"info","ts":1749544317.5334902,"caller":"app/app.go:47","msg":"starting outpost","config_path":".env","service":"api"}
{"level":"info","ts":1749544317.7974298,"caller":"app/app.go:197","msg":"no migrations applied","version":3}
failed to declare topic: failed to create topic outpost-delivery: PUT https://management.azure.com/subscriptions/21f864a1-c750-44d6-b40a-bf62a33541aa/resourceGroups/outpost-rg/providers/Microsoft.ServiceBus/namespaces/outpost-servicebus-271b4344/topics/outpost-delivery
--------------------------------------------------------------------------------
RESPONSE 429: 429 Too Many Requests
ERROR CODE: MessagingGatewayTooManyRequests
--------------------------------------------------------------------------------
{
  "error": {
    "code": "MessagingGatewayTooManyRequests",
    "message": "SubCode=40900. Conflict. You're requesting an operation that isn't allowed in the resource's current state. To know more visit https://aka.ms/sbResourceMgrExceptions.  ExceptionDescription: sb://outpost-servicebus-271b4344.servicebus.windows.net/outpost-delivery. TrackingId:49463282-fa78-475b-8110-355879cf2d3b_G13, SystemTracker:outpost-servicebus-271b4344.servicebus.windows.net:outpost-delivery, Timestamp:2025-06-10T08:32:01"
  }
}
--------------------------------------------------------------------------------

Process Exit with Code: 1

Something to do with permissions? Or a timing issue?

failed to declare topic: failed to create topic outpost-delivery: PUT https://management.azure.com/subscriptions/21f864a1-c750-44d6-b40a-bf62a33541aa/resourceGroups/outpost-rg/providers/Microsoft.ServiceBus/namespaces/outpost-servicebus-271b4344/topics/outpost-delivery

[alexluong]

@leggetter yes I noticed some issues on the first run, most likely because the 3 services are trying to create the same pubsub at the same time. I don't have a solution yet. Locally what I do is I run the API node first, then after the infra is provisioned, I run the other.

Not sure why we haven't run into this with the other mqs. We may need to implement a mechanism in the infra logic to only allow 1 provision to happen at once. Created #427 here so we can track this issue.

[leggetter]

@alexluong - configured locally and tested.

I did come across an error that I think is unrelated to this PR when running examples/demos/nodejs / npm run publish-api:

fatal error: all goroutines are asleep - deadlock!
⠙
goroutine 1 [chan receive]:
github.com/evanw/esbuild/internal/helpers.(*ThreadSafeWaitGroup).Wait(...)
        github.com/evanw/esbuild/internal/helpers/waitgroup.go:36
main.runService.func2()
        github.com/evanw/esbuild/cmd/esbuild/service.go:114 +0x8c
main.runService(0x1)
        github.com/evanw/esbuild/cmd/esbuild/service.go:160 +0x4b4
main.main()
        github.com/evanw/esbuild/cmd/esbuild/main.go:236 +0x8e4

goroutine 34 [chan receive]:
main.runService.func1()
        github.com/evanw/esbuild/cmd/esbuild/service.go:98 +0x40
created by main.runService
        github.com/evanw/esbuild/cmd/esbuild/service.go:97 +0x1a0

goroutine 35 [chan receive]:
main.(*serviceType).sendRequest(0x1400010de60, {0x104b5b4e0, 0x140003a04e0})
        github.com/evanw/esbuild/cmd/esbuild/service.go:192 +0x11c
main.runService.func3()
        github.com/evanw/esbuild/cmd/esbuild/service.go:125 +0x38
created by main.runService
        github.com/evanw/esbuild/cmd/esbuild/service.go:122 +0x308

I've also updated the config reference and created a guide to setting up Azure Service Bus based on the details you provided in this PR.

[alexluong]

@leggetter this is an issue with esbuild which uses Go under the hood. I don't think it's related to Outpost itself. Can you try investigating esbuild and maybe reinstall deps (npm rm -rf node_modules && npm install) to see if it resolves this issue?

[leggetter]

@alexluong - if required, let's capture these in an issue for later:

• set up emulator for integration testing with Azure
• light load test to make sure the Azure integration (pub, sub) can handle moderate load

---

@leggetter this is an issue with esbuild which uses Go under the hood. I don't think it's related to Outpost itself. Can you try investigating esbuild and maybe reinstall deps (npm rm -rf node_modules && npm install) to see if it resolves this issue?

Aha! Ok, this shouldn't block this from being merged. I'll investigate later.