[FEATURE]: Support for multi-node replicas with LeaderWorkerSet.

[julienmancuso]

Feature request

[FEATURE]: Support for multi-node replicas with LeaderWorkerSet.

Describe the problem you're encountering

[FEATURE]: Support for multi-node replicas with LeaderWorkerSet.

Describe alternatives you've tried

No response

[deepcoldfish]

how to use dynamo deploy sglang on multi-node with leaderworkerset like lws example of sglang ? @julienmancuso

[JesseStutler]

how to use dynamo deploy sglang on multi-node with leaderworkerset like lws example of sglang ? @julienmancuso

@deepcoldfish Seems that you're finding this https://github.com/ai-dynamo/dynamo/blob/main/components/backends/sglang/deploy/disagg.yaml ?

[JesseStutler]

Hi @julienmancuso I would like to ask why does DynamoComponentDeployment create replicas of leaderWorkerSet instead of just one leaderWorkerSet and set lws' replicas the same as dcd?

dynamo/deploy/cloud/operator/internal/controller/dynamocomponentdeployment_controller.go

Lines 219 to 256 in 20ad730

	for i := range int(desiredReplicas) {
	modified_, _, err := commonController.SyncResource(ctx, r, dynamoComponentDeployment, func(ctx context.Context) (*volcanov1beta1.PodGroup, bool, error) {
	return r.generateVolcanoPodGroup(ctx, generateResourceOption{
	dynamoComponentDeployment: dynamoComponentDeployment,
	isStealingTrafficDebugModeEnabled: false,
	containsStealingTrafficDebugModeEnabled: false,
	instanceID: &i,
	})
	})
	if err != nil {
	return ctrl.Result{}, err
	}
	if modified_ {
	anyModified = true
	}
	modified_, lwsObj, err := commonController.SyncResource(ctx, r, dynamoComponentDeployment, func(ctx context.Context) (*leaderworkersetv1.LeaderWorkerSet, bool, error) {
	return r.generateLeaderWorkerSet(ctx, generateResourceOption{
	dynamoComponentDeployment: dynamoComponentDeployment,
	isStealingTrafficDebugModeEnabled: false,
	containsStealingTrafficDebugModeEnabled: false,
	instanceID: &i,
	})
	})
	if err != nil {
	return ctrl.Result{}, err
	}
	if modified_ {
	anyModified = true
	}
	leaderWorkerSets = append(leaderWorkerSets, lwsObj)
	}

I see that Deployment and DynamoComponentDeploymentare are one-to-one mappings.

[julienmancuso]

I would like to ask why does DynamoComponentDeployment create replicas of leaderWorkerSet instead of just one leaderWorkerSet and set lws' replicas the same as dcd?

Hey @JesseStutler , We create one LeaderWorkerSet per DynamoComponentDeployment replica so that each “mini-cluster” has its own gang-scheduling group and independent lifecycle. A single LWS with replicas=N can’t express per-replica Volcano PodGroups or distinct scheduling groups, which we need for correct, granular scheduling and isolation.

basically, we need one LWS per replica to:

• Assign a distinct Volcano PodGroup and scheduling group-name per replica
• Keep gangs small and independent, improving placement success and fault isolation
• Enable clean scale-up/down and rolling updates per replica

[JesseStutler]

I would like to ask why does DynamoComponentDeployment create replicas of leaderWorkerSet instead of just one leaderWorkerSet and set lws' replicas the same as dcd?

Hey @JesseStutler , We create one LeaderWorkerSet per DynamoComponentDeployment replica so that each “mini-cluster” has its own gang-scheduling group and independent lifecycle. A single LWS with replicas=N can’t express per-replica Volcano PodGroups or distinct scheduling groups, which we need for correct, granular scheduling and isolation.

basically, we need one LWS per replica to:

• Assign a distinct Volcano PodGroup and scheduling group-name per replica
• Keep gangs small and independent, improving placement success and fault isolation
• Enable clean scale-up/down and rolling updates per replica

Thanks for your reply. LWS v0.7.0 now supports gang scheduling for each lws's replica, which means lws-controller will create volcano podgroup for each replica: kubernetes-sigs/lws#498. So I am wondering if it is necessary for the dcd controller to create a lws for each dcd replica. Besides, does dcd suppport rolling update? Seems that if dcd use lws, and if it updates, then all the lws down it will update at the same time. Is this correct? But if one dcd corresponds to one lws, it can utilize the lws's ability to rolling upgrade. @julienmancuso

[julienmancuso]

Hey @JesseStutler , this implementation was done to workaround LWS not supporting gang scheduling.
Now that new version of LWS is out, we can switch to using it. And yes we then can use only 1 LWS for 1 DCD.

[deepcoldfish]

how to use dynamo deploy sglang on multi-node with leaderworkerset like lws example of sglang ? @julienmancuso

@deepcoldfish Seems that you're finding this https://github.com/ai-dynamo/dynamo/blob/main/components/backends/sglang/deploy/disagg.yaml ?

I think it's not enough，in multi-node cmd in workerPodTemplate could be different from that in leaderPodTemplate