Skip to content

PS-1695: configurable controller-id to via CLI and env var #828

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zhming0 merged 1 commit into from
Mar 5, 2026
Merged

PS-1695: configurable controller-id to via CLI and env var #828

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions cmd/controller/controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,15 @@ unknown-field: some-value
assert.Contains(t, err.Error(), "unknown-field")
})

t.Run("ID settable via CLI flag", func(t *testing.T) {
cleanTestEnv(t)

cfg, err := buildConfig(t, []string{"--id=my-controller"}, "")
require.NoError(t, err)

assert.Equal(t, "my-controller", cfg.ID)
})

t.Run("CLI can override config file tags with empty", func(t *testing.T) {
cleanTestEnv(t)
configFile := createTempConfigFile(t, `
Expand Down Expand Up @@ -424,6 +433,7 @@ func cleanTestEnv(t *testing.T) {
"MAX_IN_FLIGHT",
"DEBUG",
"JOB_TTL",
"BUILDKITE_K8S_STACK_CONTROLLER_ID",
} {
t.Setenv(env, "")
os.Unsetenv(env)
Expand Down
2 changes: 2 additions & 0 deletions cmd/controller/flags.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ type CLI struct {
Namespace string `kong:"help='Kubernetes namespace to create resources in (default: default)'"`

// Controller settings
// name= and env= needed: Go field "ID" → Kong default "--i-d" and "$I_D", but we want "--id" and "$BUILDKITE_K8S_STACK_CONTROLLER_ID"
ID string `kong:"name='id',env='BUILDKITE_K8S_STACK_CONTROLLER_ID',help='Unique identifier for this controller instance. Used as a k8s label to filter resources and as the Stack key when registering with the Buildkite API. Must be distinct per controller when running multiple instances in the same namespace, otherwise duplicate pods may be spawned'"`
JobCreationConcurrency *int `kong:"help='Number of concurrent goroutines to run for converting Buildkite jobs into Kubernetes jobs (default: 25)'"`
AgentTokenSecret string `kong:"help='Name of the Buildkite agent token secret (default: buildkite-agent-token)'"`
Image string `kong:"help='The image to use for the Buildkite agent'"`
Expand Down
12 changes: 8 additions & 4 deletions internal/controller/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,10 +67,14 @@ type Config struct {
WorkQueueLimit int `json:"work-queue-limit" validate:"omitempty"`
// Agent endpoint is set in agent-config.

// ID is an optional uniquely ID string for the controller.
// This is useful when running multiple bk k8s controllers within the same k8s namespace.
// So the controller can target the correct pods.
// By default, if helm is used to install, this will be set as helm release full name.
// ID is an optional unique identifier for the controller instance.
// It is used as both a Kubernetes label (buildkite.com/controller-id) to filter
// resources, and as the Stack key when registering with the Buildkite API.
// When running multiple controllers in the same namespace, each must have a
// distinct ID so that they target the correct pods and register separate stacks.
// If two controllers share the same ID, both may successfully reserve the same
// job, causing duplicate pods to be spawned.
// By default, if Helm is used to install, this is set to the Helm release full name.
ID string `json:"id" validate:"omitempty"`

K8sClientRateLimiterQPS int `json:"k8s-client-rate-limiter-qps" validate:"omitempty"`
Expand Down