Skip to content

Commit afbf0de

Browse files
MarcoSabaSkraye
MarcoSaba
and
Skraye
authored
feat(helm): migrate kestra and kestra-starter charts to GCS (#15759)
* feat(helm)/migrate-artifacts-with-cicd * feat(helm): move kestra and kestra-starter charts into repo Source of truth for charts moves from kestra-io/helm-charts to this repo. Sources URL updated to kestra-io/kestra in both Chart.yaml files. Release workflow will manage version bumps and GCS uploads. * feat(helm): add helm-release workflow for GCS publishing On tag push (v*) or manual dispatch: bumps version/appVersion in both Chart.yaml files, packages kestra then kestra-starter (with kestra bundled as dep), merges into existing GCS index.yaml, uploads. Uses WIF auth — no SA key. Dry-run mode available for testing. * feat(helm)/rm-rc-trigger-helm-chart * feat(helm): run helm release after docker publish instead of on tag push * fix(helm): grant id-token:write to helm-release caller job Without explicit permissions on the caller job, GitHub mints the GITHUB_TOKEN without id-token:write, causing Workload Identity Federation auth to fail at every real release. * fix(helm): lint kestra-starter chart before packaging kestra was linted but kestra-starter was packaged directly, so template/schema errors in the starter chart could ship silently. --------- Co-authored-by: YannC. <ycoornaert@kestra.io>
1 parent 4716c3f commit afbf0de

33 files changed

Lines changed: 3149 additions & 1 deletion

‎.github/workflows/helm-migrate-artifacts.yml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
name: Helm migrate artifacts to GCS
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
dry_run:
7+
description: "If true, list files to copy but do not upload"
8+
required: true
9+
type: boolean
10+
default: true
11+
12+
permissions:
13+
contents: read
14+
id-token: write
15+
16+
jobs:
17+
migrate:
18+
name: Copy gh-pages artifacts to GCS
19+
runs-on: ubuntu-latest
20+
env:
21+
BUCKET: helm-charts-kestra-host
22+
23+
steps:
24+
- name: Checkout helm-charts gh-pages
25+
uses: actions/checkout@v5
26+
with:
27+
repository: kestra-io/helm-charts
28+
ref: gh-pages
29+
path: gh-pages
30+
31+
- name: Authenticate to GCP
32+
uses: google-github-actions/auth@v2
33+
with:
34+
workload_identity_provider: projects/230725787862/locations/global/workloadIdentityPools/github/providers/github
35+
service_account: helm-charts-publisher@kestra-host.iam.gserviceaccount.com
36+
37+
- uses: google-github-actions/setup-gcloud@v2
38+
39+
- name: List artifacts
40+
run: |
41+
echo "=== .tgz files ==="
42+
ls gh-pages/*.tgz | wc -l
43+
echo "=== index.yaml ==="
44+
ls gh-pages/index.yaml
45+
echo "=== other tracked files ==="
46+
ls gh-pages/artifacthub-repo.yml gh-pages/CNAME 2>/dev/null || true
47+
48+
- name: Dry run — show what would be uploaded
49+
if: ${{ inputs.dry_run }}
50+
run: |
51+
echo "DRY RUN — no upload performed."
52+
echo ""
53+
echo "Would upload to gs://${BUCKET}/:"
54+
gsutil ls gh-pages/*.tgz gh-pages/index.yaml | sed 's|gh-pages/| |'
55+
56+
- name: Upload artifacts
57+
if: ${{ !inputs.dry_run }}
58+
run: |
59+
set -euo pipefail
60+
61+
# Upload .tgz files first so index.yaml never points at a missing tarball.
62+
gsutil -m cp gh-pages/*.tgz gs://${BUCKET}/
63+
gsutil cp gh-pages/index.yaml gs://${BUCKET}/index.yaml
64+
gsutil cp gh-pages/artifacthub-repo.yml gs://${BUCKET}/artifacthub-repo.yml 2>/dev/null || true
65+
66+
- name: Verify count
67+
if: ${{ !inputs.dry_run }}
68+
run: |
69+
set -euo pipefail
70+
71+
LOCAL=$(ls gh-pages/*.tgz | wc -l)
72+
REMOTE=$(gsutil ls "gs://${BUCKET}/*.tgz" | wc -l)
73+
74+
echo "Local .tgz: ${LOCAL}"
75+
echo "Remote .tgz: ${REMOTE}"
76+
77+
if [ "${LOCAL}" != "${REMOTE}" ]; then
78+
echo "ERROR: count mismatch — migration incomplete."
79+
exit 1
80+
fi
81+
82+
echo "OK: ${REMOTE} .tgz files in bucket."
83+
echo ""
84+
echo "Spot-check index.yaml:"
85+
gsutil cat gs://${BUCKET}/index.yaml | head -20

‎.github/workflows/helm-release.yml‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
name: Helm release
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
version:
7+
description: 'Kestra version to release (e.g. 0.21.1, without v prefix)'
8+
required: true
9+
type: string
10+
dry_run:
11+
description: 'Dry run — package charts but do not upload to GCS'
12+
required: true
13+
type: boolean
14+
default: true
15+
workflow_call:
16+
inputs:
17+
dry_run:
18+
description: 'Dry run — package charts but do not upload to GCS'
19+
required: false
20+
type: boolean
21+
default: false
22+
23+
permissions:
24+
contents: read
25+
id-token: write
26+
27+
jobs:
28+
helm-release:
29+
name: Package and publish Helm charts
30+
runs-on: ubuntu-latest
31+
env:
32+
BUCKET: helm-charts-kestra-host
33+
REPO_URL: https://helm.kestra.io/
34+
35+
steps:
36+
- name: Checkout
37+
uses: actions/checkout@v5
38+
39+
- name: Resolve version and dry-run flag
40+
id: config
41+
run: |
42+
if [[ -n "${{ inputs.version }}" ]]; then
43+
VERSION="${{ inputs.version }}"
44+
TAG="v${VERSION}"
45+
else
46+
TAG="${GITHUB_REF_NAME}"
47+
VERSION="${TAG#v}"
48+
fi
49+
DRY_RUN="${{ inputs.dry_run }}"
50+
echo "tag=${TAG}" >> $GITHUB_OUTPUT
51+
echo "version=${VERSION}" >> $GITHUB_OUTPUT
52+
echo "dry_run=${DRY_RUN}" >> $GITHUB_OUTPUT
53+
echo "Version: ${VERSION} | Tag: ${TAG} | Dry run: ${DRY_RUN}"
54+
55+
- name: Update Chart.yaml versions
56+
env:
57+
VERSION: ${{ steps.config.outputs.version }}
58+
TAG: ${{ steps.config.outputs.tag }}
59+
run: |
60+
set -euo pipefail
61+
for chart in charts/kestra charts/kestra-starter; do
62+
sed -i "s/^version:.*/version: \"${VERSION}\"/" ${chart}/Chart.yaml
63+
sed -i "s/^appVersion:.*/appVersion: \"${TAG}\"/" ${chart}/Chart.yaml
64+
done
65+
# Update kestra dep version in kestra-starter
66+
awk -v ver="${VERSION}" '
67+
/- name: kestra/ { found=1 }
68+
found && /version:/ { sub(/version:.*/, "version: \"" ver "\""); found=0 }
69+
{ print }
70+
' charts/kestra-starter/Chart.yaml > /tmp/chart-starter.yaml
71+
mv /tmp/chart-starter.yaml charts/kestra-starter/Chart.yaml
72+
73+
- name: Lint kestra chart
74+
run: |
75+
helm lint charts/kestra \
76+
--set "configurations.application=kestra.variable.globals.foo: bar"
77+
78+
- name: Package kestra chart
79+
run: |
80+
mkdir -p release/
81+
helm package charts/kestra -d release/
82+
83+
- name: Bundle kestra as kestra-starter dependency
84+
env:
85+
VERSION: ${{ steps.config.outputs.version }}
86+
run: |
87+
# Place freshly packaged kestra tgz as bundled dep so helm package
88+
# does not need network access to resolve the kestra dependency.
89+
mkdir -p charts/kestra-starter/charts/
90+
cp release/kestra-${VERSION}.tgz charts/kestra-starter/charts/
91+
# Remove stale Chart.lock — version bump makes it invalid
92+
rm -f charts/kestra-starter/Chart.lock
93+
94+
- name: Lint kestra-starter chart
95+
run: helm lint charts/kestra-starter
96+
97+
- name: Package kestra-starter chart
98+
run: helm package charts/kestra-starter -d release/
99+
100+
- name: Verify packaged charts render
101+
env:
102+
VERSION: ${{ steps.config.outputs.version }}
103+
run: |
104+
helm template test release/kestra-${VERSION}.tgz \
105+
--set "configurations.application=kestra.variable.globals.foo: bar" > /dev/null
106+
echo "kestra-${VERSION}.tgz renders OK"
107+
helm template test release/kestra-starter-${VERSION}.tgz > /dev/null
108+
echo "kestra-starter-${VERSION}.tgz renders OK"
109+
110+
- name: Show packaged charts
111+
run: ls -lh release/*.tgz
112+
113+
- name: Authenticate to GCP
114+
uses: google-github-actions/auth@v2
115+
with:
116+
workload_identity_provider: projects/230725787862/locations/global/workloadIdentityPools/github/providers/github
117+
service_account: helm-charts-publisher@kestra-host.iam.gserviceaccount.com
118+
119+
- uses: google-github-actions/setup-gcloud@v2
120+
121+
- name: Download existing index.yaml from GCS
122+
run: gsutil cp gs://${BUCKET}/index.yaml existing-index.yaml
123+
124+
- name: Generate merged index.yaml
125+
run: |
126+
helm repo index release/ --merge existing-index.yaml --url ${REPO_URL}
127+
echo "=== Preview (first 30 lines) ==="
128+
head -30 release/index.yaml
129+
130+
- name: Dry run — show what would be uploaded
131+
if: ${{ steps.config.outputs.dry_run == 'true' }}
132+
run: |
133+
echo "DRY RUN — no upload performed."
134+
echo ""
135+
echo "Would upload to gs://${BUCKET}/:"
136+
ls -lh release/*.tgz release/index.yaml
137+
138+
- name: Upload to GCS
139+
if: ${{ steps.config.outputs.dry_run == 'false' }}
140+
run: |
141+
set -euo pipefail
142+
# Upload .tgz files first — index must never reference a missing artifact
143+
gsutil -m cp release/*.tgz gs://${BUCKET}/
144+
gsutil cp release/index.yaml gs://${BUCKET}/index.yaml
145+
echo "Published:"
146+
ls -lh release/*.tgz

‎.github/workflows/release-docker.yml‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,4 +39,15 @@ jobs:
3939
DOCKERHUB_PASSWORD: ${{ secrets.DOCKERHUB_PASSWORD }}
4040
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
4141
SLACK_RELEASES_WEBHOOK_URL: ${{ secrets.SLACK_RELEASES_WEBHOOK_URL }}
42-
GH_PERSONAL_TOKEN: ${{ secrets.GH_PERSONAL_TOKEN }}
42+
GH_PERSONAL_TOKEN: ${{ secrets.GH_PERSONAL_TOKEN }}
43+
44+
helm-release:
45+
name: Helm release
46+
needs: publish-docker
47+
if: startsWith(github.ref, 'refs/tags/v')
48+
permissions:
49+
contents: read
50+
id-token: write
51+
uses: ./.github/workflows/helm-release.yml
52+
with:
53+
dry_run: ${{ inputs.dry-run }}

‎charts/kestra-starter/.helmignore‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

‎charts/kestra-starter/Chart.lock‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
dependencies:
2+
- name: postgres
3+
repository: https://groundhog2k.github.io/helm-charts/
4+
version: 1.5.7
5+
- name: kestra
6+
repository: https://helm.kestra.io/
7+
version: 1.0.28
8+
digest: sha256:ab414a8c0a141f4ded784b7d37aaaa0153e3413ac7cb394f33b50065afb49592
9+
generated: "2026-01-21T12:08:57.418806968+01:00"

‎charts/kestra-starter/Chart.yaml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
apiVersion: v2
2+
type: application
3+
version: "1.1.23"
4+
appVersion: "v1.3.14"
5+
name: kestra-starter
6+
description: Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
7+
home: https://kestra.io
8+
icon: https://kestra.io/favicon-192x192.png
9+
sources:
10+
- https://github.com/kestra-io/kestra
11+
maintainers:
12+
- name: tchiotludo
13+
email: tchiot.ludo@gmail.com
14+
annotations:
15+
artifacthub.io/links: |
16+
- name: Documentation
17+
url: https://kestra.io/docs/
18+
artifacthub.io/screenshots: |
19+
- title: Home page
20+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/02-Homepage-Empty.png
21+
- title: Flows list
22+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/04-Flows.png
23+
- title: Flow page
24+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/05-Flows-Flow.png
25+
- title: Execution list
26+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/08-Executions.png
27+
- title: Execution page
28+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/09-Executions-Execution.png
29+
- title: Execution gantt
30+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/27-Executions-Gantt.png
31+
- title: Namespaces list
32+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/14-EE-Namespace.png
33+
- title: Global logs
34+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/10-Logs.png
35+
- title: Documentation
36+
url: https://raw.githubusercontent.com/kestra-io/kestra.io/main/public/docs/user-interface-guide/12-Documentations-Plugins-Plugin.png
37+
dependencies:
38+
- name: postgres
39+
repository: https://groundhog2k.github.io/helm-charts/
40+
version: "1.5.7"
41+
- name: kestra
42+
repository: https://helm.kestra.io/
43+
version: "1.0.53"

0 commit comments

Comments
 (0)