Skip to content

Add Helm job for OpenSearch index creation #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bramwelt wants to merge 6 commits into
base: main
Choose a base branch
from
+176 −1
Open

Add Helm job for OpenSearch index creation #36

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3512a65
Add Helm job for OpenSearch index creation
bramwelt Jan 6, 2026
234dc9c
Improve OpenSearch index creation job
bramwelt Jan 7, 2026
4f5d8bb
Use release name prefix for job and configmap names
bramwelt Jan 7, 2026
f4b068d
Merge branch 'main' into bramwelt/indexer-job
bramwelt Jan 7, 2026
34edd78
Add OpenSearch authentication support to index setup job
bramwelt Jan 7, 2026
f9577f5
Merge branch 'main' into bramwelt/indexer-job
bramwelt Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/lfx-v2-indexer-service/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ apiVersion: v2
name: lfx-v2-indexer-service
description: LFX Platform V2 Indexer Service chart
type: application
version: 0.4.12
version: 0.5.0
appVersion: "latest"
66 changes: 66 additions & 0 deletions charts/lfx-v2-indexer-service/templates/indexing-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Copyright The Linux Foundation and each contributor to LFX.
# SPDX-License-Identifier: MIT
{{- if .Values.opensearch.indexingJob.enabled }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: opensearch-index-config
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ConfigMap name is hardcoded and doesn't include the Release name, which could cause conflicts if multiple releases of this chart are installed in the same namespace. Consider using a template that includes the release name, such as {{ include "lfx-v2-indexer-service.fullname" . }}-index-config or {{ .Release.Name }}-opensearch-index-config to ensure uniqueness across releases.

Suggested change
name: opensearch-index-config
name: {{ include "lfx-v2-indexer-service.fullname" . }}-opensearch-index-config

Copilot uses AI. Check for mistakes.
namespace: {{ .Release.Namespace }}
data:
resources-index.json: |
Copy link
Contributor

@andrest50 andrest50 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This index json content doesn't quite match what we should be actually using, which is https://github.com/linuxfoundation/lfx-architecture-scratch/blob/main/2024-12%20ReBAC%20Demo/opensearch-resources-index.json. But on that note, could we have this json content in a file instead and then have it read into the k8 template? I think that would be better.

Copy link
Author

@bramwelt bramwelt Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! I realized I hadn't pulled the repo for a while and was using an outdated index.

I've moved the index JSON to it's own file and have it being included now with .Files.Get. I also update the job to not attempt to create the index if it already exists.

{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 1
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The index settings (number_of_shards and number_of_replicas) are hardcoded in the ConfigMap. Consider making these configurable through values.yaml to allow users to adjust shard and replica counts based on their cluster size and requirements without modifying the ConfigMap template directly.

Suggested change
"number_of_shards": 1,
"number_of_replicas": 1
"number_of_shards": {{ .Values.opensearch.indexingJob.numberOfShards | default 1 }},
"number_of_replicas": {{ .Values.opensearch.indexingJob.numberOfReplicas | default 1 }}

Copilot uses AI. Check for mistakes.
}
},
"mappings": {
"properties": {
"object_ref": {
"type": "keyword"
},
"object_type": {
"type": "keyword"
},
"object_id": {
"type": "keyword"
},
"latest": {
"type": "boolean"
},
"public": {
"type": "boolean"
},
"parent_refs": {
"type": "keyword"
},
"name_and_aliases": {
"type": "search_as_you_type"
},
"tags": {
"type": "keyword"
},
"access_check_object": {
"type": "keyword"
},
"access_check_relation": {
"type": "keyword"
},
"history_check_object": {
"type": "keyword"
},
"history_check_relation": {
"type": "keyword"
},
"access_check_query": {
"type": "keyword"
},
"history_check_query": {
"type": "keyword"
}
}
}
}
{{- end }}
34 changes: 34 additions & 0 deletions charts/lfx-v2-indexer-service/templates/job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Copyright The Linux Foundation and each contributor to LFX.
# SPDX-License-Identifier: MIT
{{- if .Values.opensearch.indexingJob.enabled }}
---
apiVersion: batch/v1
kind: Job
metadata:
name: opensearch-index-setup
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Job name is hardcoded and doesn't include the Release name, which could cause conflicts if multiple releases of this chart are installed in the same namespace. Consider using a template that includes the release name, such as {{ include "lfx-v2-indexer-service.fullname" . }}-index-setup or {{ .Release.Name }}-opensearch-index-setup to ensure uniqueness across releases.

Suggested change
name: opensearch-index-setup
name: {{ include "lfx-v2-indexer-service.fullname" . }}-opensearch-index-setup

Copilot uses AI. Check for mistakes.
namespace: {{ .Release.Namespace }}
spec:
ttlSecondsAfterFinished: 300
backoffLimit: 3
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Job's ttlSecondsAfterFinished (300 seconds) and backoffLimit (3) are hardcoded. Consider making these values configurable through values.yaml (e.g., opensearch.indexingJob.ttlSecondsAfterFinished and opensearch.indexingJob.backoffLimit) to allow users to adjust cleanup timing and retry behavior based on their operational requirements.

Suggested change
ttlSecondsAfterFinished: 300
backoffLimit: 3
ttlSecondsAfterFinished: {{ .Values.opensearch.indexingJob.ttlSecondsAfterFinished | default 300 }}
backoffLimit: {{ .Values.opensearch.indexingJob.backoffLimit | default 3 }}

Copilot uses AI. Check for mistakes.
template:
spec:
restartPolicy: Never
volumes:
- name: index-config
configMap:
name: opensearch-index-config
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ConfigMap reference uses a hardcoded name that should match the ConfigMap resource name. If the ConfigMap name is updated to include the Release name (as recommended), this reference must also be updated to use the same templated name to ensure proper binding.

Suggested change
name: opensearch-index-config
name: {{ .Release.Name }}-opensearch-index-config

Copilot uses AI. Check for mistakes.
containers:
- name: curl
image: curlimages/curl:8.11.1
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The curl image and version are hardcoded in the Job template. Consider making the image repository and tag configurable through values.yaml (e.g., opensearch.indexingJob.image and opensearch.indexingJob.imageTag) to allow users to specify alternative images or pin specific versions according to their organization's policies.

Suggested change
image: curlimages/curl:8.11.1
image: {{ default "curlimages/curl" .Values.opensearch.indexingJob.image }}:{{ default "8.11.1" .Values.opensearch.indexingJob.imageTag }}

Copilot uses AI. Check for mistakes.
volumeMounts:
- name: index-config
mountPath: /config
readOnly: true
Comment on lines 25 to 56
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Job container doesn't define a securityContext, while the main application deployment includes one with allowPrivilegeEscalation: false. For consistency and security best practices, consider adding a similar securityContext to the Job container to run with minimal privileges.

Copilot uses AI. Check for mistakes.
command:
- sh
- -c
- |
curl -X PUT "{{ .Values.opensearch.url | trimSuffix "/" }}/{{ .Values.opensearch.index }}" \
-H 'Content-Type: application/json' \
-d @/config/resources-index.json
{{- end }}
4 changes: 4 additions & 0 deletions charts/lfx-v2-indexer-service/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ opensearch:
url: http://opensearch-cluster-master.lfx.svc.cluster.local:9200/
# index is the index name for storing resources
index: resources
# indexingJob is the configuration for the OpenSearch index creation job
indexingJob:
# enabled is a boolean to determine if the indexing job should be created
enabled: true

# heimdall is the configuration for the heimdall middleware
heimdall:
Expand Down