Skip to content

Add Helm job for OpenSearch index creation #36

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bramwelt wants to merge 6 commits into
base: main
Choose a base branch
from
+176 −1
Open

Add Helm job for OpenSearch index creation #36

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3512a65
Add Helm job for OpenSearch index creation
bramwelt Jan 6, 2026
234dc9c
Improve OpenSearch index creation job
bramwelt Jan 7, 2026
4f5d8bb
Use release name prefix for job and configmap names
bramwelt Jan 7, 2026
f4b068d
Merge branch 'main' into bramwelt/indexer-job
bramwelt Jan 7, 2026
34edd78
Add OpenSearch authentication support to index setup job
bramwelt Jan 7, 2026
f9577f5
Merge branch 'main' into bramwelt/indexer-job
bramwelt Jan 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/lfx-v2-indexer-service/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,5 @@ apiVersion: v2
name: lfx-v2-indexer-service
description: LFX Platform V2 Indexer Service chart
type: application
version: 0.4.12
version: 0.5.0
appVersion: "latest"
48 changes: 48 additions & 0 deletions charts/lfx-v2-indexer-service/files/opensearch-resources-index.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
{
"mappings": {
"properties": {
"object_ref": { "type": "keyword" },
"object_type": { "type": "keyword" },
"object_id": { "type": "keyword" },
"parent_refs": { "type": "keyword" },
"sort_name": { "type": "keyword" },
"name_and_aliases": { "type": "search_as_you_type" },
"tags": { "type": "keyword" },
"public": { "type": "boolean" },
"access_check_query": { "type": "keyword" },
"history_check_query": { "type": "keyword" },
"latest": { "type": "boolean" },
"created_at": { "type": "date" },
"created_by": { "type": "keyword" },
"created_by_principals": { "type": "keyword" },
"created_by_emails": { "type": "keyword" },
"updated_at": { "type": "date" },
"updated_by": { "type": "keyword" },
"updated_by_principals": { "type": "keyword" },
"updated_by_emails": { "type": "keyword" },
"deleted_at": { "type": "date" },
"deleted_by": { "type": "keyword" },
"deleted_by_principals": { "type": "keyword" },
"deleted_by_emails": { "type": "keyword" },
"data": {
"type": "flat_object"
},
"fulltext": {
"type": "match_only_text"
},
"contacts": {
"type": "nested",
"properties": {
"lfx_principal": { "type": "search_as_you_type" },
"name": { "type": "search_as_you_type" },
"emails": { "type": "search_as_you_type" },
"bot": { "type": "boolean" },
"profile": { "type": "flat_object" }
}
},
"v1_data": {
"type": "flat_object"
}
}
}
}
13 changes: 13 additions & 0 deletions charts/lfx-v2-indexer-service/templates/indexing-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright The Linux Foundation and each contributor to LFX.
# SPDX-License-Identifier: MIT
{{- if .Values.opensearch.indexingJob.enabled }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-opensearch-index-config
namespace: {{ .Release.Namespace }}
data:
resources-index.json: |
Copy link
Contributor

@andrest50 andrest50 Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This index json content doesn't quite match what we should be actually using, which is https://github.com/linuxfoundation/lfx-architecture-scratch/blob/main/2024-12%20ReBAC%20Demo/opensearch-resources-index.json. But on that note, could we have this json content in a file instead and then have it read into the k8 template? I think that would be better.

Copy link
Author

@bramwelt bramwelt Jan 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback! I realized I hadn't pulled the repo for a while and was using an outdated index.

I've moved the index JSON to it's own file and have it being included now with .Files.Get. I also update the job to not attempt to create the index if it already exists.

{{ .Files.Get "files/opensearch-resources-index.json" | nindent 4 }}
{{- end }}
51 changes: 51 additions & 0 deletions charts/lfx-v2-indexer-service/templates/job.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Copyright The Linux Foundation and each contributor to LFX.
# SPDX-License-Identifier: MIT
{{- if .Values.opensearch.indexingJob.enabled }}
---
apiVersion: batch/v1
kind: Job
metadata:
name: {{ .Release.Name }}-opensearch-index-setup
namespace: {{ .Release.Namespace }}
spec:
ttlSecondsAfterFinished: 300
backoffLimit: 3
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Job's ttlSecondsAfterFinished (300 seconds) and backoffLimit (3) are hardcoded. Consider making these values configurable through values.yaml (e.g., opensearch.indexingJob.ttlSecondsAfterFinished and opensearch.indexingJob.backoffLimit) to allow users to adjust cleanup timing and retry behavior based on their operational requirements.

Suggested change
ttlSecondsAfterFinished: 300
backoffLimit: 3
ttlSecondsAfterFinished: {{ .Values.opensearch.indexingJob.ttlSecondsAfterFinished | default 300 }}
backoffLimit: {{ .Values.opensearch.indexingJob.backoffLimit | default 3 }}

Copilot uses AI. Check for mistakes.
template:
spec:
restartPolicy: OnFailure
volumes:
- name: index-config
configMap:
name: {{ .Release.Name }}-opensearch-index-config
containers:
- name: curl
image: curlimages/curl:8.11.1
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The curl image and version are hardcoded in the Job template. Consider making the image repository and tag configurable through values.yaml (e.g., opensearch.indexingJob.image and opensearch.indexingJob.imageTag) to allow users to specify alternative images or pin specific versions according to their organization's policies.

Suggested change
image: curlimages/curl:8.11.1
image: {{ default "curlimages/curl" .Values.opensearch.indexingJob.image }}:{{ default "8.11.1" .Values.opensearch.indexingJob.imageTag }}

Copilot uses AI. Check for mistakes.
volumeMounts:
- name: index-config
mountPath: /config
readOnly: true
Comment on lines 25 to 56
Copy link

Copilot AI Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Job container doesn't define a securityContext, while the main application deployment includes one with allowPrivilegeEscalation: false. For consistency and security best practices, consider adding a similar securityContext to the Job container to run with minimal privileges.

Copilot uses AI. Check for mistakes.
command:
- sh
- -c
- |
set -e
OPENSEARCH_URL="{{ .Values.opensearch.url | trimSuffix "/" }}"
INDEX_NAME="{{ .Values.opensearch.index }}"

# Check if index already exists
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" -X HEAD "${OPENSEARCH_URL}/${INDEX_NAME}")

if [ "$HTTP_CODE" = "200" ]; then
echo "Index '${INDEX_NAME}' already exists, skipping creation"
exit 0
elif [ "$HTTP_CODE" = "404" ]; then
echo "Index '${INDEX_NAME}' does not exist, creating..."
curl -f -X PUT "${OPENSEARCH_URL}/${INDEX_NAME}" \
-H 'Content-Type: application/json' \
-d @/config/resources-index.json
echo "Index '${INDEX_NAME}' created successfully"
else
echo "Unexpected response checking index: HTTP ${HTTP_CODE}"
exit 1
fi
{{- end }}
4 changes: 4 additions & 0 deletions charts/lfx-v2-indexer-service/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ opensearch:
url: http://opensearch-cluster-master.lfx.svc.cluster.local:9200/
# index is the index name for storing resources
index: resources
# indexingJob is the configuration for the OpenSearch index creation job
indexingJob:
# enabled is a boolean to determine if the indexing job should be created
enabled: true

# heimdall is the configuration for the heimdall middleware
heimdall:
Expand Down