Validate kibana tags duplicates #1042

teresaromero · 2025-12-16T15:20:20Z

What does this PR do?

This PR implements validation into kibana tags assets. When a tag is defined inside the tags.yml is duplicated and an asset is created when exporting the dashboard with elastic-package.

Why is it important?

Avoid duplicate tags at kibana UI, as .json assets when is already declared at tags.yml

Checklist

I have added test packages to test/packages that prove my change is effective.
I have added an entry in spec/changelog.yml.

Related issues

Resolves [Change Proposal] Add validation to avoid duplicate tags #977

teresaromero · 2025-12-16T15:22:57Z

code/go/internal/validator/spec.go

 		{fn: semantic.ValidateIntegrationPolicyTemplates, types: []string{"integration"}},
 		{fn: semantic.ValidatePipelineTags, types: []string{"integration"}, since: semver.MustParse("3.6.0")},
 		{fn: semantic.ValidateStaticHandlebarsFiles, types: []string{"integration", "input"}},
+		{fn: semantic.ValidateKibanaTagDuplicates, types: []string{"integration"}},


@jsoriano
content packages also have a tags.yml file, so perhaps we should check if inside the file there are duplicates? this is done inside this function now..

regarding the version, i guess this will "break" more than one integration, as for example crowdstrike is having the duplicate tags. should we address this fix at the integration repository and add this validation for all, or add it from 3.6.0 as a breaking change?

content packages also have a tags.yml file, so perhaps we should check if inside the file there are duplicates? this is done inside this function now..

Yes please, let's check this in all cases. We can probably enable this check for all types, if the files don't exist it doesn't do anything, right?

Suggested change

{fn: semantic.ValidateKibanaTagDuplicates, types: []string{"integration"}},

{fn: semantic.ValidateKibanaTagDuplicates},

regarding the version, i guess this will "break" more than one integration, as for example crowdstrike is having the duplicate tags. should we address this fix at the integration repository and add this validation for all, or add it from 3.6.0 as a breaking change?

I think this should to be reviewed in existing integrations, so I would check this in all packages, or at least in all versions where tags.yml is supported (it is supported since 2.10.0), what would be equivalent.

As escape hatch, I would assign it an error code, so it can be skipped if needed.

We would need to check in any case how many packages in the integrations repository are affected.

looking into the integrations test, as it was expected, mostly all are failing because of the Security Solution exception. Inline with my comment on elastic/elastic-package#3163 (comment)

i am guessing at least all the json corresponding to the security solution tag can be removed (safely)

i am guessing at least all the json corresponding to the security solution tag can be removed (safely)

Yes, but we would also need to remove the references to these ids from the dashboards, right?

yes, the json and the reference to that id in the dasboards or any other object where they are referenced

jsoriano

Overall approach LGTM 👍

code/go/internal/validator/semantic/validate_kibana_tag_duplicates.go

jsoriano · 2025-12-16T18:54:42Z

code/go/internal/validator/spec.go

 		{fn: semantic.ValidateIntegrationPolicyTemplates, types: []string{"integration"}},
 		{fn: semantic.ValidatePipelineTags, types: []string{"integration"}, since: semver.MustParse("3.6.0")},
 		{fn: semantic.ValidateStaticHandlebarsFiles, types: []string{"integration", "input"}},
+		{fn: semantic.ValidateKibanaTagDuplicates, types: []string{"integration"}},


content packages also have a tags.yml file, so perhaps we should check if inside the file there are duplicates? this is done inside this function now..

Yes please, let's check this in all cases. We can probably enable this check for all types, if the files don't exist it doesn't do anything, right?

Suggested change

{fn: semantic.ValidateKibanaTagDuplicates, types: []string{"integration"}},

{fn: semantic.ValidateKibanaTagDuplicates},

regarding the version, i guess this will "break" more than one integration, as for example crowdstrike is having the duplicate tags. should we address this fix at the integration repository and add this validation for all, or add it from 3.6.0 as a breaking change?

I think this should to be reviewed in existing integrations, so I would check this in all packages, or at least in all versions where tags.yml is supported (it is supported since 2.10.0), what would be equivalent.

As escape hatch, I would assign it an error code, so it can be skipped if needed.

We would need to check in any case how many packages in the integrations repository are affected.

code/go/pkg/validator/validator_test.go

code/go/internal/validator/semantic/validate_kibana_tag_duplicates.go

jsoriano · 2025-12-16T19:04:24Z

code/go/internal/validator/semantic/validate_kibana_tag_duplicates_test.go

+		require.Len(t, errs, 1)
+		require.Contains(t, errs[0].Error(), "duplicate tag name 'tag1' found in kibana/tags.yml")
+		require.Len(t, tagMap, 2)
+		require.Contains(t, tagMap, "tag1")
+		require.Contains(t, tagMap, "tag2")


Nit. Even on failure it is nice to return as much information as possible, use assert and conditions when the test can continue if the assertion fails.

Suggested change

require.Len(t, errs, 1)

require.Contains(t, errs[0].Error(), "duplicate tag name 'tag1' found in kibana/tags.yml")

require.Len(t, tagMap, 2)

require.Contains(t, tagMap, "tag1")

require.Contains(t, tagMap, "tag2")

if assert.Len(t, errs, 1) {

assert.Contains(t, errs[0].Error(), "duplicate tag name 'tag1' found in kibana/tags.yml")

}

assert.Len(t, tagMap, 2)

assert.Contains(t, tagMap, "tag1")

assert.Contains(t, tagMap, "tag2")

…e related tests

…alidate_kibana_tag_duplicates_test.go for consistency

teresaromero · 2025-12-17T10:34:23Z

test integrations

elastic-vault-github-plugin-prod · 2025-12-17T10:41:25Z

Created or updated PR in integrations repository to test this version. Check elastic/integrations#16597

code/go/internal/validator/semantic/validate_kibana_tag_duplicates.go

code/go/internal/validator/semantic/validate_kibana_tag_duplicates_test.go

mrodm · 2025-12-22T09:28:16Z

docs/validations.md

 | [SVR00004]          | Visualization by value                |
 | [SVR00005]          | Minimum Kibana version                |
 | [SVR00006]          | Processor tag is required             |
+| [SVR00007]          | Kibana tag is duplicate               |


There could be some conflicts with this other PR that is also adding the same validation code errors: #1038

https://github.com/elastic/package-spec/pull/1038/files#diff-d8846111760decbd765dc97b91c095ed8264c3f87d42449304e4735930437d1fR25-R26

…-tag-duplicates

elasticmachine · 2026-01-07T10:09:27Z

💚 Build Succeeded

Buildkite Build
Commit: ae229cb

History

💚 Build #1717 succeeded 031791d
💚 Build #1705 succeeded 60a9a0e
💚 Build #1702 succeeded 99a973e
💔 Build #1700 failed 8d7dc40

teresaromero added 3 commits December 16, 2025 11:33

Implement validation for duplicate Kibana tags in YAML and JSON files

82bd903

Handle non-existent directory errors gracefully in JSON tag validation

6fe6465

add validation for duplicate Kibana tag names

8d7dc40

teresaromero commented Dec 16, 2025

View reviewed changes

jsoriano reviewed Dec 16, 2025

View reviewed changes

teresaromero added 5 commits December 17, 2025 09:03

add copyright notice to validate_kibana_tag_duplicates_test.go

3b17a29

Apply tag duplicate validation to all packages

2d25032

Refactor tag validation to handle duplicates in Kibana tags and updat…

f1b5bea

…e related tests

Update validations and changelog for Kibana tag duplicate validation

1673c25

Refactor import statements in validate_kibana_tag_duplicates.go and v…

99a973e

…alidate_kibana_tag_duplicates_test.go for consistency

teresaromero changed the title ~~[draft] validate kibana tags duplicates~~ Validate kibana tags duplicates Dec 17, 2025

teresaromero marked this pull request as ready for review December 17, 2025 09:26

teresaromero requested a review from a team as a code owner December 17, 2025 09:26

elastic-vault-github-plugin-prod bot mentioned this pull request Dec 17, 2025

Test branch package-spec#1042 - DO NOT MERGE elastic/integrations#16597

Draft

jsoriano previously approved these changes Dec 18, 2025

View reviewed changes

code/go/internal/validator/semantic/validate_kibana_tag_duplicates.go Outdated Show resolved Hide resolved

code/go/internal/validator/semantic/validate_kibana_tag_duplicates_test.go Outdated Show resolved Hide resolved

Refactor path usage in tag validation functions to use filepath package

60a9a0e

teresaromero dismissed jsoriano’s stale review via 60a9a0e December 18, 2025 12:36

teresaromero mentioned this pull request Dec 19, 2025

Fix integrations with duplicated shared tags elastic/integrations#16634

Open

5 tasks

mrodm previously approved these changes Dec 22, 2025

View reviewed changes

teresaromero added 2 commits January 7, 2026 09:58

Merge branch 'main' of github.com:elastic/package-spec into 977-avoid…

031791d

…-tag-duplicates

change changelog entry to next patch

ae229cb

teresaromero dismissed mrodm’s stale review via ae229cb January 7, 2026 09:55

jsoriano approved these changes Jan 7, 2026

View reviewed changes

teresaromero merged commit 10653e8 into elastic:main Jan 8, 2026
3 checks passed

teresaromero mentioned this pull request Jan 8, 2026

feat: Add validators for pipeline on_failure handler #1038

Open

2 tasks

	{fn: semantic.ValidateKibanaTagDuplicates, types: []string{"integration"}},
	{fn: semantic.ValidateKibanaTagDuplicates},

Validate kibana tags duplicates #1042

Validate kibana tags duplicates #1042

Uh oh!

Conversation

teresaromero commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Related issues

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jsoriano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teresaromero commented Dec 17, 2025

Uh oh!

elastic-vault-github-plugin-prod bot commented Dec 17, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jan 7, 2026

💚 Build Succeeded

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

teresaromero commented Dec 16, 2025 •

edited

Loading