feat: honeycomb scaler #6896

kmoonwright · 2025-07-09T03:19:09Z

Honeycomb Scaler: Add New External Metrics Scaler

This PR introduces a new scaler for Honeycomb.io, which enables KEDA to scale workloads based on observability data and queries executed against Honeycomb datasets.

What Has Changed

New scaler implementation:
Adds scalers/honeycomb_scaler.go (and associated test files) that allows users to scale workloads based on Honeycomb queries.
Metadata and config parsing:
Supports both legacy config fields and flexible queryRaw JSON for arbitrary query construction.
API integration:
Interacts with the Honeycomb API for query creation, execution, and polling.
Edge case handling:
Handles response parsing, missing fields, non-numeric results, and timeouts robustly.
Unit tests:
Comprehensive tests for metadata parsing and result field extraction.
CRD and deployment instructions:
Updated CRD and deployment YAMLs to ensure compatibility with the new scaler.
Documentation PR:
See documentation PR for keda-docs repository #1587

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes #
Relates to #<link docs/helm PRs when opened>

Notes:

Please let me know if you’d like any adjustments to config or interface for the scaler before merging.
I will open a documentation PR after initial review.

dttung2905

Hey @kmoonwright , thanks so much for working on this. I strongly believe this is a useful addition to the list of scaler we support. I know it is still a draft but i want to give some feedbacks/ nits as early as possible. Hope you don't mind 😄

pkg/scalers/honeycomb_scaler.go

kmoonwright · 2025-07-09T18:45:12Z

Great feedback, thank you! Much appreciated even in the draft

…e#6856) Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: kmoonwright <[email protected]>

…re#6860) When the ScaledObject adopts an HPA or if the hpaName in the ScaledObject status is removed for any reason, the hpaName status will no longer get set. It is only set when the HPA is created by the ScaledObject. We add setting the HPA name in the ScaledObject status to the regular reconciliation loop for the ScaledObject. See kedacore#6336 Signed-off-by: Justin Miron <[email protected]> Signed-off-by: kmoonwright <[email protected]>

Signed-off-by: kmoonwright <[email protected]>

Signed-off-by: Jan Wozniak <[email protected]> Signed-off-by: kmoonwright <[email protected]>

…ter usage (kedacore#6843) Signed-off-by: Rick Brouwer <[email protected]> Signed-off-by: kmoonwright <[email protected]>

* Update Signed-off-by: zhenghanzhou <[email protected]> * Update Signed-off-by: zhenghanzhou <[email protected]> * Fix Signed-off-by: zhenghanzhou <[email protected]> * Update Signed-off-by: zhenghanzhou <[email protected]> --------- Signed-off-by: zhenghanzhou <[email protected]> Co-authored-by: zhenghanzhou <[email protected]> Co-authored-by: Zbynek Roubalik <[email protected]> Signed-off-by: kmoonwright <[email protected]>

…re#6780) Signed-off-by: Alexander Huck <[email protected]> Signed-off-by: Jorge Turrado Ferrero <[email protected]> Co-authored-by: Jorge Turrado Ferrero <[email protected]> Signed-off-by: kmoonwright <[email protected]>

) * fix: e2e test checks label value by name and not by index Signed-off-by: Jorge Turrado <[email protected]> * fix: e2e test checks label value by name and not by index Signed-off-by: Jorge Turrado <[email protected]> * fix: e2e test checks label value by name and not by index Signed-off-by: Jorge Turrado <[email protected]> --------- Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: kmoonwright <[email protected]>

Signed-off-by: kmoonwright <[email protected]>

zroubalik

Hi @kmoonwright, thanks for the contribution.

Could you please also add e2e tests for this scaler? This is a requirement for us in order to proceed forward towards merging.

Thanks!

Copilot

Pull Request Overview

This PR introduces a new Honeycomb scaler that enables KEDA to scale workloads based on observability data from Honeycomb.io. The scaler supports both legacy configuration fields and flexible raw JSON queries, integrates with the Honeycomb API for query execution, and includes comprehensive error handling and edge case management.

Adds complete Honeycomb scaler implementation with API integration
Provides flexible query configuration supporting both legacy fields and raw JSON
Includes comprehensive unit tests for metadata parsing and result extraction

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
pkg/scaling/scalers_builder.go	Registers the new honeycomb scaler in the scaler factory
pkg/scalers/honeycomb_scaler.go	Complete scaler implementation with API client and query logic
pkg/scalers/honeycomb_scaler_test.go	Comprehensive unit tests for metadata parsing and result extraction
CHANGELOG.md	Documents the addition of the new Honeycomb scaler

Comments suppressed due to low confidence (1)

pkg/scalers/honeycomb_scaler.go:175

[nitpick] The error message contains a very long URL and detailed explanation that may be truncated in logs. Consider shortening the message or moving the detailed explanation to documentation.

		return 0, fmt.Errorf("honeycomb: unauthorized (401) - an Enterprise API key is required, check your API key permissions. See: https://api-docs.honeycomb.io/api/query-data for details. Response: %s", string(body))

pkg/scalers/honeycomb_scaler.go

Signed-off-by: kmoonwright <[email protected]>

…ithin 10 seconds Signed-off-by: kmoonwright <[email protected]>

…te limits Signed-off-by: kmoonwright <[email protected]>

Signed-off-by: kmoonwright <[email protected]>

Resolved merge conflict in CHANGELOG.md by properly placing Honeycomb Scaler entry in the New section following contributing guidelines.

Signed-off-by: kmoonwright <[email protected]>

* refactor: rename temporal scaler files for consistency Renames the temporal scaler files to follow the convention used by other scalers in the project. - -> - -> - -> Signed-off-by: Aniruddh Chaturvedi <[email protected]> * address feedback Signed-off-by: Aniruddh Chaturvedi <[email protected]> --------- Signed-off-by: Aniruddh Chaturvedi <[email protected]> Signed-off-by: Aniruddh Chaturvedi <[email protected]> Signed-off-by: kmoonwright <[email protected]>

Reorder AWS DynamoDB scalers to match KEDA's lexicographic sorting requirements: - aws-dynamodb-streams now comes before aws-dynamodb - This resolves the CI failure for scaler alphabetical ordering Signed-off-by: kmoonwright <[email protected]>

- Reorder AWS DynamoDB scalers: aws-dynamodb before aws-dynamodb-streams - Reorder external scalers: external before external-mock and external-push - Reorder redis scalers: redis first, then redis-cluster, redis-cluster-streams, etc. - This matches the exact order in the main repository Signed-off-by: kmoonwright <[email protected]>

rickbrouwer · 2025-08-06T12:19:11Z

/run-e2e honeycomb
Update: You can check the progress here

JorTurFer · 2025-08-16T16:08:56Z

CHANGELOG.md


 ## Unreleased

+- **General**: Introduce new Honeycomb Scaler ([#6896](https://github.com/kedacore/keda/pull/6896/))


Move this to New section

JorTurFer · 2025-08-16T16:09:15Z

go.mod

 module github.com/kedacore/keda/v2

-go 1.23.8
+go 1.23


Why is this required?

JorTurFer · 2025-08-16T16:15:06Z

pkg/scalers/honeycomb_scaler.go

+}
+
+type honeycombMetadata struct {
+	APIKey              string                 `keda:"name=apiKey, order=authParams;triggerMetadata"`


We don't support reading secrets from metadata

Suggested change

APIKey string `keda:"name=apiKey, order=authParams;triggerMetadata"`

APIKey string `keda:"name=apiKey, order=authParams"`

pkg/scalers/honeycomb_scaler.go

JorTurFer · 2025-08-16T16:19:34Z

pkg/scalers/honeycomb_scaler.go

+	meta.TriggerIndex = config.TriggerIndex
+
+	// Use queryRaw if provided, else build query from legacy fields
+	if raw, ok := config.TriggerMetadata["queryRaw"]; ok && raw != "" {


Can't we just use the parsed value?

Suggested change

if raw, ok := config.TriggerMetadata["queryRaw"]; ok && raw != "" {

if meta.QueryRaw != "" {

JorTurFer · 2025-08-16T16:34:41Z

pkg/scalers/honeycomb_scaler.go

+		"disable_total_by_aggregate": true,
+		"disable_other_by_aggregate": true,
+		// Query results limit is 10000, see https://api-docs.honeycomb.io/api for details
+		"limit": honeycombQueryResultsLimit,


Does it make sense to expose this to users? I mean, the limit must be enforced, but maybe a user prefers to query X items instead of the limit. Maybe we can expose this to users and check that value doesn't pass the limit.
Or maybe this is not a valid use case, saying the truth, I don't have any experience with honeycomb

JorTurFer · 2025-08-16T16:35:14Z

pkg/scalers/honeycomb_scaler.go

+		body, _ := io.ReadAll(runResp.Body)
+		return 0, fmt.Errorf("honeycomb: unauthorized (401) - an Enterprise API key is required, check your API key permissions. See: https://api-docs.honeycomb.io/api/query-data for details. Response: %s", string(body))
+	}
+	if runResp.StatusCode != 200 && runResp.StatusCode != 201 {


Same as above

pkg/scalers/honeycomb_scaler.go

JorTurFer · 2025-08-16T16:42:33Z

pkg/scalers/honeycomb_scaler.go

+			statusResp.Body.Close()
+			return 0, errors.New("honeycomb: rate limited (429) on poll, back off and try again later")
+		}
+		if statusResp.StatusCode != 200 && statusResp.StatusCode != 201 {


same as above

JorTurFer · 2025-08-16T16:45:25Z

tests/scalers/honeycomb/honeycomb_test.go

@@ -0,0 +1,308 @@
+//go:build e2e


How are the metrics sent to honeycomb? I don't see any agent or so that collects them

zroubalik

@kmoonwright any update here please?

kmoonwright marked this pull request as draft July 9, 2025 14:25

kmoonwright changed the title ~~Feat/honeycomb scaler~~ feat: honeycomb scaler Jul 9, 2025

kmoonwright force-pushed the feat/honeycomb-scaler branch from d7a2845 to d66d23f Compare July 9, 2025 17:44

dttung2905 reviewed Jul 9, 2025

View reviewed changes

pkg/scalers/honeycomb_scaler.go Outdated Show resolved Hide resolved

pkg/scalers/honeycomb_scaler.go Outdated Show resolved Hide resolved

pkg/scalers/honeycomb_scaler.go Outdated Show resolved Hide resolved

kmoonwright force-pushed the feat/honeycomb-scaler branch from 2a97850 to 8e7d334 Compare July 9, 2025 18:53

JorTurFer and others added 19 commits July 9, 2025 12:57

fix: e2e test on ARM/s390x are correctly reported during PRs (kedacor…

44e2604

…e#6856) Signed-off-by: Jorge Turrado <[email protected]> Signed-off-by: kmoonwright <[email protected]>

Add Honeycomb to builder

4ebe3cc

Signed-off-by: kmoonwright <[email protected]>

Create Honeycomb integration and test file

81a8005

Signed-off-by: kmoonwright <[email protected]>

Update scaler query capabilities, and testing suite

86c53b0

Signed-off-by: kmoonwright <[email protected]>

Update test cases with response parser

a86e243

Signed-off-by: kmoonwright <[email protected]>

Update test cases with response parser

6ee4929

Signed-off-by: kmoonwright <[email protected]>

Formatting...

928ceea

Signed-off-by: kmoonwright <[email protected]>

chore: changelog and issue template v2.17.2 (kedacore#6845)

1c79b2d

Signed-off-by: Jan Wozniak <[email protected]> Signed-off-by: kmoonwright <[email protected]>

fix: resolve race condition in NSQ scaler tests by fixing atomic coun…

92cc9fe

…ter usage (kedacore#6843) Signed-off-by: Rick Brouwer <[email protected]> Signed-off-by: kmoonwright <[email protected]>

Add changelog entry for new Honeycomb Scaler

3fa2ca1

Signed-off-by: kmoonwright <[email protected]>

Change limit parameter to a constant

8243f56

Signed-off-by: kmoonwright <[email protected]>

Add error handling for json marchale and run query request

79d26d4

Signed-off-by: kmoonwright <[email protected]>

Add another error handling for create query

938274a

Signed-off-by: kmoonwright <[email protected]>

Add better response closing in polling for query results

123fccd

Signed-off-by: kmoonwright <[email protected]>

Formatting to go standards

2d6edac

Signed-off-by: kmoonwright <[email protected]>

kmoonwright force-pushed the feat/honeycomb-scaler branch 2 times, most recently from 1338abf to 2d6edac Compare July 9, 2025 20:27

kmoonwright and others added 3 commits July 9, 2025 13:27

Merge branch 'main' into feat/honeycomb-scaler

190c396

Update changelog to note new scaler in proper section

6f41074

Signed-off-by: kmoonwright <[email protected]>

Fix test imports for linter

8acf30e

Signed-off-by: kmoonwright <[email protected]>

kmoonwright marked this pull request as ready for review July 9, 2025 22:28

kmoonwright marked this pull request as draft July 9, 2025 23:40

kmoonwright and others added 3 commits July 10, 2025 15:19

Merge branch 'main' into feat/honeycomb-scaler

4e42692

Add error case for invalid key permissions for second Honeycomb call

b5c8bc8

Signed-off-by: kmoonwright <[email protected]>

Merge branch 'main' into feat/honeycomb-scaler

a8945b1

kmoonwright marked this pull request as ready for review July 16, 2025 20:16

zroubalik requested a review from Copilot July 21, 2025 15:25

zroubalik reviewed Jul 21, 2025

View reviewed changes

Copilot AI reviewed Jul 21, 2025

View reviewed changes

pkg/scalers/honeycomb_scaler.go Outdated Show resolved Hide resolved

pkg/scalers/honeycomb_scaler.go Show resolved Hide resolved

pkg/scalers/honeycomb_scaler.go Show resolved Hide resolved

kmoonwright and others added 14 commits July 25, 2025 13:01

Add initial e2e test file

64a1e66

Signed-off-by: kmoonwright <[email protected]>

Initial e2e test implementation

100678f

Signed-off-by: kmoonwright <[email protected]>

Add comment on Query Result API limitations and reference link

30b43dc

Signed-off-by: kmoonwright <[email protected]>

Add comment on query result timeout, which is limited to a response w…

c5bbf81

…ithin 10 seconds Signed-off-by: kmoonwright <[email protected]>

Add comment on polling configurations to work within query results ra…

016d171

…te limits Signed-off-by: kmoonwright <[email protected]>

Format it

9032827

Signed-off-by: kmoonwright <[email protected]>

Merge upstream/main into feat/honeycomb-scaler

7d88eca

Resolved merge conflict in CHANGELOG.md by properly placing Honeycomb Scaler entry in the New section following contributing guidelines.

Formatting for CI checks

01dcc86

Signed-off-by: kmoonwright <[email protected]>

Format E2E test

caaacd0

Signed-off-by: kmoonwright <[email protected]>

More... formatting

c8058bf

Signed-off-by: kmoonwright <[email protected]>

Reorder scaler builders alphabetically

4275ae7

Signed-off-by: kmoonwright <[email protected]>

kmoonwright force-pushed the feat/honeycomb-scaler branch from 3298c42 to f8d02ed Compare July 29, 2025 16:51

kmoonwright added 2 commits July 29, 2025 09:52

Merge branch 'main' into feat/honeycomb-scaler

dc6d97f

Merge branch 'main' into feat/honeycomb-scaler

cefafe4

JorTurFer reviewed Aug 16, 2025

View reviewed changes

zroubalik reviewed Aug 26, 2025

View reviewed changes

rickbrouwer added the waiting-author-response label Oct 2, 2025


		## Unreleased

		- General: Introduce new Honeycomb Scaler ([#6896](https://github.com/kedacore/keda/pull/6896/))

	APIKey string `keda:"name=apiKey, order=authParams;triggerMetadata"`
	APIKey string `keda:"name=apiKey, order=authParams"`

	if raw, ok := config.TriggerMetadata["queryRaw"]; ok && raw != "" {
	if meta.QueryRaw != "" {

feat: honeycomb scaler #6896

Are you sure you want to change the base?

feat: honeycomb scaler #6896

Conversation

kmoonwright commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Honeycomb Scaler: Add New External Metrics Scaler

What Has Changed

Checklist

Uh oh!

dttung2905 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kmoonwright commented Jul 9, 2025

Uh oh!

zroubalik left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rickbrouwer commented Aug 6, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zroubalik left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

kmoonwright commented Jul 9, 2025 •

edited

Loading

rickbrouwer commented Aug 6, 2025 •

edited by github-actions bot

Loading