Skip to content

feat: Render Helm ValuesObject as YAML in log output instead of binary (#18342)#27649

Merged
agaudreault merged 28 commits into
argoproj:masterfrom
subhramit:truncate-large-binary-logs
May 14, 2026
Merged

feat: Render Helm ValuesObject as YAML in log output instead of binary (#18342)#27649
agaudreault merged 28 commits into
argoproj:masterfrom
subhramit:truncate-large-binary-logs

Conversation

@subhramit

@subhramit subhramit commented May 2, 2026

Copy link
Copy Markdown
Contributor

Closes #18342

I have used a different approach compared to #23717 by keeping the fix entirely server-side.
In short, ValuesObject dumped raw bytes like [123 34 ...] which was both unreadable and large.
So by rendering it as YAML instead, the output is human-readable and compact, so it stops flooding Loki storage.
This was achieved by suppressing the stringer at proto-level and providing a custom implementation for String().

Also fixed some wrong log texts in repository.go where cache was being cleared instead of set (and the wording conveyed otherwise).

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Title of the PR
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.
  • Optional. My organization is added to USERS.md.
  • Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

Signed-off-by: subhramit <subhramit.bb@live.in>
@subhramit subhramit requested a review from a team as a code owner May 2, 2026 09:59
@bunnyshell

bunnyshell Bot commented May 2, 2026

Copy link
Copy Markdown

❌ Preview Environment deleted from Bunnyshell

Available commands (reply to this comment):

  • 🚀 /bns:deploy to deploy the environment

Signed-off-by: subhramit <subhramit.bb@live.in>
@codecov

codecov Bot commented May 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.87179% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.08%. Comparing base (5471c94) to head (3748168).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
reposerver/repository/repository.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #27649      +/-   ##
==========================================
+ Coverage   64.04%   64.08%   +0.04%     
==========================================
  Files         422      422              
  Lines       57828    57865      +37     
==========================================
+ Hits        37037    37084      +47     
+ Misses      17305    17295      -10     
  Partials     3486     3486              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@subhramit

Copy link
Copy Markdown
Contributor Author

ccing @agaudreault for a review, if this approach makes sense.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses repo-server log bloat caused by Helm valuesObject being rendered as raw byte arrays in manifest-cache log messages. It adds log-specific formatting for application sources so cache-related repo-server logs emit readable values text instead of binary-looking output.

Changes:

  • Add LogString() helpers for ApplicationSourceHelm and ApplicationSource to render Helm ValuesObject through the existing values helpers.
  • Update manifest cache hit/miss/error log statements in reposerver/repository/repository.go to use the new log formatter.
  • Add unit tests covering the new logging string behavior for Helm/application sources.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File Description
reposerver/repository/repository.go Swaps manifest-cache log messages over to the new log-specific source formatter.
pkg/apis/application/v1alpha1/values.go Adds LogString() helpers that rewrite ValuesObject into log-friendly text.
pkg/apis/application/v1alpha1/values_test.go Adds unit tests for the new LogString() behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/apis/application/v1alpha1/values.go Outdated
if s.Helm == nil || s.Helm.ValuesObject == nil {
return s.String()
}
sourceCopy := s.DeepCopy()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in d74a732

Comment thread reposerver/repository/repository.go Outdated
Comment on lines +1060 to +1064
log.Infof("manifest error cache miss: %s/%s", q.ApplicationSource.LogString(), cacheKey)
return false, res.ManifestResponse, nil
}

log.Infof("manifest cache hit: %s/%s", q.ApplicationSource.String(), cacheKey)
log.Infof("manifest cache hit: %s/%s", q.ApplicationSource.LogString(), cacheKey)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has merit, but was not introduced as a regression in this PR.
Happy to address separately in another PR, if that's fine? @agaudreault

Comment thread reposerver/repository/repository.go Outdated
Comment on lines +1013 to +1015
err = s.cache.DeleteManifests(cacheKey, q.ApplicationSource, q.RefSources, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, refSourceCommitSHAs, q.InstallationID)
if err != nil {
log.Warnf("manifest cache set error %s/%s: %v", q.ApplicationSource.String(), cacheKey, err)
log.Warnf("manifest cache set error %s/%s: %v", q.ApplicationSource.LogString(), cacheKey, err)

@subhramit subhramit May 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wording is also pre-existing, hence out of the PR's scope.
But I think the concern is valid.

Edit: this was a tiny one, so addressed in 0fd6f6b

Comment thread reposerver/repository/repository.go Outdated
Comment on lines +1028 to +1030
err = s.cache.DeleteManifests(cacheKey, q.ApplicationSource, q.RefSources, q, q.Namespace, q.TrackingMethod, q.AppLabelKey, q.AppName, refSourceCommitSHAs, q.InstallationID)
if err != nil {
log.Warnf("manifest cache set error %s/%s: %v", q.ApplicationSource.String(), cacheKey, err)
log.Warnf("manifest cache set error %s/%s: %v", q.ApplicationSource.LogString(), cacheKey, err)

@subhramit subhramit May 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as #27649 (comment)

Addressed in 0fd6f6b

Comment thread pkg/apis/application/v1alpha1/values.go Outdated
Comment on lines +66 to +68
helmCopy := *h
helmCopy.Values = h.ValuesString()
helmCopy.ValuesObject = nil

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea.
Covered in 1bdcc8f

subhramit added 4 commits May 3, 2026 00:01
Signed-off-by: subhramit <subhramit.bb@live.in>
Signed-off-by: subhramit <subhramit.bb@live.in>
Signed-off-by: subhramit <subhramit.bb@live.in>
@Qodo-Free-For-OSS

Copy link
Copy Markdown

Hi, Switching reposerver cache logs from .String() to .LogString() makes Helm ValuesObject content human-readable YAML, increasing the likelihood of accidental secret disclosure via Info/Warn logs if users store sensitive data in Helm values.

Severity: remediation recommended | Category: security

How to fix: Redact/truncate logged Helm values

Agent prompt to fix - you can give this to your LLM of choice:

Issue description

LogString() makes Helm values readable in reposerver logs. If Helm values contain secrets, this increases accidental exposure risk.

Issue Context

The previous logging also contained the data (as bytes), but the new format is substantially easier to read/search/copy.

Fix Focus Areas

  • pkg/apis/application/v1alpha1/values.go[56-88]
  • reposerver/repository/repository.go[939-1072]

Suggested approach

  • Implement a conservative max-length truncation for the rendered values in LogString().
  • Optionally redact common sensitive keys (e.g., password, token, secret, key) from the YAML string before logging.
  • Consider lowering verbosity of cache hit/miss logs or avoiding logging full values at Info/Warn levels.

Found by Qodo. Free code review for open-source maintainers.

@subhramit

subhramit commented May 3, 2026

Copy link
Copy Markdown
Contributor Author

Hi, Switching reposerver cache logs from .String() to .LogString() makes Helm ValuesObject content human-readable YAML, increasing the likelihood of accidental secret disclosure via Info/Warn logs if users store sensitive data in Helm values.

Severity: remediation recommended | Category: security

The information was disclosed previously as well, just in a different format and also trivially decodable into JSON back.
For a solution to the original issue which I intend to fix, I think either truncation or the YAML based approach suffice, with the latter being the neater one, but both together may be overkill (in the context of that issue).

Truncation/security concerns can be raised as a separate issue.

  • Optionally redact common sensitive keys (e.g., password, token, secret, key) from the YAML string before logging.

I don't agree - it would miss everything that doesn't match the wordlist. Helm values are user-defined arbitrary YAML which we should not concern ourselves with. It would also produce false positives that destroy debuggability. For example key: matches kafka_bootstrap_key:, partition_key:, private_key_path:, accessKeyId etc. would be blocked.

  • Consider lowering verbosity of cache hit/miss logs or avoiding logging full values at Info/Warn levels.

This may be a reasonable thing to consider but is pre-existing and orthogonal to this fix.
@agaudreault what do you think?

Comment thread pkg/apis/application/v1alpha1/values.go Outdated
Comment on lines +56 to +59
// LogString returns a human-readable string representation of ApplicationSourceHelm
// suitable for logging. It renders ValuesObject as its YAML equivalent instead of
// raw bytes, preventing binary data from flooding log output.
func (h *ApplicationSourceHelm) LogString() string {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not using the String function directly? This makes it error prone since most people will not think of using LogString

Suggested change
// LogString returns a human-readable string representation of ApplicationSourceHelm
// suitable for logging. It renders ValuesObject as its YAML equivalent instead of
// raw bytes, preventing binary data from flooding log output.
func (h *ApplicationSourceHelm) LogString() string {
func (h ApplicationSourceHelm) String() string {

@subhramit subhramit May 7, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had also thought about it, but generated.pb.go already has func (this *ApplicationSourceHelm) String() string. Since go doesn't allow value-receiver and pointer-receiver methods with the same name on a single type, it will not compile.

Image

IMO in hindsight, chances are non-zero, but unlikely that developers make that mistake with today's AI-assisted code suggestions, development tools and code reviews.

@subhramit subhramit May 8, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, I dug a little deeper - there seems to be no workaround for this except proto suppression protobuf.options.(gogoproto.goproto_stringer)=false, which IMO is not clean at all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agaudreault just a gentle ping in case you can go through the above and think the current state is good to go.

(Whenever you get time - I know you may be busy, I'm an OSS maintainer myself, but looking at the sheer number of PRs open I'd prefer this doesn't get buried if it adds value)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think setting the stringer=false annotation is best, so you can provide an implementation that prints the RawExtension type correctly.

@subhramit subhramit May 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@subhramit subhramit requested a review from agaudreault May 7, 2026 17:27
@subhramit subhramit changed the title fix: Render Helm ValuesObject as YAML in log output instead of binary (#18342) feat: Render Helm ValuesObject as YAML in log output instead of binary (#18342) May 8, 2026
subhramit and others added 9 commits May 8, 2026 22:04
Signed-off-by: subhramit <subhramit.bb@live.in>
Signed-off-by: subhramit-suki <sbasu@suki.ai>
Signed-off-by: subhramit-suki <sbasu@suki.ai>
Signed-off-by: subhramit-suki <sbasu@suki.ai>
Signed-off-by: subhramit-suki <sbasu@suki.ai>
Signed-off-by: subhramit-suki <sbasu@suki.ai>
subhramit added 3 commits May 12, 2026 00:08
Signed-off-by: subhramit <subhramit.bb@live.in>
Signed-off-by: subhramit <subhramit.bb@live.in>
@subhramit

This comment was marked as resolved.

// only observable difference is the ValuesObject rendering. If new fields are
// added to ApplicationSourceHelm, they must be added here as well - there is
// no codegen for this method.
func (h *ApplicationSourceHelm) String() string {

@subhramit subhramit May 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the additional "syntactical" differences from the codegen implementation are enhancements to satisfy the linters.

Semantically it's exactly the same (except for our fix on ValuesObject).

Comment thread assets/swagger.json
"v1alpha1ApplicationSourceHelm": {
"type": "object",
"title": "ApplicationSourceHelm holds helm specific options",
"title": "ApplicationSourceHelm holds helm specific options\n+protobuf.options.(gogoproto.goproto_stringer)=false",

@subhramit subhramit May 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't find a workaround for this, but the codebase contains many other similar examples:

argo-cd/assets/swagger.json

Lines 6521 to 6522 in 682de3e

"v1GroupKind": {
"description": "+protobuf.options.(gogoproto.goproto_stringer)=false",

argo-cd/assets/swagger.json

Lines 5831 to 5832 in 682de3e

"intstrIntOrString": {
"description": "+protobuf=true\n+protobuf.options.(gogoproto.goproto_stringer)=false\n+k8s:openapi-gen=true",

etc.

@agaudreault agaudreault left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but it might be better if we can wrap the RawExtension type

Comment thread pkg/apis/application/v1alpha1/types.go
@agaudreault agaudreault merged commit c61c1dc into argoproj:master May 14, 2026
28 checks passed
@subhramit subhramit deleted the truncate-large-binary-logs branch May 22, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Large binary log output in Loki

5 participants