Skip to content

Conversation

@afdesk
Copy link
Contributor

@afdesk afdesk commented Sep 26, 2025

Description

This PR disables parallel traversal of Kubernetes artifacts when the fs cache is enabled.

Since the fs cache relies on bbolt as its underlying database, any parallel traversal inevitably becomes bottlenecked by database access. As a result, concurrency provides no real performance benefit in this context.

To reflect this change, integration tests have been updated. Instead of only verifying the presence of vulnerabilities and misconfigurations, the tests now validate exact results, ensuring stronger guarantees and more reliable regression detection.

Before:

$ trivy k8s --report=summary --scanners=vuln -q --include-namespaces kube-system
2025-09-22T11:42:05+06:00       ERROR   Error during vulnerabilities or misconfiguration scan   err="scan error: unable to initialize a scan service: unable to initialize an image scan service: unable to initialize fs cache: cache may be in use by another process: timeout"
2025-09-22T11:42:05+06:00       ERROR   Error during vulnerabilities or misconfiguration scan   err="scan error: unable to initialize a scan service: unable to initialize an image scan service: unable to initialize fs cache: cache may be in use by another process: timeout"
2025-09-22T11:42:05+06:00       ERROR   Error during vulnerabilities or misconfiguration scan   err="scan error: unable to initialize a scan service: unable to initialize an image scan service: unable to initialize fs cache: cache may be in use by another process: timeout"
2025-09-22T11:42:05+06:00       ERROR   Error during vulnerabilities or misconfiguration scan   err="scan error: unable to initialize a scan service: unable to initialize an image scan service: unable to initialize fs cache: cache may be in use by another process: timeout"

Summary Report for minikube


Workload Assessment
┌───────────┬──────────┬───────────────────┐
│ Namespace │ Resource │  Vulnerabilities  │
│           │          ├───┬───┬───┬───┬───┤
│           │          │ C │ H │ M │ L │ U │
└───────────┴──────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Infra Assessment
┌─────────────┬─────────────────────────────────────────┬──────────────────────┐
│  Namespace  │                Resource                 │   Vulnerabilities    │
│             │                                         ├───┬────┬────┬────┬───┤
│             │                                         │ C │ H  │ M  │ L  │ U │
├─────────────┼─────────────────────────────────────────┼───┼────┼────┼────┼───┤
│ kube-system │ ControlPlaneComponents/k8s.io/apiserver │   │    │    │ 1  │   │
│ kube-system │ Pod/storage-provisioner                 │ 5 │ 56 │ 42 │ 1  │   │
│ kube-system │ DaemonSet/kube-proxy                    │   │ 1  │ 1  │ 16 │   │
└─────────────┴─────────────────────────────────────────┴───┴────┴────┴────┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

After:

./trivy k8s --report=summary --scanners=vuln -q --include-namespaces kube-system

Summary Report for minikube


Workload Assessment
┌───────────┬──────────┬───────────────────┐
│ Namespace │ Resource │  Vulnerabilities  │
│           │          ├───┬───┬───┬───┬───┤
│           │          │ C │ H │ M │ L │ U │
└───────────┴──────────┴───┴───┴───┴───┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN


Infra Assessment
┌─────────────┬─────────────────────────────────────────┬──────────────────────┐
│  Namespace  │                Resource                 │   Vulnerabilities    │
│             │                                         ├───┬────┬────┬────┬───┤
│             │                                         │ C │ H  │ M  │ L  │ U │
├─────────────┼─────────────────────────────────────────┼───┼────┼────┼────┼───┤
│ kube-system │ DaemonSet/kube-proxy                    │   │ 1  │ 1  │ 16 │   │
│ kube-system │ ControlPlaneComponents/k8s.io/apiserver │   │    │    │ 1  │   │
│ kube-system │ Deployment/coredns                      │   │ 4  │ 10 │    │   │
│ kube-system │ Pod/etcd-minikube                       │   │ 7  │ 13 │    │   │
│ kube-system │ Pod/storage-provisioner                 │ 5 │ 55 │ 43 │ 1  │   │
└─────────────┴─────────────────────────────────────────┴───┴────┴────┴────┴───┘
Severities: C=CRITICAL H=HIGH M=MEDIUM L=LOW U=UNKNOWN

Related issues

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

@knqyf263
Copy link
Collaborator

I have another idea for simplifying this implementation. What if merging #9504 and including this PR in v0.67.1?

@afdesk afdesk mentioned this pull request Sep 29, 2025
@afdesk
Copy link
Contributor Author

afdesk commented Sep 29, 2025

I have another idea for simplifying this implementation. What if merging #9504 and including this PR in v0.67.1?

we discussed this PR with @DmitriyLewen, and there is a good idea.
so I hope to follow up it today

#9504 is using in memory cache, actually, it's not a good way for k8s clusters with thouthands similar images...

@afdesk afdesk closed this Sep 29, 2025
@afdesk afdesk force-pushed the fix/k8s/common-fscache branch from 6cb1367 to e149094 Compare September 29, 2025 06:31
@afdesk afdesk reopened this Sep 29, 2025
@afdesk afdesk requested a review from DmitriyLewen September 29, 2025 10:25
@afdesk afdesk changed the title fix(k8s): use common fs cache for k8s image scans fix(k8s): disable parallel traversal with fs cache for k8s images Sep 29, 2025
@afdesk afdesk marked this pull request as ready for review September 29, 2025 10:38
Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@simar7 simar7 added this pull request to the merge queue Sep 30, 2025
Merged via the queue into aquasecurity:main with commit c0c7a6b Sep 30, 2025
13 checks passed
@afdesk afdesk deleted the fix/k8s/common-fscache branch September 30, 2025 05:30
alexlebens pushed a commit to alexlebens/infrastructure that referenced this pull request Sep 30, 2025
This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [mirror.gcr.io/aquasec/trivy](https://www.aquasec.com/products/trivy/) ([source](https://github.com/aquasecurity/trivy)) | minor | `0.66.0` -> `0.67.0` |

---

### Release Notes

<details>
<summary>aquasecurity/trivy (mirror.gcr.io/aquasec/trivy)</summary>

### [`v0.67.0`](https://github.com/aquasecurity/trivy/blob/HEAD/CHANGELOG.md#0670-2025-09-30)

[Compare Source](aquasecurity/trivy@v0.66.0...v0.67.0)

##### Features

- add documentation URL for database lock errors ([#&#8203;9531](aquasecurity/trivy#9531)) ([eba48af](aquasecurity/trivy@eba48af))
- **cli:** change --list-all-pkgs default to true ([#&#8203;9510](aquasecurity/trivy#9510)) ([7b663d8](aquasecurity/trivy@7b663d8))
- **cloudformation:** support default values and list results in Fn::FindInMap ([#&#8203;9515](aquasecurity/trivy#9515)) ([42b3bf3](aquasecurity/trivy@42b3bf3))
- **cyclonedx:** preserve SBOM structure when scanning SBOM files with vulnerability updates ([#&#8203;9439](aquasecurity/trivy#9439)) ([aff03eb](aquasecurity/trivy@aff03eb))
- **redhat:** add os-release detection for RHEL-based images ([#&#8203;9458](aquasecurity/trivy#9458)) ([cb25a07](aquasecurity/trivy@cb25a07))
- **sbom:** added support for CoreOS ([#&#8203;9448](aquasecurity/trivy#9448)) ([6d562a3](aquasecurity/trivy@6d562a3))
- **seal:** add seal support ([#&#8203;9370](aquasecurity/trivy#9370)) ([e4af279](aquasecurity/trivy@e4af279))

##### Bug Fixes

- **aws:** use `BuildableClient` insead of `xhttp.Client` ([#&#8203;9436](aquasecurity/trivy#9436)) ([fa6f1bf](aquasecurity/trivy@fa6f1bf))
- close file descriptors and pipes on error paths ([#&#8203;9536](aquasecurity/trivy#9536)) ([a4cbd6a](aquasecurity/trivy@a4cbd6a))
- **db:** Dowload database when missing but metadata still exists ([#&#8203;9393](aquasecurity/trivy#9393)) ([92ebc7e](aquasecurity/trivy@92ebc7e))
- **k8s:** disable parallel traversal with fs cache for k8s images ([#&#8203;9534](aquasecurity/trivy#9534)) ([c0c7a6b](aquasecurity/trivy@c0c7a6b))
- **misconf:** handle tofu files in module detection ([#&#8203;9486](aquasecurity/trivy#9486)) ([bfd2f6b](aquasecurity/trivy@bfd2f6b))
- **misconf:** strip build metadata suffixes from image history ([#&#8203;9498](aquasecurity/trivy#9498)) ([c938806](aquasecurity/trivy@c938806))
- **misconf:** unmark cty values before access ([#&#8203;9495](aquasecurity/trivy#9495)) ([8e40d27](aquasecurity/trivy@8e40d27))
- **misconf:** wrap legacy ENV values in quotes to preserve spaces ([#&#8203;9497](aquasecurity/trivy#9497)) ([267a970](aquasecurity/trivy@267a970))
- **nodejs:** parse workspaces as objects for package-lock.json files ([#&#8203;9518](aquasecurity/trivy#9518)) ([404abb3](aquasecurity/trivy@404abb3))
- **nodejs:** use snapshot string as `Package.ID` for pnpm packages ([#&#8203;9330](aquasecurity/trivy#9330)) ([4517e8c](aquasecurity/trivy@4517e8c))
- **vex:** don't  suppress vulns for packages with infinity loop ([#&#8203;9465](aquasecurity/trivy#9465)) ([78f0d4a](aquasecurity/trivy@78f0d4a))
- **vuln:** compare `nuget` package names in lower case ([#&#8203;9456](aquasecurity/trivy#9456)) ([1ff9ac7](aquasecurity/trivy@1ff9ac7))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xMTYuNiIsInVwZGF0ZWRJblZlciI6IjQxLjExNi42IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbWFnZSJdfQ==-->

Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1622
Co-authored-by: Renovate Bot <[email protected]>
Co-committed-by: Renovate Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(k8s): unable to initialize fs cache

4 participants