24.8 Scan files for secrets in _upload_file_to_s3

[strtgbb]

• CI Fix or Improvement (changelog entry is not required)

Exclude tests:

• All Regression
• Disable CI Cache

Scan all plain text S3 uploads for strings that may reference secrets and for secrets found in the environment.
Upon finding such a fail, an error is raised and the workflow should abort.

Currently, I log the line number of the offending string, but since the full log might not be uploaded, would more context be desired?

Do we want to add scanning inside tgz/rpm/deb packages?

[altinity-robot]

This is an automated comment for commit 79ce999 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Check name	Description	Status
AST fuzzer	Runs randomly generated queries to catch program errors. The build type is optionally given in parenthesis. If it fails, ask a maintainer for help	❌ failure
Integration tests	The integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests	❌ failure
Sign aarch64	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	❌ error
Sign release	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	❌ error
Stateless tests	Runs stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	❌ failure

Successful checks

Check name	Description	Status
Builds	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Compatibility check	Checks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help	✅ success
Docker keeper image	The check to build and optionally push the mentioned image to docker hub	✅ success
Docker server image	The check to build and optionally push the mentioned image to docker hub	✅ success
Install packages	Checks that the built packages are installable in a clear environment	✅ success
Ready for release	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
Stateful tests	Runs stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	✅ success
Stress test	Runs stateless functional tests concurrently from several clients to detect concurrency-related errors	✅ success

[strtgbb]

There's a common false positive MINIO_ROOT_PASSWORD.

A few possible solutions

• Ignore *MINIO*
• Ignore =clickhouse
• Don't check for sensitive variables at all, just check sensitive values from the environment.