WIP Add e2e automation for audit logging feature

[BhargaviGudi]

Added e2e tests for audit logging feature testing

What type of PR is this?

/kind cleanup
/kind feature

What this PR does / why we need it:

This PR adds comprehensive end-to-end test automation for the audit logging feature (JSON enricher) in the Security Profiles Operator. It adds 7 new automated test cases (TC3-TC9) covering various
scenarios:

• TC3: Privileged Pods - Tests audit logging for privileged containers with root user execution
• TC4: File Output, Filtering and Rotation - Validates file-based output with filtering rules and log rotation
• TC5: Multi-Namespace and Concurrency - Tests concurrent audit logging across multiple namespaces
• TC6: Negative Scenarios - Tests error handling including profile deletion while pods are running
• TC7: API Correlation and Webhooks - Validates webhook injection (SPO_EXEC_REQUEST_UID) and API correlation
• TC8: Uninstall and Cleanup - Tests cleanup behavior when pods and profiles are deleted
• TC9: CRI-O Configuration - CRI-O specific scenarios for seccomp profile handling

Additionally, this PR includes documentation for 2 manual test scenarios (TC10: OCP Upgrade and TC11: SPO Upgrade/Downgrade) that require actual cluster/operator upgrades.

Minor cleanup: Fixed kubectl debug commands by removing unnecessary -i/-it flags from non-interactive debug sessions.

Which issue(s) this PR fixes:

None

Does this PR have test?

This PR adds comprehensive end-to-end test automation for the audit logging feature (JSON enricher) in the Security Profiles Operator. It adds 7 new automated test cases (TC3-TC9) covering various
scenarios:

• TC3: Privileged Pods - Tests audit logging for privileged containers with root user execution
• TC4: File Output, Filtering and Rotation - Validates file-based output with filtering rules and log rotation
• TC5: Multi-Namespace and Concurrency - Tests concurrent audit logging across multiple namespaces
• TC6: Negative Scenarios - Tests error handling including profile deletion while pods are running
• TC7: API Correlation and Webhooks - Validates webhook injection (SPO_EXEC_REQUEST_UID) and API correlation
• TC8: Uninstall and Cleanup - Tests cleanup behavior when pods and profiles are deleted
• TC9: CRI-O Configuration - CRI-O specific scenarios for seccomp profile handling

Additionally, this PR includes documentation for 2 manual test scenarios (TC10: OCP Upgrade and TC11: SPO Upgrade/Downgrade) that require actual cluster/operator upgrades.

Minor cleanup: Fixed kubectl debug commands by removing unnecessary -i/-it flags from non-interactive debug sessions.

Special notes for your reviewer:

• All tests follow existing patterns and use jsonEnricherOnlyTestCase() helper to ensure they run only when JSON enricher is enabled
• TC9 includes a runtime check and will skip if not running on CRI-O
• TC10 and TC11 are documented as manual test cases (in code comments) since they require actual cluster upgrades
• Tests include proper cleanup using defer statements
• Total additions: ~700 lines across 2 files (test/e2e_test.go and test/tc_json_enricher_test.go)

Does this PR introduce a user-facing change?

None

[k8s-ci-robot]

Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: BhargaviGudi
Once this PR has been reviewed and has the lgtm label, please assign saschagrunert for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @BhargaviGudi. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 24.15%. Comparing base (11d77f4) to head (a724f9f).
⚠️ Report is 1045 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #3050       +/-   ##
===========================================
- Coverage   45.50%   24.15%   -21.35%     
===========================================
  Files          79      125       +46     
  Lines        7782    17770     +9988     
===========================================
+ Hits         3541     4293      +752     
- Misses       4099    13193     +9094     
- Partials      142      284      +142     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.