MLE-25548: Support update and hot reload for Fluent-Bit #116
Conversation
pengzhouml
requested review from
Copilot,
rwinieski,
sumanthravipati and
vitalykorolev
There was a problem hiding this comment.
Pull request overview
Adds Fluent Bit hot reload and update support plus comprehensive e2e tests for log collection scenarios (disabled, partial, custom resources, custom filters). Key changes include: conditional ConfigMap update logic with patch comparison, enabling Fluent Bit HTTP server/metrics/hot_reload, and adjusting log volume mount to a Logs subPath.
- Added e2e tests covering multiple LogCollection configurations (enable/disable, selective files, resources, filters).
- Implemented patch-based reconciliation to update existing ConfigMaps when data changes and added hot reload/server settings to Fluent Bit configuration.
- Modified StatefulSet Fluent Bit volume mount to use a subPath "Logs".
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
| test/e2e/6_log_collection_test.go | New e2e tests for various LogCollection configurations and behaviors. |
| pkg/k8sutil/statefulset.go | Added subPath "Logs" to Fluent Bit volume mount. |
| pkg/k8sutil/configmap.go | Added patch-based update logic and hot reload / HTTP server settings to Fluent Bit ConfigMap generation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pkg/k8sutil/configmap.go
Outdated
| if !patchDiff.IsEmpty() { | ||
| logger.Info("MarkLogic ConfigMap data has changed, updating it") | ||
| configmap.Data = desiredConfigMap.Data |
There was a problem hiding this comment.
[nitpick] Patch-based update logic here is duplicated in ReconcileFluentBitConfigMap; consider extracting a shared helper to reduce repetition and ensure consistent handling of annotations and future changes.
pkg/k8sutil/configmap.go
Outdated
| if !patchDiff.IsEmpty() { | ||
| logger.Info("Fluent Bit ConfigMap data has changed, updating it") | ||
| configmap.Data = desiredConfigMap.Data |
There was a problem hiding this comment.
[nitpick] Same patch/update pattern as earlier; refactoring into a generic ConfigMap reconcile helper would simplify maintenance and reduce risk of divergence.
add sleep Wait for MarkLogic services to fully initialize
No description provided.