chore: creating volumne for otel path in container

[pankaj101A]

Getting

Error: failed to create logger: open sink "/var/log/otelcol.log": open /var/log/otelcol.log: permission denied

Reason
container has to be mounted to a volume with given path, it is not done by default. for each path has to provided in operator's handler.go. On local test worked because test config present in repo mounts that explicitly.

Tested on ec2 by building custom operator and tailing sidecar

[ec2-user@ip-172-31-3-66 tailing-sidecar]$ kubectl describe pod daemonset-with-annotations-b8lxr -n my-namespace
Name: daemonset-with-annotations-b8lxr
Namespace: my-namespace
Priority: 0
Service Account: default
Node: kind-control-plane/172.18.0.2
Start Time: Wed, 07 Jan 2026 06:06:54 +0000
Labels: app=daemonset-with-annotations
controller-revision-hash=7b6c4bb48d
pod-template-generation=1
Annotations: tailing-sidecar: varlog:/var/log/example0.log;varlog:/var/log/example1.log
Status: Running
IP: 10.244.0.14
IPs:
IP: 10.244.0.14
Controlled By: DaemonSet/daemonset-with-annotations
Containers:
count:
Container ID: containerd://14ccc2353ab011c28061f520ed60216395e2a13a636da3720218029a77634e17
Image: busybox
Image ID: docker.io/library/busybox@sha256:2383baad1860bbe9d8a7a843775048fd07d8afe292b94bd876df64a69aae7cb1
Port:
Host Port:
Args:
/bin/sh
-c
i=0; while true; do
echo "example0: $i $(date)" >> /var/log/example0.log;
echo "example1: $i $(date)" >> /var/log/example1.log;
echo "example2: $i $(date)" >> /varconfig/log/example2.log;
i=$((i+1));
sleep 1;
done

State:          Running
  Started:      Wed, 07 Jan 2026 06:06:55 +0000
Ready:          True
Restart Count:  0
Environment:    <none>
Mounts:
  /var/log from varlog (rw)
  /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-l2f84 (ro)
  /varconfig/log from varlogconfig (rw)

tailing-sidecar-0:
Container ID: containerd://f60907e4c6561e9607befa67b365f65b4a4a208f51ac8b955b5bde910af88b23
Image: sidecar:test
Image ID: docker.io/library/import-2026-01-07@sha256:ecac9e239d165259db3a6bcbeb94f3fa8905651969e7b6ddbef54fc60e553ca5
Port:
Host Port:
State: Running
Started: Wed, 07 Jan 2026 06:06:55 +0000
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment:
PATH_TO_TAIL: /var/log/example0.log
TAILING_SIDECAR: true
OTEL_FILE_STORAGE_PATH: /var/lib/otc/tailing-sidecar-0
SIDECAR_OTEL_LOG_PATH: /var/log/tailing-sidecar-0
SIDECAR_CONTAINER_NAME: tailing-sidecar-0
Mounts:
/etc/otel/ from tailing-sidecar-configuration (rw)
/tailing-sidecar/var from volume-sidecar-0 (rw)
/var/lib/otc/tailing-sidecar-0 from tailing-sidecar-otel-file-storage-tailing-sidecar-0 (rw)
/var/log from varlog (rw)
/var/log/tailing-sidecar-0 from tailing-sidecar-otel-logs-tailing-sidecar-0 (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-l2f84 (ro)
tailing-sidecar-1:
Container ID: containerd://176e20c66989778396bb55942250f40e0eeb7dee787a5827d2212ffbfc2c4fe4
Image: sidecar:test
Image ID: docker.io/library/import-2026-01-07@sha256:ecac9e239d165259db3a6bcbeb94f3fa8905651969e7b6ddbef54fc60e553ca5
Port:
Host Port:
State: Running
Started: Wed, 07 Jan 2026 06:06:56 +0000
Ready: True
Restart Count: 0
Limits:
cpu: 0
memory: 0
Requests:
cpu: 0
memory: 0
Environment:
PATH_TO_TAIL: /var/log/example1.log
TAILING_SIDECAR: true
OTEL_FILE_STORAGE_PATH: /var/lib/otc/tailing-sidecar-1
SIDECAR_OTEL_LOG_PATH: /var/log/tailing-sidecar-1
SIDECAR_CONTAINER_NAME: tailing-sidecar-1
Mounts:
/etc/otel/ from tailing-sidecar-configuration (rw)
/tailing-sidecar/var from volume-sidecar-1 (rw)
/var/lib/otc/tailing-sidecar-1 from tailing-sidecar-otel-file-storage-tailing-sidecar-1 (rw)
/var/log from varlog (rw)
/var/log/tailing-sidecar-1 from tailing-sidecar-otel-logs-tailing-sidecar-1 (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-l2f84 (ro)
Conditions:
Type Status
PodReadyToStartContainers True
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
varlog:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
varlogconfig:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
kube-api-access-l2f84:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
Optional: false
DownwardAPI: true
volume-sidecar-0:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
tailing-sidecar-otel-logs-tailing-sidecar-0:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
tailing-sidecar-otel-file-storage-tailing-sidecar-0:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
volume-sidecar-1:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
tailing-sidecar-otel-logs-tailing-sidecar-1:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
tailing-sidecar-otel-file-storage-tailing-sidecar-1:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit:
tailing-sidecar-configuration:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: my-release-sidecar-config
Optional: false
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/disk-pressure:NoSchedule op=Exists
node.kubernetes.io/memory-pressure:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists
node.kubernetes.io/pid-pressure:NoSchedule op=Exists
node.kubernetes.io/unreachable:NoExecute op=Exists
node.kubernetes.io/unschedulable:NoSchedule op=Exists
Events:
Type Reason Age From Message

---

Normal Scheduled 49s default-scheduler Successfully assigned my-namespace/daemonset-with-annotations-b8lxr to kind-control-plane
Normal Pulling 50s kubelet spec.containers{count}: Pulling image "busybox"
Normal Pulled 49s kubelet spec.containers{count}: Successfully pulled image "busybox" in 896ms (896ms including waiting). Image size: 2224358 bytes.
Normal Created 49s kubelet spec.containers{count}: Created container count
Normal Started 49s kubelet spec.containers{count}: Started container count
Normal Pulled 49s kubelet spec.containers{tailing-sidecar-0}: Container image "sidecar:test" already present on machine
Normal Created 49s kubelet spec.containers{tailing-sidecar-0}: Created container tailing-sidecar-0
Normal Started 49s kubelet spec.containers{tailing-sidecar-0}: Started container tailing-sidecar-0
Normal Pulled 49s kubelet spec.containers{tailing-sidecar-1}: Container image "sidecar:test" already present on machine
Normal Created 48s kubelet spec.containers{tailing-sidecar-1}: Created container tailing-sidecar-1
Normal Started 48s kubelet spec.containers{tailing-sidecar-1}: Started container tailing-sidecar-1
[ec2-user@ip-172-31-3-66 tailing-sidecar]$ kubectl logs daemonset-with-annotations-2mql9 -c tailing-sidecar-1 -n my-namespace
error: error from server (NotFound): pods "daemonset-with-annotations-2mql9" not found in namespace "my-namespace"
[ec2-user@ip-172-31-3-66 tailing-sidecar]$ kubectl logs daemonset-with-annotations-b8lxr -c tailing-sidecar-1 -n my-namespace
example1: 0 Wed Jan 7 06:06:55 UTC 2026
example1: 1 Wed Jan 7 06:06:56 UTC 2026
example1: 2 Wed Jan 7 06:06:57 UTC 2026

[Copilot]

Pull request overview

This PR fixes a permission error in the OpenTelemetry Collector sidecar by adding proper volume mounts for OTel file storage and logs paths. Previously, the sidecar container couldn't write to /var/log/otelcol.log and /var/lib/otc because these paths weren't mounted to volumes.

Key Changes:

• Parameterized OTel log and file storage paths using environment variables
• Added volume mounts for each sidecar's OTel logs and file storage directories
• Updated default configuration mount path from /fluent-bit/etc/ to /etc/otel/

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sidecar/otelcol/config.yaml	Updated to use environment variables for OTel file storage and log paths
operator/handler/handler.go	Added volume creation and mounting logic for OTel paths per sidecar
operator/handler/testdata/*.json	Updated test fixtures to include new volumes and environment variables
helm/tailing-sidecar-operator/values.yaml	Changed default config mount path to /etc/otel/
helm/tests/values.withFluentBitCustomConfiguration.yaml	Added explicit mount path for FluentBit configuration

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jagan2221]

Looks good, Please monitor for few hours whether the older issue is occurring once you sync this in stag.
Also shouldn't we sync this to stag after QE testing of helm chart? Or we are using stag for dev testing the changes before QE testing?

[Copilot]

@pankaj101A I've opened a new pull request, #825, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

@pankaj101A I've opened a new pull request, #826, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

@pankaj101A I've opened a new pull request, #827, to work on those changes. Once the pull request is ready, I'll request review from you.

[pankaj101A]

@jagan2221 Have done Dev testing on EC2

QE is planned to test change on long by running E2E, no manual testing required According to Arpita