2026-01 kola: ext.config.selinux.file-context-policy-match test is failing consistently

[dustymabe]

The ext.config.selinux.file-context-policy-match test has been failing for a while but we haven't noticed because of a few reasons:

1. The test is marked a non-exclusive, which means it runs with other tests on the same VM.
2. Failures in the pipeline are rerun for flake detection and if they pass it is considered a flake and the run continues.

In this case the failure happens in the first run because there is a conflict of some sort with another test (I don't know which one). The later rerun succeeds because the test it run on a VM by itself (since it was the only failure in the run).

Details

[2026-01-16T07:55:24.272Z] === RUN non-exclusive-test-bucket-0/ext.config.selinux.file-context-policy-match
[2026-01-16T07:55:27.523Z] systemctl status kola-runext-65.service:
[2026-01-16T07:55:27.523Z] × kola-runext-65.service
[2026-01-16T07:55:27.523Z] Loaded: loaded (/etc/systemd/system/kola-runext-65.service; static)
[2026-01-16T07:55:27.523Z] Drop-In: /usr/lib/systemd/system/service.d
[2026-01-16T07:55:27.523Z] └─10-timeout-abort.conf
[2026-01-16T07:55:27.523Z] Active: failed (Result: exit-code) since Fri 2026-01-16 07:55:26 UTC; 543ms ago
[2026-01-16T07:55:27.523Z] Duration: 1.580s
[2026-01-16T07:55:27.523Z] Invocation: d19ef30bae654f419f5d76951771138e
[2026-01-16T07:55:27.523Z] Process: 21317 ExecStart=/usr/local/bin/kola-runext-file-context-policy-match (code=exited, status=1/FAILURE)
[2026-01-16T07:55:27.523Z] Main PID: 21317 (code=exited, status=1/FAILURE)
[2026-01-16T07:55:27.523Z] Mem peak: 55.2M
[2026-01-16T07:55:27.523Z] CPU: 1.460s
[2026-01-16T07:55:27.523Z]
[2026-01-16T07:55:27.523Z] Jan 16 07:55:24 qemu0 kola-runext-file-context-policy-match[21317]: ++ IFS=' '
[2026-01-16T07:55:27.523Z] Jan 16 07:55:24 qemu0 kola-runext-file-context-policy-match[21317]: ++ read -r -a cmdline
[2026-01-16T07:55:27.523Z] Jan 16 07:55:24 qemu0 kola-runext-file-context-policy-match[21319]: ++ find /sysroot -context 'unlabeled_t' -not -regex '.*.0/var/mnt' -print0
[2026-01-16T07:55:27.523Z] Jan 16 07:55:24 qemu0 kola-runext-file-context-policy-match[21320]: ++ xargs --null '-I{}' ls -ldZ '{}'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:25 qemu0 kola-runext-file-context-policy-match[21317]: + unlabeled=
[2026-01-16T07:55:27.523Z] Jan 16 07:55:25 qemu0 kola-runext-file-context-policy-match[21317]: + '[' -n '' ']'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:25 qemu0 kola-runext-file-context-policy-match[21321]: ++ restorecon -vnr /var/ /etc/ /usr/ /boot/
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + mislabeled=$'Would relabel /var/home/core/.local/share/containers/storage/overlay-containers from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0\nWould relabel /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0\nWould relabel /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + '[' -n $'Would relabel /var/home/core/.local/share/containers/storage/overlay-containers from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0\nWould relabel /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0\nWould relabel /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0' ']'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + exceptions=(['/var/opt/cni']='1' ['/etc/iscsi/initiatorname.iscsi']='1')
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + declare -A exceptions
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21323]: ++ echo $'Would relabel /var/home/core/.local/share/containers/storage/overlay-containers from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0\nWould relabel /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0\nWould relabel /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json from system_u:object_r:data_home_t:s0 to system_u:object_r:container_ro_file_t:s0'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21324]: ++ grep 'Would relabel'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21325]: ++ cut -d ' ' -f 3
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + paths=$'/var/home/core/.local/share/containers/storage/overlay-containers\n/var/home/core/.local/share/containers/storage/overlay-containers/containers.lock\n/var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + found=
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + read -r path
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + [[ noexception == noexception ]]
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + echo 'Unexpected mislabeled file found: /var/home/core/.local/share/containers/storage/overlay-containers'
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: Unexpected mislabeled file found: /var/home/core/.local/share/containers/storage/overlay-containers
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + found=1
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + read -r path
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + [[ noexception == noexception ]]
[2026-01-16T07:55:27.523Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + echo 'Unexpected mislabeled file found: /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock'
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: Unexpected mislabeled file found: /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + found=1
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + read -r path
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + [[ noexception == noexception ]]
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + echo 'Unexpected mislabeled file found: /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json'
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: Unexpected mislabeled file found: /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + found=1
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + read -r path
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + '[' 1 == 1 ']'
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + fatal 'Some unexpected mislabeled files were found.'
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + echo 'Some unexpected mislabeled files were found.'
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: Some unexpected mislabeled files were found.
[2026-01-16T07:55:27.524Z] Jan 16 07:55:26 qemu0 kola-runext-file-context-policy-match[21317]: + exit 1

[travier]

Given that this is about (temporary?) files in container storage, I would make the test exclusive for now so that it does not run with tests that run containers:

Would relabel /var/home/core/.local/share/containers/storage/overlay-containers
  from system_u:object_r:data_home_t:s0
  to system_u:object_r:container_ro_file_t:s0
Would relabel /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock
  from system_u:object_r:data_home_t:s0
  to system_u:object_r:container_ro_file_t:s0
Would relabel /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json
  from system_u:object_r:data_home_t:s0
  to system_u:object_r:container_ro_file_t:s0

It might be make sense to report those errors upstream to containers-selinux probably.

[travier]

On my system:

$ ls -alhZ .local/share/containers
total 0
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0  36 Nov 28  2024 .
drwxr-xr-x. 1 tim tim system_u:object_r:data_home_t:s0     808 Jan 29 16:11 ..
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0  50 Jan 29 11:50 cache
drwxr-xr-x. 1 tim tim unconfined_u:object_r:data_home_t:s0  14 Nov 28  2024 podman
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0 262 Jan 29 12:33 storage
$ ls -alhZ .local/share/containers/storage
total 236K
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0          262 Jan 29 12:33 .
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0           36 Nov 28  2024 ..
-rw-r--r--. 1 tim tim unconfined_u:object_r:data_home_t:s0         228K Jan 29 12:33 db.sql
-rw-r--r--. 1 tim tim unconfined_u:object_r:data_home_t:s0            8 Nov 28  2024 defaultNetworkBackend
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0            0 Nov 28  2024 libpod
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0           26 Nov 28  2024 networks
drwxr-xr-x. 1 tim tim unconfined_u:object_r:container_ro_file_t:s0  45K Jan 29 11:57 overlay
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0          364 Jan 29 11:57 overlay-containers
drwx------. 1 tim tim unconfined_u:object_r:container_ro_file_t:s0  940 Jan 29 11:50 overlay-images
drwx------. 1 tim tim unconfined_u:object_r:container_ro_file_t:s0  54K Jan 29 11:57 overlay-layers
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0           24 Nov 28  2024 secrets
-rw-r--r--. 1 tim tim unconfined_u:object_r:data_home_t:s0           64 Jan 29 10:36 storage.lock
-rw-r--r--. 1 tim tim unconfined_u:object_r:data_home_t:s0            0 Nov 28  2024 userns.lock
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0            0 Jun 24  2025 volumes
$ ls -alhZ .local/share/containers/storage/overlay-containers
total 12K
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0  364 Jan 29 11:57 .
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0  262 Jan 29 12:33 ..
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0   16 Jan 26 11:17 2af5faacfea544a8e0b7def2d884beb85dd1118562cacae8f35bc53ba190c4f7
-rw-------. 1 tim tim unconfined_u:object_r:data_home_t:s0 1,8K Jan 26 11:18 containers.json
-rw-r--r--. 1 tim tim unconfined_u:object_r:data_home_t:s0   64 Jan 29 11:57 containers.lock
drwx------. 1 tim tim unconfined_u:object_r:data_home_t:s0   16 Jan 26 11:18 ea7ab04297da7b0ee80ced70a1529324a5192c254beca6bac95024274225ceb8
-rw-------. 1 tim tim unconfined_u:object_r:data_home_t:s0    2 Jan 29 11:57 volatile-containers.json

[HuijingHei]

This is added by containers/container-selinux@9c3cc4c

[HuijingHei]

I am confused that why the /var/home/core/.local/share/containers/storage/overlay-containers is not created with container_ro_file_t.

[coreos-assembler]$ cosa run --qemu-image fedora-coreos-43.20260309.20.1-qemu.x86_64.qcow2 -m 4096

[core@cosa-devsh ~]$ rpm -qa | grep container-selinux
container-selinux-2.245.0-1.fc43.noarch

[core@cosa-devsh ~]$ ls -alZ /var/home/core/.local/share/containers/storage/overlay-containers
ls: cannot access '/var/home/core/.local/share/containers/storage/overlay-containers': No such file or directory
[core@cosa-devsh ~]$ podman run --rm -it fedora:43

[core@cosa-devsh ~]$ ls -alZ /var/home/core/.local/share/containers/storage/overlay-containers
total 12
drwx------. 2 core core unconfined_u:object_r:data_home_t:s0   61 Mar 10 08:21 .
drwx------. 9 core core unconfined_u:object_r:data_home_t:s0 4096 Mar 10 08:21 ..
-rw-r--r--. 1 core core unconfined_u:object_r:data_home_t:s0   64 Mar 10 08:21 containers.lock
-rw-------. 1 core core unconfined_u:object_r:data_home_t:s0    2 Mar 10 08:21 volatile-containers.json

[core@cosa-devsh ~]$ sudo restorecon -nvr /var/home/core/.local/share/containers/storage/
Would relabel /var/home/core/.local/share/containers/storage/overlay-containers from unconfined_u:object_r:data_home_t:s0 to unconfined_u:object_r:container_ro_file_t:s0
Would relabel /var/home/core/.local/share/containers/storage/overlay-containers/containers.lock from unconfined_u:object_r:data_home_t:s0 to unconfined_u:object_r:container_
ro_file_t:s0
Would relabel /var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json from unconfined_u:object_r:data_home_t:s0 to unconfined_u:object_r:c
ontainer_ro_file_t:s0

[Rolv-Apneseth]

With the above merged, I guess we just need to wait for another release. Do we make the test exclusive temporarily like @travier suggested?

[HuijingHei]

Maybe safe to add the files in exceptions (https://github.com/coreos/fedora-coreos-config/blob/testing-devel/tests/kola/selinux/file-context-policy-match#L33) for files, and revert this once we have the new release.

/var/home/core/.local/share/containers/storage/overlay-containers
/var/home/core/.local/share/containers/storage/overlay-containers/containers.lock
/var/home/core/.local/share/containers/storage/overlay-containers/volatile-containers.json

[dustymabe]

Maybe safe to add the files in exceptions (https://github.com/coreos/fedora-coreos-config/blob/testing-devel/tests/kola/selinux/file-context-policy-match#L33) for files, and revert this once we have the new release.

Maybe, but the test has been failing for a good while and it passes in the re-run (when it is run by itself) so I'd say let's just not do anything and wait for the fix to land. (that way we don't have to remember to revert anything).