Skip to content

CAPD tries to request device /dev/fuse:: on rootless podman which leads to invalid device mode error #12485

@taladar

Description

@taladar

What steps did you take and what happened?

Run rootless podman (ideally on loglevel debug)
Start kind management cluster using KIND_EXPERIMENTAL_PROVIDER=podman and kind config (replace commands as the shell would with current user uid and name)
Install cluster-api with infrastructure provider docker

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraMounts:
      - hostPath: /run/user/$(id -u)/podman/podman.sock # Podman socket
        containerPath: /var/run/docker.sock         # Path CAPD expects
      - hostPath: /home/$(whoami)/.local/share/containers/storage # Rootless Podman storage
        containerPath: /var/lib/docker              # Path CAPD expects Docker's data

Try to create a cluster with clusterctl generate output (flavor development, infrastructure docker), instead we get an API error about invalid device mode, when looking at the debug logs of podman we can see

podman[1519286]: time="2025-07-14T12:51:02+02:00" level=debug msg="Non-CDI device /dev/fuse::; assuming standard device"
podman[1519286]: time="2025-07-14T12:51:02+02:00" level=info msg="Request Failed(Internal Server Error): container create: invalid device mode: "

This suggests that CAPD tries to send the /dev/fuse device with two trailing colons and indeed calling podman with a device like that manually leads to the same error

podman run -it --rm --device /dev/fuse:: debian:bookworm /bin/bash
Error: invalid device mode:

Looking at the cluster-api source code there is a section mentionin /dev/fuse

  // enable /dev/fuse explicitly for fuse-overlayfs
  // (Rootless Docker does not automatically mount /dev/fuse with --privileged)
  if d.mountFuse(info) {
    hostConfig.Devices = append(hostConfig.Devices, dockercontainer.DeviceMapping{PathOnHost: "/dev/fuse"})
  }

I am not familiar enough with the docker API or Golang to know if this is the problem but presumably this refers to https://pkg.go.dev/github.com/docker/docker/api/types/container#DeviceMapping which has three fields and the last one is a permission field which would fit the error about an invalid mode.

Docker also seems to dislike this format at least on the CLI

docker run --device /dev/fuse:: -it --rm debian:bookworm /bin/bash
docker: bad mode specified:

What did you expect to happen?

Containers should be started until the cluster is up or in other words, CAPD should send a valid device field in the request to create the containers.

Cluster API version

1.10.1

Kubernetes version

1.32.2 (for the kind cluster)

Anything else you would like to add?

No response

Label(s) to be applied

/kind bug
One or more /area label. See https://github.com/kubernetes-sigs/cluster-api/labels?q=area for the list of labels.

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedDenotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.kind/bugCategorizes issue or PR as related to a bug.priority/backlogHigher priority than priority/awaiting-more-evidence.triage/acceptedIndicates an issue or PR is ready to be actively worked on.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions