[rootless] question: plan for supporting cgroups?

[AkihiroSuda]

Rootless mode could support cgroups when pam_cgfs.so is available (
opencontainers/runc#1839 cc @cyphar), but it is not available on Fedora (AFAIK)

Is there plan for supporting pam_cgfs.so or any equivalent of that?

(This question is not specific to podman, and I'm not sure this repo is the right place to ask this question :p)

[rhatdan]

Is there a way to handle this via communications with systemd? It would seem that systemd probably provides a mechanism for user apps to manipulate the cgroups available to the user?

[brauner]

systemd won't do unprivileged cgroup delegation on v1 hierarchies since there's no way to do it safely so it needs to be up to the administrator to switch this on. So on v1 hierarchies now way to get this going without pam_cgfs.so that's why we @lxc wrote it in the first place. The trick is to limit this to cgroups you really really care about for your runtime, that's why the pam module takes arguments which the administrators needs to explicitly set.
Now, the story is different for cgroup v2. You can talk to systemd via dbus if you feel like linking against a bunch of xml. Or you request the delegated property in a service file or - if you have a daemon - the daemon requests the delegation and creates two parallel cgroups on the same level of the hierarchy and moves itself into one and the container into the other. In any case this requires that the runtime never escapes to the root cgroup.

[cyphar]

@brauner Does systemd remount /sys/fs/cgroup with nsdelegate now? Or is this something that systemd has yet to come up with a proper setup for -- since it would technically allow cgroupv2 delegation without systemd integration by just using a cgroup namespace.

[rhatdan]

From devel list on fedora, this is @poettering response.

I am not sure what pam_cgfs.so precisely does, but do note that on
systemd systems (which includes Fedora) systemd is the owner of the
cgroup tree, and the only means by which other components may manage
their own subtrees is through cgroup delegation (which includes
delegation to less privileged users), which you can request from
systemd.

You can request cgroup delegation from the system service manager, for which
you need to be privileged (but you can request it on behalf of an
unprivileged user).

You can also request cgroup delegation from your private user service
manager instance, for which you do not need to be privileged.

The APIs for requesting cgroup delegation from the system service
manager or your user service manager is the same, the only difference
is whether you do so through the system or user dbus bus.

Note that on cgroupsv1 delegation of *controllers* (i.e. "cpu",
"cpuset", "memory", "blkio", …) to unprivileged processes is not safe
(this is a kernel limitation) and systemd won't do it hence. On
cgroupsv2 it is safe however, and hence you will get "memory" and
"tasks" delegated by default (though not "cpu" by default as the
runtime impact of that is still too high).

Do note however, that Docker is blocking us from switching Fedora over
to cgroupsv2 though, as there is still no working support for
cgroupsv2 in Docker, nor support for requested cgroup tree delegation
from systemd. It's a shame that Docker is hindering us from making the
switch, but it is how it is. Docker currently bypasses systemd
entirely when it comes to cgroupsv2 and considers itself to be owner
of the cgroup tree which is a mess on cgroupsv1 (though you have a
chance of getting away with it) and doesn't work at all on cgroupsv2.

Or in other words: if you are looking for a way to get your own
per-user delegated cgroup subtree, simply ask systemd for it by
setting Delegate=yes in your service unit file, or by asking for
a scope unit to be registered, also with Delegate=yes set. Nothing
else is supported.

Lennart

[rhatdan]

Of course if Podman is successful we could add better support for V2 CGroups to allow distributions to have the option.

[cyphar]

I thought you guys used the systemd cgroup driver on Fedora/RHEL -- that does exactly what Lennart is referring to as "hindering [him]". In addition, it's untrue that Docker (or runc) are entirely responsible for blocking cgroupv2 adoption -- a lack of the freezer cgroup (and a usable devices cgroup that doesn't depend entirely on eBPF) is blocking a wholesale switch to cgroupv2.

Most of the points Lennart made are just longer versions of what @brauner said earlier as well (though his disdain of pam_cgfs.so even though he doesn't know what it does ignores that @brauner explicitly said that you need to be careful when using it precisely for that reason -- many cgroupv1 controllers are safe for unprivileged use).

But I guess that answers my question on whether systemd intends to support nsdelegate -- it appears the answer is "no" given that he is discussing asking systemd for permission to delegate cgroupv2 cgroups even though the kernel has an explicit feature to allow this (nsdelegate and CLONE_NEWCGROUP).

[jerboaa]

How do people feel about mentioning this root-less issue in the man page or provide some warning when --memory is known to NOT work? Currently I see this in podman-run's man page:

   -m, --memory=""

   Memory limit (format: <number>[<unit>], where unit = b, k, m or g)

   Allows you to constrain the memory available to a container. If the host supports swap memory, then the -m memory setting can be larger than physical RAM. If a limit of 0 is specified (not using
   -m), the container's memory is not limited. The actual limit may be rounded up to a multiple of the operating system's page size (the value would be very large, that's millions of trillions).

It doesn't say that one needs to run podman as root in order for it to be effective. Neither mentions the command itself any limitation :-( Something like this would have been helpful:

 $ whoami
 someuser
 $ podman run [...] --memory=10m fedora:28 /bin/bash
 Warning: --memory as unprivileged user has no effect.
 bash-4.4#

[brauner]

Why not simply fail if a user requests memory constraints or any cgroup constraints that the user requested but that runC can't fulfill. Seems printing a simple warning can be easily overlooked.

[mheon]

I agree that failing it probably the appropriate course of action.

On the manpages - they really need a thorough overhaul to show what can and cannot be done with rootless.

[cyphar]

@brauner runc should already do that (we have tests to make sure and everything 😉), I have the feeling that podman is removing cgroup entries or something fishy?

[rhatdan]

@giuseppe PTAL

[rhatdan]

@brauner Please open a PR for the updated man page. --memory should definitely error out.