Skip to content

Conversation

@thechristschn
Copy link
Contributor

With Kernel 6.12 the cgroup behavior changed which causes cgroup detection in JVM to break.

The previous fix in #3263 wasn't sufficient, we also need to enable CONFIG_MEMCG_V1.

Fixes flatcar/Flatcar#1884

How to use

Build and deploy flatcar with the kernel parameter and execute tests below.

Testing done

To validate that the first attempt to fix the problem in #3263 is necessary as well:

docker run -it -m 1g fedora:41 bash -c "dnf install -y java-headless && java -XX:MaxRAMPercentage=50 -XshowSettings -Xlog:os+container=debug --version"
[...]
[0.000s][debug][os,container] Detected optional pids controller entry in /proc/cgroups
[0.000s][debug][os,container] controller cpuset is not enabled

[0.000s][debug][os,container] controller memory is not enabled

[0.001s][debug][os,container] One or more required controllers disabled at kernel level.
VM settings:
    Max. Heap Size (Estimated): 980.00M
    Using VM: OpenJDK 64-Bit Server VM

openjdk 21.0.8 2025-07-15
OpenJDK Runtime Environment (Red_Hat-21.0.8.0.9-1) (build 21.0.8+9)
OpenJDK 64-Bit Server VM (Red_Hat-21.0.8.0.9-1) (build 21.0.8+9, mixed mode, sharing)

Java complains, that cpuset and memory controller are missing and falls back to available system memory, which was fixed in #3263.

Behavior with current alpha/beta release on system with 2GB memory:

docker run -it -m 1g fedora:41 bash -c "dnf install -y java-headless && java -XX:MaxRAMPercentage=50 -XshowSettings -Xlog:os+container=debug --version"
[...]
[0.000s][debug][os,container] Detected optional cpuset controller entry in /proc/cgroups=debug --version
[0.000s][debug][os,container] Detected optional pids controller entry in /proc/cgroups
[0.000s][debug][os,container] controller memory is not enabled

[0.000s][debug][os,container] One or more required controllers disabled at kernel level.
VM settings:
    Max. Heap Size (Estimated): 980.00M
    Using VM: OpenJDK 64-Bit Server VM

openjdk 21.0.8 2025-07-15
OpenJDK Runtime Environment (Red_Hat-21.0.8.0.9-1) (build 21.0.8+9)
OpenJDK 64-Bit Server VM (Red_Hat-21.0.8.0.9-1) (build 21.0.8+9, mixed mode, sharing)

50% of the host memory is used for heap instead of 50% of the container memory limit. The debug logs complain about missing memory controller.

Behavior with this change:

docker run -it -m 1g fedora:41 bash -c "dnf install -y java-headless && java -XX:MaxRAMPercentage=50 -XshowSettings -Xlog:os+container=debug --version"
[...]
[0.000s][debug][os,container] Detected optional cpuset controller entry in /proc/cgroups=debug --version
[0.000s][debug][os,container] Detected optional pids controller entry in /proc/cgroups
[0.000s][debug][os,container] Detected cgroups v2 unified hierarchy
VM settings:
    Max. Heap Size (Estimated): 494.94M
    Using VM: OpenJDK 64-Bit Server VM

openjdk 21.0.8 2025-07-15
OpenJDK Runtime Environment (Red_Hat-21.0.8.0.9-1) (build 21.0.8+9)
OpenJDK 64-Bit Server VM (Red_Hat-21.0.8.0.9-1) (build 21.0.8+9, mixed mode, sharing)

This time 50% of the container memory limit is used for heap, which is what we want. Also the debug logs show, that cgroupv2 is detected.

  • Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
  • Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

@jepio
Copy link
Member

jepio commented Sep 19, 2025

@thechristschn you're running nodes with cgroups v2 but the jre version you're using checks that the host kernel still supports v1 controllers? Even if these v1 controllers are not active and managed by cgroupsv2?

@github-actions
Copy link

github-actions bot commented Sep 19, 2025

@sfudeus
Copy link
Contributor

sfudeus commented Sep 19, 2025

@jepio Unfortunately that's a deficiency of OpenJDK, see https://bugs.openjdk.org/browse/JDK-8346874 and https://bugs.openjdk.org/browse/JDK-8347129

@chewi
Copy link
Contributor

chewi commented Sep 25, 2025

Someone please rebase this.

@thechristschn
Copy link
Contributor Author

Someone please rebase this.

@chewi Done

@chewi
Copy link
Contributor

chewi commented Sep 30, 2025

Sorry, I forgot that the CI still didn't work properly for forks. I'll handle it.

Signed-off-by: Christian Baumann <[email protected]>
Signed-off-by: James Le Cuirot <[email protected]>
Signed-off-by: Christian Baumann <[email protected]>
Signed-off-by: James Le Cuirot <[email protected]>
Copy link
Contributor

@chewi chewi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I ran CI in a local branch here and it basically passed. Merging now. Sorry for the wait!

@chewi chewi merged commit 11609fc into flatcar:main Sep 30, 2025
9 checks passed
@github-project-automation github-project-automation bot moved this from ✅ Testing / in Review to Implemented in Flatcar tactical, release planning, and roadmap Sep 30, 2025
@tormath1
Copy link
Contributor

tormath1 commented Nov 7, 2025

@chewi can you verify if this has been backported to Beta at one point? I don't see it in the release notes (only on current Alpha)

@chewi
Copy link
Contributor

chewi commented Nov 7, 2025

It wasn't. I've applied it to 4459 now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

Cgroup changes in Kernel 6.12 affects Java applications

5 participants