docs: document kubelet cgroup driver detection from the runtime #42160

marquiz · 2023-07-24T07:49:35Z

Documentation for KEP-4033.

Document the KubeletCgroupDriverFromCRI feature gate. Also, add notes of this feature in parts of the documentation that describe cgroup driver configuration.

Refs:

KEP-4033 (issue Discover cgroup driver from CRI enhancements#4033)
k/k PRs:
- kubelet: get cgroup driver config from CRI kubernetes#118770
- kubelet: update documentation of the cgroupDriver setting kubernetes#119441

NOTE: no container runtime supports this new feature, yet, so we cannot refer to specific versions of runtimes (yet)

NOTE: replaces/re-submits #42091 which was unwittingly submitted against the main branch

Document the KubeletCgroupDriverFromCRI feature gate. Also, add notes of this feature in parts of the documentation that describe cgroup driver configuration.

netlify · 2023-07-24T07:49:40Z

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Name	Link
🔨 Latest commit	`fa73830`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/64ccdbffbad5fd00083ee14f

sftim · 2023-07-24T10:30:17Z

/sig node

sftim

Thanks

I've added feedback to align this with our style guide

content/en/docs/reference/command-line-tools-reference/feature-gates.md

content/en/docs/setup/production-environment/container-runtimes.md

content/en/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver.md

Co-authored-by: Tim Bannister <[email protected]>

marquiz · 2023-07-24T19:14:39Z

I've added feedback to align this with our style guide

Thanks @sftim for the review. I incorporated the changes you suggested as a single commit.

sftim

Looks good for docs.

Can we get a tech review from SIG Node?

marquiz · 2023-07-25T04:52:44Z

Can we get a tech review from SIG Node?

/cc @mikebrow @haircommander

marquiz · 2023-07-25T05:00:17Z

/cc mrunalp

mikebrow

see comments, cheers

mikebrow · 2023-07-25T22:43:31Z

content/en/docs/setup/production-environment/container-runtimes.md

 the `cgroupDriver` field under `KubeletConfiguration`, kubeadm defaults it to `systemd`.
 {{< /note >}}

+For Kubernetes v1.28 and later, with the `KubeletCgroupDriverFromCRI`


bit of a tldr issue I believe.. giving the instructions above for kubelet config driven and default if not configured.. then ending with 1.28 alpha feature gate info.

Suggest mentioning the feature gate above in direct association with the kubelet config steps / options. Something like unless the alpha level feature gate is set to automatically retrieve the appropriate cgroup driver from the runtime, additional configuration may be required...

I'm a bit cautious on this ("advertising" too much) as this is an alpha feature and there are no released container runtime supporting this, yet. When we get to beta, sure. WDYT @sftim

good point...

We should document how Kubernetes behaves, even if the compatible runtimes don't exist on release day.

mikebrow · 2023-07-25T22:52:36Z

content/en/docs/setup/production-environment/container-runtimes.md

 When using kubeadm, manually configure the
 [cgroup driver for kubelet](/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/#configuring-the-kubelet-cgroup-driver).

+Starting with v1.28 and later, you can enable automatic detection of the


Suggested change

Starting with v1.28 and later, you can enable automatic detection of the

Starting with Kubernetes v1.28 and later, you can enable automatic detection of the

Aside, but also a bit relevant:
I'm slightly wary of saying “and later” about alpha features that we can't prove will graduate to stable.

Fair point, have to agree on that. I now changed all of these to In Kubernetes {{< skew currentVersion >}}...

ok @sftim @mikebrow ?

given the concern over skew "and later" comments perhaps.. for Kubernetes v1.2.8, you can..

mikebrow · 2023-07-25T22:53:09Z

content/en/docs/setup/production-environment/container-runtimes.md

 cgroup driver configuration of the kubelet (usually done via kubeadm) and CRI-O
 in sync.

+Starting with v1.28 and later, you can enable automatic detection of the


Suggested change

Starting with v1.28 and later, you can enable automatic detection of the

Starting with Kubernetes v1.28 and later, you can enable automatic detection of the

Fixed with In Kubernetes {{< skew currentVersion >}}

mikebrow · 2023-07-25T22:53:58Z

content/en/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver.md

 In v1.22 and later, if the user does not set the `cgroupDriver` field under `KubeletConfiguration`,
 kubeadm defaults it to `systemd`.
+
+Starting with v1.28 and later, you can enable automatic detection of the


see tldr comment

sftim · 2023-07-25T22:57:21Z

Wording looks fine for alpha, what this really needs now is tech signoff.

mikebrow · 2023-07-25T23:51:00Z

Wording looks fine for alpha, what this really needs now is tech signoff.

to that end for containerd .. I'm reviewing @marquiz impl in containerd containerd/containerd#8722, would like to see that merged before providing my signoff. Cheers!

tengqm · 2023-07-26T15:08:09Z

content/en/docs/setup/production-environment/container-runtimes.md

 the `cgroupDriver` field under `KubeletConfiguration`, kubeadm defaults it to `systemd`.
 {{< /note >}}

+In Kubernetes {{< skew currentVersion >}}, with the `KubeletCgroupDriverFromCRI`


Suppose we are in v1.29 now, the above line will be rendered into something like
"In Kubernetes 1.29, with the KubeletCgroupDriverFromCRI".
Is this statement still appropriate?
Suppose we are in v1.34 now and the feature gate has been GA'ed and removed,
the above line will be rendered into something like
"In Kubernetes 1.34, with the KubeletCgroupDriverFromCRI".
Is this statement still appropriate?

Clearly stating that this feature is in Alpha starting 1.28 is good enough.
Why do we use the skew shortcode the make this release number a moving target?

I'm indifferent on this, can change it to v1.29. This (In Kubernetes {{< skew currentVersion >}}) was something that @sftim suggested in one of his earlier comments so I took it from there.

Suppose we are in v1.34 now and the feature gate has been GA'ed and removed,

When we change Kubernetes, we should remove the docs. If we don't trust ourselves to do that, I would focus on fixing that lack of trust.

If we could trust the release docs process, I think this change would be fine.

As a contributor I'm inclined to agree with @sftim about the docs process here. Especially with a KEP involved, the process should ensure that the documentation is changed when the feature is moved to beta and later GA (or removed before graduating further). WDYT @tengqm?

Per #42160 (comment), let's follow the path of least risk to the release, at least for now.

K, I changed this now to In Kubernetes v1.28, ... (not to Starting with v1.28 ... as per #42160 (comment))

PTAL @sftim @tengqm

tengqm · 2023-07-26T15:09:45Z

content/en/docs/setup/production-environment/container-runtimes.md

 When using kubeadm, manually configure the
 [cgroup driver for kubelet](/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver/#configuring-the-kubelet-cgroup-driver).

+In Kubernetes {{< skew currentVersion >}}, you can enable automatic detection of the


Same as above.

In addition to the above confusion, using this skew shortcode makes it difficult
to remove contents that are no longer relevant in the future.

tengqm · 2023-07-26T15:09:57Z

content/en/docs/setup/production-environment/container-runtimes.md

 cgroup driver configuration of the kubelet (usually done via kubeadm) and CRI-O
 in sync.

+In Kubernetes {{< skew currentVersion >}}, you can enable automatic detection of the


tengqm · 2023-07-26T15:10:06Z

content/en/docs/tasks/administer-cluster/kubeadm/configure-cgroup-driver.md

 In v1.22 and later, if the user does not set the `cgroupDriver` field under `KubeletConfiguration`,
 kubeadm defaults it to `systemd`.
+
+In Kubernetes {{< skew currentVersion >}}, you can enable automatic detection of the


marquiz · 2023-08-04T09:17:37Z

ping @mikebrow the containerd implementation (containerd/containerd#8722) is now merged

ping @sftim what's your take on using {{< skew currentVersion >}} (#42160 (comment))?

sftim · 2023-08-04T09:35:58Z

ping @sftim what's your take on using {{< skew currentVersion >}} (#42160 (comment))?

I very much like like timeless documentation, but I also like having the docs ready for release. I'm reluctantly OK with not using that shortcode if it unblocks approval. After the release, I could propose putting it back.

sftim

/lgtm
/approve

k8s-ci-robot · 2023-08-05T13:29:46Z

LGTM label has been added.

Details

Git tree hash: 81a58ecd287120b77b79111f9812be9dff4008cd

k8s-ci-robot · 2023-08-05T13:29:56Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: sftim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [sftim]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…autoconfig-dev-1.28 docs: document kubelet cgroup driver detection from the runtime

docs: document kubelet cgroup driver detection from the runtime

c50c151

Document the KubeletCgroupDriverFromCRI feature gate. Also, add notes of this feature in parts of the documentation that describe cgroup driver configuration.

k8s-ci-robot added this to the 1.28 milestone Jul 24, 2023

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 24, 2023

k8s-ci-robot requested a review from mengjiao-liu July 24, 2023 07:49

k8s-ci-robot added the language/en Issues or PRs related to English language label Jul 24, 2023

k8s-ci-robot requested a review from vincepri July 24, 2023 07:49

k8s-ci-robot added sig/docs Categorizes an issue or PR as relevant to SIG Docs. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 24, 2023

This was referenced Jul 24, 2023

docs: document kubelet cgroup driver detection from the runtime #42091

Closed

Discover cgroup driver from CRI kubernetes/enhancements#4033

Closed

k8s-ci-robot added the sig/node Categorizes an issue or PR as relevant to SIG Node. label Jul 24, 2023

sftim reviewed Jul 24, 2023

View reviewed changes

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jul 24, 2023

marquiz force-pushed the devel/cgroup-driver-autoconfig-dev-1.28 branch from f32795c to b177d37 Compare July 24, 2023 19:10

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 24, 2023

docs: incorporate review comments from sftim

7e208b0

Co-authored-by: Tim Bannister <[email protected]>

marquiz force-pushed the devel/cgroup-driver-autoconfig-dev-1.28 branch from b177d37 to 7e208b0 Compare July 24, 2023 19:12

sftim reviewed Jul 24, 2023

View reviewed changes

k8s-ci-robot requested review from haircommander and mikebrow July 25, 2023 04:52

k8s-ci-robot requested a review from mrunalp July 25, 2023 05:00

mikebrow reviewed Jul 25, 2023

View reviewed changes

docs: change wording of k8s versioning on KubeletCgroupDriverFromCRI

0d2962b

tengqm reviewed Jul 26, 2023

View reviewed changes

marquiz mentioned this pull request Aug 1, 2023

Implement RuntimeConfig CRI call containerd/containerd#8722

Merged

docs: another change of wording of k8s versioning

fa73830

sftim reviewed Aug 5, 2023

View reviewed changes

k8s-ci-robot assigned sftim Aug 5, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 5, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 5, 2023

k8s-ci-robot merged commit 8478392 into kubernetes:dev-1.28 Aug 5, 2023

marquiz deleted the devel/cgroup-driver-autoconfig-dev-1.28 branch August 7, 2023 05:22

Rishit-dagli pushed a commit to Rishit-dagli/website that referenced this pull request Aug 12, 2023

Merge pull request kubernetes#42160 from marquiz/devel/cgroup-driver-…

4e4d103

…autoconfig-dev-1.28 docs: document kubelet cgroup driver detection from the runtime

	Starting with v1.28 and later, you can enable automatic detection of the
	Starting with Kubernetes v1.28 and later, you can enable automatic detection of the

docs: document kubelet cgroup driver detection from the runtime #42160

docs: document kubelet cgroup driver detection from the runtime #42160

Uh oh!

Conversation

marquiz commented Jul 24, 2023

Uh oh!

netlify bot commented Jul 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Uh oh!

sftim commented Jul 24, 2023

Uh oh!

sftim left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

marquiz commented Jul 24, 2023

Uh oh!

sftim left a comment

Choose a reason for hiding this comment

Uh oh!

marquiz commented Jul 25, 2023

Uh oh!

marquiz commented Jul 25, 2023

Uh oh!

mikebrow left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sftim commented Jul 25, 2023

Uh oh!

mikebrow commented Jul 25, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

netlify bot commented Jul 24, 2023 •

edited

Loading