|
| 1 | +# CRI API CHANGELOG |
| 2 | + |
| 3 | +* [Overview](#overview "Overview") |
| 4 | +* [Requirements](#requirements "Requirements") |
| 5 | +* [The Changes Of CRI API](#the-changes-of-cri-api "The changes of CRI API") |
| 6 | + * [UpdateContainerResources](#updatecontainerresources "UpdateContainerResources()") |
| 7 | + * [ContainerStatus](#containerstatus "ContainerStatus()") |
| 8 | + * [ImageStatus](#imagestatus "ImageStatus()") |
| 9 | + * [CreateContainer](#createcontainer "CreateContainer()") |
| 10 | + * [RemoveVolume](#removevolume "RemoveVolume()") |
| 11 | +* [Pull Request](#pull-request) |
| 12 | + |
| 13 | +## Overview |
| 14 | + |
| 15 | +Because the CRI interface of Kubernetes cannot meet the customized development of Kubelet at present, it is necessary to provide the required functions of Kubelet by extending the API of CRI. On top of CRI's existing functionality, use the field extension or add method to make CRI meet the requirements. |
| 16 | + |
| 17 | +Kubernetes Version: V1.10.0+ |
| 18 | + |
| 19 | +## Requirements |
| 20 | + |
| 21 | +1. Support the ability to update the container diskquota. |
| 22 | + - Scenario: |
| 23 | + - Limits the directory size within the container. |
| 24 | +2. Support the ability to acquire the volumes of the Image. |
| 25 | + - Scenario: |
| 26 | + - In the Upgrade process, the volumes and mountpoints of the old container need to be read. At the same time, the volumes in the new image also need to be read. If the mount point is the same, the volumes in the new image cover the volumes in the old container. |
| 27 | +3. Support to the ability to acquire the volumes of the Container. |
| 28 | + - Scenario: |
| 29 | + - In the Upgrade process, the volumes of the new container needs to remain consistent with that of the old container, so the volumes of the old container needs to be read. |
| 30 | + |
| 31 | +# The Changes Of CRI API |
| 32 | + |
| 33 | +## UpdateContainerResources |
| 34 | + |
| 35 | +### What To Solve? |
| 36 | + |
| 37 | +1. Support the ability to update the container quotadir. |
| 38 | + |
| 39 | +### Modification |
| 40 | + |
| 41 | ++ Add the DiskQuota field in LinuxContainerResources, referring to the definition of Resources in mobyfor better compatibility. The new fields are as follows: |
| 42 | + |
| 43 | +``` |
| 44 | +type LinuxContainerResources struct { |
| 45 | + ...... |
| 46 | + //***New Fields***// |
| 47 | + // DiskQuota constrains the disk. Default: none (not specified) |
| 48 | + DiskQuota map[string]string `protobuf:"bytes,100,req,name=disk_quota,json=diskQuota,proto3" json:"disk_quota,omitempty"` |
| 49 | + // Block IO weight (relative weight vs. other containers) |
| 50 | + BlkioWeight uint16 `protobuf:"bytes,101,opt,name=blkio_weight,json=blkioWeight,proto3" json:"blkio_weight,omitempty"` |
| 51 | + BlkioWeightDevice []*WeightDevice `protobuf:"bytes,102,req,name=blkio_weight_device,json=blkioWeightDevice,proto3" json:"blkio_weight_device,omitempty"` |
| 52 | + BlkioDeviceReadBps []*ThrottleDevice `protobuf:"bytes,103,req,name=blkio_device_read_bps,json=blkioDeviceReadBps,proto3" json:"blkio_device_read_bps,omitempty"` |
| 53 | + BlkioDeviceWriteBps []*ThrottleDevice `protobuf:"bytes,104,req,name=blkio_device_write_bps,json=blkioDeviceWriteBps,proto3" json:"blkio_device_write_bps,omitempty"` |
| 54 | + BlkioDeviceReadIOps []*ThrottleDevice `protobuf:"bytes,105,req,name=blkio_device_read_IOps,json=blkioDeviceReadIOps,proto3" json:"blkio_device_read_IOps,omitempty"` |
| 55 | + BlkioDeviceWriteIOps []*ThrottleDevice `protobuf:"bytes,106,req,name=blkio_device_write_IOps,json=blkioDeviceWriteIOps,proto3" json:"blkio_device_write_IOps,omitempty"` |
| 56 | + // Kernel memory limit (in bytes) |
| 57 | + KernelMemory int64 `protobuf:"bytes,107,opt,name=kernel_memory,json=kernelMemory,proto3" json:"kernel_memory,omitempty"` |
| 58 | + // Memory soft limit (in bytes) |
| 59 | + MemoryReservation int64 `protobuf:"bytes,108,opt,name=memory_reservation,json=memoryReservation,proto3" json:"memory_reservation,omitempty"` |
| 60 | + // Tuning container memory swappiness behaviour |
| 61 | + MemorySwappiness int64 `protobuf:"bytes,109,opt,name=memory_swappiness,json=memorySwappiness,proto3" json:"memory_swappiness,omitempty"` |
| 62 | + // List of ulimits to be set in the container |
| 63 | + Ulimits []*Ulimit `protobuf:"bytes,110,opt,name=ulimits,json=ulimits,proto3" json:"ulimits,omitempty"` |
| 64 | +} |
| 65 | +
|
| 66 | +// WeightDevice is a structure that holds device:weight pair |
| 67 | +type WeightDevice struct { |
| 68 | + // Path of weightdevice. |
| 69 | + Path string `protobuf:"bytes,1,opt,name=path,json=path,proto3" json:"path,omitempty"` |
| 70 | + // Weight of weightdevice. |
| 71 | + Weight uint16 `protobuf:"bytes,2,opt,name=weight,json=weight,proto3" json:"weight,omitempty"` |
| 72 | +} |
| 73 | +
|
| 74 | +// ThrottleDevice is a structure that holds device:rate_per_second pair |
| 75 | +type ThrottleDevice struct { |
| 76 | + // Path of throttledevice. |
| 77 | + Path string `protobuf:"bytes,1,opt,name=path,json=path,proto3" json:"path,omitempty"` |
| 78 | + // Rate of throttledevice. |
| 79 | + Rate uint64 `protobuf:"bytes,1,opt,name=rate,json=rate,proto3" json:"rate,omitempty"` |
| 80 | +} |
| 81 | +
|
| 82 | +// Ulimit is a human friendly version of Rlimit. |
| 83 | +type Ulimit struct { |
| 84 | + // Name of ulimit. |
| 85 | + Name string `protobuf:"bytes,1,opt,name=name,json=name,proto3" json:"name,omitempty"` |
| 86 | + // Hard of ulimit. |
| 87 | + Hard int64 `protobuf:"bytes,2,opt,name=hard,json=hard,proto3" json:"hard,omitempty"` |
| 88 | + // Soft of Ulimit. |
| 89 | + Soft int64 `protobuf:"bytes,3,opt,name=soft,json=soft,proto3" json:"soft,omitempty"` |
| 90 | +} |
| 91 | +``` |
| 92 | + |
| 93 | +The changes need to be made in the proto file are as follows: |
| 94 | + |
| 95 | +``` |
| 96 | +message LinuxContainerResources { |
| 97 | + ...... |
| 98 | + //***New Fields***// |
| 99 | + // DiskQuota constrains the disk |
| 100 | + map<string,string> disk_quota = 100; |
| 101 | + // Block IO weight (relative weight vs. other containers) |
| 102 | + uint32 blkio_weight = 101; |
| 103 | + repeated WeightDevice blkio_weight_device = 102; |
| 104 | + repeated ThrottleDevice blkio_device_read_bps = 103; |
| 105 | + repeated ThrottleDevice blkio_device_write_bps = 104; |
| 106 | + repeated ThrottleDevice blkio_device_read_IOps = 105; |
| 107 | + repeated ThrottleDevice blkio_device_write_IOps = 106; |
| 108 | + // Kernel memory limit (in bytes) |
| 109 | + int64 kernel_memory = 107; |
| 110 | + // Memory soft limit (in bytes) |
| 111 | + int64 memory_reservation = 108; |
| 112 | + // Tuning container memory swappiness behaviour |
| 113 | + Int64Value memory_swappiness = 109; |
| 114 | + // List of ulimits to be set in the container |
| 115 | + repeated Ulimit ulimits = 110; |
| 116 | +} |
| 117 | +
|
| 118 | +// WeightDevice is a structure that holds device:weight pair |
| 119 | +message WeightDevice { |
| 120 | + // Path of weightdevice. |
| 121 | + string path = 1; |
| 122 | + // Weight of weightdevice. |
| 123 | + uint32 Weight = 2; |
| 124 | +} |
| 125 | +
|
| 126 | +// ThrottleDevice is a structure that holds device:rate_per_second pair |
| 127 | +message ThrottleDevice { |
| 128 | + // Path of throttledevice. |
| 129 | + string path = 1; |
| 130 | + // Rate of throttledevice. |
| 131 | + uint64 rate = 2; |
| 132 | +} |
| 133 | +
|
| 134 | +// Ulimit is a human friendly version of Rlimit. |
| 135 | +message Ulimit { |
| 136 | + // Name of ulimit. |
| 137 | + string name = 1; |
| 138 | + // Hard limit of ulimit. |
| 139 | + int64 hard = 2; |
| 140 | + // Soft limit of Ulimit. |
| 141 | + int64 soft = 3; |
| 142 | +} |
| 143 | +``` |
| 144 | + |
| 145 | +## ContainerStatus |
| 146 | + |
| 147 | +### What To Solve? |
| 148 | + |
| 149 | ++ Support to the ability to acquire the volumes of the Container. |
| 150 | ++ Support to the ability to acquire the Resource of the Container. |
| 151 | ++ Pass the quotaID generated when the container is created for disk reuse. |
| 152 | + |
| 153 | +### Modification |
| 154 | + |
| 155 | ++ The ContainerStatus struct is used only in ContainerStatusResponse in CRI, so the volumes of the container can be obtained by directly adding volume field to the ContainerStatus struct. |
| 156 | ++ Add Resources field to support the retrieval of container's resource. |
| 157 | ++ When get ContainerStatus, the return object of ContainerStatusResponse will contain the field of QuotaId . |
| 158 | + |
| 159 | +``` |
| 160 | +// ContainerStatus represents the status of a container. |
| 161 | +type ContainerStatus struct { |
| 162 | + ...... |
| 163 | + //***New Fields***// |
| 164 | + // Volumes of container |
| 165 | + Volumes map[string]struct{} `protobuf:"bytes,100,opt,name=volumes,json=volumes" json:"volumes,omitempty"` |
| 166 | + // Resources specification for the container |
| 167 | + Resources *LinuxContainerResources `protobuf:"bytes,101,opt,name=resources" json:"resources,omitempty"` |
| 168 | + // QuotaId of the container |
| 169 | + QuotaId string `protobuf:"bytes,102,opt,name=quota_id,json=quotaId,proto3" json:"quota_id,omitempty"` |
| 170 | +} |
| 171 | +``` |
| 172 | + |
| 173 | +The changes need to be made in the proto file are as follows: |
| 174 | + |
| 175 | +``` |
| 176 | +// ContainerStatus represents the status of a container. |
| 177 | +message ContainerStatus { |
| 178 | + ...... |
| 179 | + //***New Fields***// |
| 180 | + // Volumes of container |
| 181 | + map<string, Volume> volumes= 100; |
| 182 | + // Resources specification for the container |
| 183 | + LinuxContainerResources resources = 101; |
| 184 | + // QuotaId of the container |
| 185 | + string quota_id = 102; |
| 186 | +} |
| 187 | +
|
| 188 | +message Volume { |
| 189 | +} |
| 190 | +``` |
| 191 | + |
| 192 | +## ImageStatus |
| 193 | + |
| 194 | +### What To Solve? |
| 195 | + |
| 196 | ++ Support the ability to acquire the volumes of the Image. |
| 197 | + |
| 198 | +### Modification |
| 199 | + |
| 200 | ++ Add volumes field in the Image struct. |
| 201 | + |
| 202 | +``` |
| 203 | +// Basic information about a container image. |
| 204 | +type Image struct { |
| 205 | + ...... |
| 206 | + //***New Fields***// |
| 207 | + // Volumes of image |
| 208 | + Volumes map[string]struct{} `protobuf:"bytes,7,opt,name=volumes,json=volumes" json:"volumes,omitempty"` |
| 209 | +} |
| 210 | +``` |
| 211 | + |
| 212 | +The changes need to be made in the proto file are as follows: |
| 213 | + |
| 214 | +``` |
| 215 | +// Basic information about a container image. |
| 216 | +message Image { |
| 217 | + ...... |
| 218 | + //***New Fields***// |
| 219 | + // Volumes of image |
| 220 | + map<string, Volume> volumes= 100; |
| 221 | +} |
| 222 | +``` |
| 223 | + |
| 224 | +## CreateContainer |
| 225 | + |
| 226 | +###What To Solve? |
| 227 | + |
| 228 | ++ Support the ability to set DiskQuota. |
| 229 | ++ Add missing fields. |
| 230 | + |
| 231 | +### Modification |
| 232 | + |
| 233 | ++ LinuxContainerConfig contains LinuxContainerResources, which have changed in UpdateContainerResources().So after changing LinuxContainerResources, the Create process already supports the setting of DiskQuota. |
| 234 | ++ For missing fields are as follows (not all): |
| 235 | + + NetPriority : Set network priorities |
| 236 | ++ QuotaId : When creating container, pass parameters of the DiskQuota and QuotaId. (When QuotaId is -1, QuotaId will be automatically generated) |
| 237 | + |
| 238 | +``` |
| 239 | +type ContainerConfig struct { |
| 240 | + ...... |
| 241 | + //***New Fields***// |
| 242 | + // NetPriority of the container |
| 243 | + NetPriority int64 `protobuf:"bytes,100,opt,name=net_priority" json:"net_priority,omitempty"` |
| 244 | + // QuotaId of the container |
| 245 | + QuotaId string `protobuf:"bytes,101,opt,name=quota_id,json=quotaId,proto3" json:"quota_id,omitempty"` |
| 246 | +} |
| 247 | +``` |
| 248 | + |
| 249 | +The changes need to be made in the proto file are as follows: |
| 250 | + |
| 251 | +``` |
| 252 | +message ContainerConfig { |
| 253 | + ...... |
| 254 | + //***New Fields***// |
| 255 | + // NetPriority of the containeri |
| 256 | + int64 net_priority = 100; |
| 257 | + // QuotaId of the container |
| 258 | + string quota_id = 101; |
| 259 | +} |
| 260 | +``` |
| 261 | + |
| 262 | +## RemoveVolume |
| 263 | + |
| 264 | +### What To Solve? |
| 265 | + |
| 266 | ++ After kubelet performs in-situ upgrade, the container after upgrade cannot delete the anonymous volume inherited. |
| 267 | + |
| 268 | +### Modification |
| 269 | + |
| 270 | ++ Provides an interface for removing volume. |
| 271 | ++ The containerstatus interface supports querying volume by name. |
| 272 | ++ The changes need to be made in the proto file are as follows: |
| 273 | + |
| 274 | +``` |
| 275 | +service VolumeService { |
| 276 | + // RemoveVolume volume an volume |
| 277 | + rpc RemoveVolume(RemoveVolumeRequest) returns (RemoveVolumeResponse) {} |
| 278 | +} |
| 279 | +
|
| 280 | +message RemoveVolumeRequest { |
| 281 | + // name of the volume to remove |
| 282 | + string volume_name = 1; |
| 283 | +} |
| 284 | +message RemoveVolumeResponse {} |
| 285 | +``` |
| 286 | + |
| 287 | +Add name field in the Mount struct: |
| 288 | + |
| 289 | +``` |
| 290 | +// Mount specifies a host volume to mount into a container. |
| 291 | +message Mount { |
| 292 | + ...... |
| 293 | + //***New Fields***// |
| 294 | + // Name of volume |
| 295 | + string name = 100; |
| 296 | +} |
| 297 | +
|
| 298 | +``` |
| 299 | + |
| 300 | +## Pull Request |
| 301 | + |
| 302 | ++ feature: extend cri apis for special needs [#1617](https://github.com/alibaba/pouch/pull/1617) |
| 303 | ++ feature: extend cri apis for remove volume [#2124](https://github.com/alibaba/pouch/pull/2124) |
| 304 | ++ feature: extend cri apis for support quotaID [#2138](https://github.com/alibaba/pouch/pull/2138) |
| 305 | + |
0 commit comments