|
| 1 | +--- |
| 2 | +title: Implementing LoadBalancer services on Cluster API KubeVirt clusters using Cloud Provider KubeVirt |
| 3 | +date: 2025-01-09 |
| 4 | +description: Had trouble getting load balancer services working on Cluster API KubeVirt clusters? This guide will get you sorted out. |
| 5 | +avatar: tie-blue |
| 6 | +images: |
| 7 | + ogPath: 1x1.png |
| 8 | + ldPaths: |
| 9 | + - 1x1.png |
| 10 | + - 4x3.png |
| 11 | + - 16x9.png |
| 12 | +--- |
| 13 | + |
| 14 | +This article is the beginning of a series on taking Cluster API managed clusters |
| 15 | +on KubeVirt from where the documentation leaves you to fully functioning |
| 16 | +production clusters. |
| 17 | +Make sure to check out the next parts in the future. |
| 18 | + |
| 19 | +Recently I've been moving my personal infrastructure to Kubernetes clusters |
| 20 | +managed by Cluster API on KubeVirt. |
| 21 | +After going through the Cluster API documentation, and getting my first clusters |
| 22 | +up and running I encountered a problem: how do I get a working load balancer |
| 23 | +implementation? |
| 24 | + |
| 25 | +After going through the Kubernetes Slack I found the |
| 26 | +[Cloud Provider KubeVirt](https://github.com/kubevirt/cloud-provider-kubevirt) |
| 27 | +project. Yet it lacks documentation, and it is not clear how to get it working. |
| 28 | +This post will explain how it works and how to set it up. |
| 29 | + |
| 30 | +Since terminology with Cluster API is a bit confusing, this article will refer to |
| 31 | +the cluster running KubeVirt and Cluster API as the management cluster, and to |
| 32 | +clusters managed by Cluster API as tenant clusters. |
| 33 | + |
| 34 | +## The problem with load balancer implementations |
| 35 | + |
| 36 | +Using MetalLB, Kube VIP or other ARP/BGP based load balancer implementations does |
| 37 | +not work inside KubeVirt Cluster API tenant clusters. Tenant Nodes are connected to the management cluster network without |
| 38 | +direct access to the outside network, preventing load balancers from functioning. |
| 39 | + |
| 40 | +## The solution to provisioning load balancer services |
| 41 | + |
| 42 | +KubeVirt offers a cloud controller manager that provisions LoadBalancer type |
| 43 | +services on the management cluster. |
| 44 | + |
| 45 | +Cloud Provider KubeVirt runs on the management cluster and watches for Service resources |
| 46 | +of type LoadBalancer in the tenant cluster. |
| 47 | +When a Service of type LoadBalancer is created in the tenant cluster, Cloud Provider |
| 48 | +KubeVirt creates a matching service of type LoadBalancer in the management cluster and |
| 49 | +connects it to the service in the tenant via node ports. |
| 50 | + |
| 51 | +## Installing the cloud controller manager |
| 52 | + |
| 53 | +The Cluster API Provider KubeVirt project supplies templates with the cloud |
| 54 | +controller manager. To use them specify `--flavor lb-kccm` when generating |
| 55 | +manifests with `clusterctl`. |
| 56 | +`kccm`, `passt-kccm` and `persistent-storage-kccm` template flavors are also |
| 57 | +avaiable. |
| 58 | +In this example we'll use the `lb-kccm` template. |
| 59 | + |
| 60 | +Set environment variables configuring `clusterctl` manifest templating: |
| 61 | + |
| 62 | +```sh |
| 63 | +export NODE_VM_IMAGE_TEMPLATE='quay.io/capk/ubuntu-2204-container-disk:v1.30.1' |
| 64 | +export CRI_PATH='/var/run/containerd/containerd.sock' |
| 65 | +``` |
| 66 | + |
| 67 | +Generate manifests with `clusterctl`: |
| 68 | + |
| 69 | +```sh |
| 70 | +clusterctl generate cluster capi-quickstart --infrastructure kubevirt:v0.1.9 --flavor lb-kccm --kubernetes-version v1.30.1 --control-plane-machine-count 1 --worker-machine-count 1 > capi-quickstart.yml |
| 71 | +``` |
| 72 | + |
| 73 | +Apply the manifests: |
| 74 | + |
| 75 | +```sh |
| 76 | +kubectl apply -f capi-quickstart.yml |
| 77 | +``` |
| 78 | + |
| 79 | +Installing a network plugin, creating a Pod and LoadBalancer type service have |
| 80 | +been omitted for brevity. |
| 81 | + |
| 82 | +For a full example [see the cluster chart from my infrastructure charts |
| 83 | +repository](https://github.com/SneakyBugs/Helm-Charts/tree/main/charts/cluster). |
| 84 | +Specifically [the cloud-controller-manager templates subdirectory.](https://github.com/SneakyBugs/Helm-Charts/tree/main/charts/cluster/templates/cloud-controller-manager) |
| 85 | +The chart includes installation of a network plugin and ingress exposed over a |
| 86 | +LoadBalancer type service. |
0 commit comments