This is a Vagrant Environment for a playing with Talos.
For playing with Sidero see the rgl/sidero-vagrant repository.
- Architecture
- Usage
- Network Packet Capture
- Network Booting
- Troubleshoot
- Alternatives and Related Projects
- References
Install docker, vagrant, vagrant-libvirt, and the Ubuntu Base Box.
Login into docker hub to have a higher rate limits.
If you want to connect to the external physical network, you must configure your host network as described in rgl/ansible-collection-tp-link-easy-smart-switch (e.g. have the br-rpi linux bridge) and set CONFIG_PANDORA_BRIDGE_NAME in the Vagrantfile.
Bring up the cluster virtual machines:
time ./bring-up.sh | tee bring-up.logAccess talos:
export TALOSCONFIG="$PWD/shared/talosconfig"
./shared/talosctl --nodes cp1,w1 versionAccess kubernetes:
export KUBECONFIG="$PWD/shared/kubeconfig"
./shared/kubectl get nodes -o wideStart an example service in each worker node:
vagrant ssh -c 'bash /vagrant/provision-example-daemonset.sh' pandoraAccess the example service:
vagrant ssh -c "watch -n .2 'wget -qO- http://example-daemonset.\$(hostname --domain)?format=text | tail -25; kubectl get pod -l app=example-daemonset -o=custom-columns=NODE:.spec.nodeName,STATUS:.status.phase,NAME:.metadata.name'" pandoraList this repository dependencies (and which have newer versions):
export GITHUB_COM_TOKEN='YOUR_GITHUB_PERSONAL_TOKEN'
./renovate.shYou can easily capture and see traffic from the host with the wireshark.sh
script, e.g., to capture the traffic from the eth1 interface:
./wireshark.sh pandora eth1To delegate the talos.test zone to the kubernetes managed external dns server (running in pandora) you need to configure your system to delegate that DNS zone to the pandora DNS server, for that, you can configure your system to only use dnsmasq.
For example, on my Ubuntu 22.04 Desktop, I have uninstalled resolvconf, disabled NetworkManager, and manually configured the network interfaces:
sudo su -l
for n in NetworkManager NetworkManager-wait-online NetworkManager-dispatcher network-manager; do
systemctl mask --now $n
done
apt-get remove --purge resolvconf
cat >/etc/network/interfaces <<'EOF'
# interfaces(5) file used by ifup(8) and ifdown(8)
auto lo
iface lo inet loopback
auto enp3s0
iface enp3s0 inet dhcp
EOF
rebootThen, replaced systemd-resolved with dnsmasq:
sudo su -l
apt-get install -y --no-install-recommends dnsutils dnsmasq
systemctl mask --now systemd-resolved
cat >/etc/dnsmasq.d/local.conf <<EOF
no-resolv
bind-interfaces
interface=lo
listen-address=127.0.0.1
# delegate to the Cloudflare/APNIC Public DNS IP addresses.
# NB iif there's no entry in /etc/hosts.
server=1.1.1.1
server=1.0.0.1
# delegate to the Google Public DNS IP addresses.
# NB iif there's no entry in /etc/hosts.
#server=8.8.8.8
#server=8.8.4.4
EOF
cat >/etc/dnsmasq.d/talos.test.conf <<EOF
# delegate the talos.test zone to the pandora DNS server IP address.
# NB use the CONFIG_PANDORA_IP variable value defined in the Vagrantfile.
server=/talos.test/10.10.0.2
EOF
rm /etc/resolv.conf
cat >/etc/resolv.conf <<EOF
nameserver 127.0.0.1
EOF
systemctl restart dnsmasq
exitThen start all the machines and test the DNS resolution:
vagrant up
dig pandora.talos.testThis environment uses PXE/TFTP/iPXE/HTTP/UEFI-HTTP to network boot the machines.
The Virtual Machines are automatically configured to network boot.
To boot Physical Machines you have to:
- Create a Linux Bridge that can reach a Physical Switch that connects to
your Physical Machines.
- This environment assumes you have a setup like rgl/ansible-collection-tp-link-easy-smart-switch.
- To configure it otherwise you must modify the
Vagrantfile.
- Add your machines to
machines.yaml. - Configure your machines to PXE boot.
This was tested on the following physical machines and boot modes:
- Seeed Studio Odyssey X86J4105
- It boots using UEFI/HTTP/PXE.
- HP EliteDesk 800 35W G2 Desktop Mini
- It boots using UEFI/TFTP/PXE.
- This machine can be remotely managed with MeshCommander.
- It was configured as described at rgl/intel-amt-notes.
- Raspberry Pi 4 (8GB)
- It boots using UEFI/HTTP/iPXE.
- The machine boot order must be
diskandnetwork.- Talos expects to be run from disk.
- Do not configure any default nodes with
talosctl config node.- Instead, explicitly target the node with
talosctl -n {node}. - Having default nodes could lead to mistakes (e.g. upgrading the whole cluster at the same time).
- Instead, explicitly target the node with
- The user only needs to access the talos control plane machines.
- A control plane machine will proxy the requests to the internal cluster nodes.
- Talos
- Troubleshooting Control Plane
talosctl -n cp1 service etcd statustalosctl -n cp1 etcd memberstalosctl -n cp1 get memberstalosctl -n cp1 dashboardtalosctl -n cp1 logs controller-runtimetalosctl -n cp1 logs kubelettalosctl -n cp1 diskstalosctl -n cp1 mountstalosctl -n cp1 get resourcedefinitionstalosctl -n cp1 get machineconfigs -o yamltalosctl -n cp1 get staticpods -o yamltalosctl -n cp1 get staticpodstatustalosctl -n cp1 get manifeststalosctl -n cp1 get servicestalosctl -n cp1 get extensionstalosctl -n cp1 get addressestalosctl -n cp1 get nodeaddressestalosctl -n cp1 list -l -r -t f /etctalosctl -n cp1 list -l -r -t f /systemtalosctl -n cp1 list -l -r -t f /vartalosctl -n cp1 list -l /sys/fs/cgrouptalosctl -n cp1 read /proc/cmdline | tr ' ' '\n'talosctl -n cp1 read /proc/mounts | sorttalosctl -n cp1 read /etc/resolv.conftalosctl -n cp1 read /etc/containerd/config.tomltalosctl -n cp1 read /etc/cri/containerd.tomltalosctl -n cp1 read /etc/cri/conf.d/cri.toml(registry credentials)talosctl -n cp1 read /etc/cri/conf.d/hosts/docker.io/hosts.toml(registry mirror)talosctl -n cp1 pstalosctl -n cp1 containers -ktalos-poke cp1
- Kubernetes
kubectl get events --all-namespaces --watchkubectl --namespace kube-system get events --watchkubectl run busybox -it --rm --restart=Never --image=busybox:1.33 -- nslookup -type=a pandora.talos.test
- Talos
- Linux
- iPXE
- Raspberry Pi
- Matchbox
- Dynamic Host Configuration Protocol (DHCP)