k8s nodes running in proxmox

  • 4 CPUs
  • 8 GB RAM
  • 150 GB root disk (TrueNAS iSCSI)

os + kubelet

booted talos from netboot.xyz over PXE and followed directions from talos setup

surprisingly easy, going to tear down and recreate cluster to see how quickly it can be done.

network

needs

  • http + tls ingress/gateway api
    • traefik
    • istio
    • contour
  • virtual IPs
    • metal lb
      • L2 mode is only viable option
      • single node bottleneck
      • slow failover
    • cilium-lb
      • native part of cilium
      • activated when ip pool created
    • kube-vip
      • seems to be connected to cilium staff
  • automated certs
    • cert-manager + cloudflare - EZ

storage

options

  • NFS
    • multi-attach
    • wide support
    • slow