Single-node Kubernetes on Home Lab using MicroK8s, Metallb, and Traefik
MicroK8s: High availability, Low-ops, Minimal Kubernetes for All-Size Clusters
In my recent blog, I have setup a single-node Kubernetes cluster on my home lab server using k0s. It is a good tool that make it easy to setup a cluster in just one or two commands. However, as it may be relatively new, I found a problem when starting the cluster that worker node sometime isn’t up and running and you need to restart the service.
Setup Single-node Kubernetes Cluster on a Home Lab server using k0s
Reducing your cloud bill and make use of your PC at home
In this blog, I will try another tool called MicroK8s which is around for sometime and hopefully will be more stable than k0s to setup a single-node Kubernetes cluster on Ubuntu Linux 20.04 on my home lab box.
- A public IP address from your internet provider, either dynamic or static.
- A domain name mapped to your static IP or a dynamic DNS domain name for dynamic IP which can be configured on your modem router to sync with the Dynamic DNS provider e.g. no-ip.
- You have a fresh install of Ubuntu 20.04 on your home lab server — see this blog for how-to.
- SSH has been configured for remote access on your home lab server— see my previous blogs on how to setup on home lab or Linode VM.
- kubectl, git, and helm are installed on your local machine
In this section, we will install MicroK8s on our Ubuntu server.
On your server, use snap to install the MicroK8s package.
sudo snap install microk8s --classic --channel=1.21
Add yourself into the group
microk8s, gain access to
.kube caching directory, and refresh the session so group update takes effect.
sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
su - $USER
Monitor the cluster provisioning status. This may take a few minutes until the cluster is ready.
microk8s status --wait-ready
Get nodes and services on the cluster.
microk8s kubectl get nodes
microk8s kubectl get services
Enable foundation addons
microk8s enable dns storage
To enable remote access to the api-server using kubectl, edit the file
/var/snap/microk8s/current/certs/csr.conf.template on your server and add domain name and/or IP address (if static) of your server under
alt_names section that reachable from the internet.
[ alt_names ]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster
DNS.5 = kubernetes.default.svc.cluster.local
DNS.6 = your.dnsname.com
IP.1 = 127.0.0.1
IP.2 = 10.152.183.1
IP.3 = 192.168.1.xx
IP.4 = 123.456.789.0
Export kubeconfig to file.
microk8s config > admin.config
On your local machine, copy the kubeconfig file from the server to your machine.
scp email@example.com:~/admin.config .
admin.config file and update
clusters.cluster.server by replacing the IP and port with your domain name or public IP (if static). It is recommended to use the port other than 16443 for better security and then configure your router to forward the specified port to your server’s port 16443.
Test the connection
kubectl get nodes
In this section, we will deploy an application named
whoami for testing purpose.
First, clone this repository and apply deployment and service on the cluster.
git clone https://github.com/pacroy/whoami.git
kubectl create ns whoami
kubectl apply -f whoami.yml -n whoami
Once the pod is up and running, forward port to its service.
kubectl port-forward service/whoami 8080:80 -n whoami
Go to localhost:8080 on your local machine and you should see the response like this:
To expose a service with LoadBalancer, we need to enable metallb which a load balancer for bare metal.
microk8s enable metallb
It will ask to input IP address range allocated for load balancers. It is good to assign s small IP address pool within your subnet that are outside your DHCP range to avoid collision.
In this case I choose
Enter each IP address range delimited by comma (e.g. ‘10.64.140.43–10.64.140.49,192.168.0.105–192.168.0.111’): 192.168.1.85–192.168.1.89
Update whoami service type to LoadBalancer.
apply -f service-lb.yml -n whoami
Test the load balancer by taking note the EXTERNAL-IP and opening it in your browser (in this case it is http://192.168.1.85) and you should see the same result as before.
Restore the service type to ClusterIP as we will expose it via Ingress in the next section.
kubectl apply -f whoami.yml -n whoami
Install Traefik Ingress
On your server, enable traefik ingress controller for external access.
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
kubectl create namespace traefik
helm install traefik traefik/traefik -n traefik
Check component and you should see the traefik service is exposed via LoadBalancer on both port 80 and 443, by default.
Restore whoami service type to ClusterIP and create an ingress which route traffics at any host with URI
/whoami to our service.
kubectl apply -f ingress.yml -n whoami
Configure your modem router to forward port 80 and 443 to your LoadBalancer IP address. Instruction varies by router’s brand and model and won’t be covered here.
Open your browser and go to your domain name or public IP and access URI
/whoami. In my case, it is http://microk8s.ddns.net/whoami.
You can access traefik dashboard by forwarding port 9000 from a traefik pod.
kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -n traefik) -n traefik 9000:9000
Then access the traefik dashboard at http://localhost:9000/dashboard
UPDATE: You may wanna check first if your domain name can work with Let’s Encrypt by follow what I did in this blog.
Now, if you try to access https://your.dnsname.com/whoami then you will get this warning page.
This is because the traefik is using a self-signed certificate which is not trusted by the browser.
To make our whoami website trusted by the browser, we need to tell traefik to get and use a certificate that issued by a trusted certificate authority (CA) e.g. Let’s Encrypt.
Delete the ingress as we will use IngressRoute instead in the next section.
kubectl delete ingress/whoami -n whoami
Instead of using standard Ingress object with lots of annotations, traefik also provides their own CRD called IngressRoute that make the configuration more readable and structured.
Let’s try IngressRoute. Edit the file
ingressroute.yml and update the hostname
microk8s.ddns.net with your domain name then apply.
kubectl apply -f ingressroute.yml -n whoami
Try accessing whoami again at http://your.dnsname.com/whoami and you should see the same result.
Next, we need to upgrade the traefik to include a certificate resolver. Edit the file values.yml and update
firstname.lastname@example.org with your email address. This will be used by Let’s Encrypt to send notification email when the certificate nearly expires.
Upgrade the traefik release.
helm upgrade traefik traefik/traefik -n traefik --values values.yml
Check if the arguments are applied correctly by looking the Deployment’s manifest.
helm get manifest traefik -n traefik
Check traefik’s log and you should see no error about the certificate resolver.
kubectl logs $(kubectl get pod -n traefik -o name) -n traefik -f
IngressRoute with TLS
Now, let’s apply a new IngressRoute that using the certificate resolver. Don’t forget to update the hostname to yours!
kubectl apply ingressroute-tls.yml -n whoami
Check if the certificate is created properly.
kubectl exec -it $(kubectl get pod -n traefik -o name) -n traefik -- cat /data/letsencrypt.json
NOTE: It may take a while before the certificate shows up for the configured domain.
Test accessing your application at https://your.dnsname.com/whoami and the browser should show the page with a valid certificate without warning.
This section will show you how to enhance security for our application.
TLS version 1.2
HTTPS should no longer support SSL and TLS v1.1 and TLS v1.2 as they are weak in term of security. If we scan our endpoint with SSLLabs, you will get rating at B.
We can make our traefik ingress to accept connection with minimum TLS v1.2 by creating a new TLSOption in default namespace.
kubectl apply -f tlsoption.yml
This create a default TLSoption that will apply to all traefik routers by default (if not explicitly overridden). Scan the endpoint again with SSLLabs and you will now get rating A.
Redirect to HTTPS
So far, our whoami application serves HTTP traffic at port 80 and HTTPS traffic at port 443. What if we want to redirect all HTTP traffics to HTTPS. We need to leverage RedirectScheme middleware.
Let’s create a new middleware in default namespace.
kubectl apply -f middleware-https.yml
Then replace IngressRoute with the new one with the middleware.
kubectl apply -f ingressroute-http.yml -n whoami
Test your HTTP endpoint and it should now redirect to HTTPS.
$ curl http://microk8s.ddns.net/whoami -D-
HTTP/1.1 301 Moved Permanently
Date: Mon, 12 Apr 2021 14:14:36 GMT
Content-Type: text/plain; charset=utf-8Moved Permanently