Single-node Kubernetes on Home Lab using MicroK8s, Metallb, and Traefik

MicroK8s: High availability, Low-ops, Minimal Kubernetes for All-Size Clusters

In my recent blog, I have setup a single-node Kubernetes cluster on my home lab server using k0s. It is a good tool that make it easy to setup a cluster in just one or two commands. However, as it may be relatively new, I found a problem when starting the cluster that worker node sometime isn’t up and running and you need to restart the service.

TL;DR

In this blog, I will try another tool called MicroK8s which is around for sometime and hopefully will be more stable than k0s to setup a single-node Kubernetes cluster on Ubuntu Linux 20.04 on my home lab box.

Prerequisites

  • A public IP address from your internet provider, either dynamic or static.
  • A domain name mapped to your static IP or a dynamic DNS domain name for dynamic IP which can be configured on your modem router to sync with the Dynamic DNS provider e.g. no-ip.
  • You have a fresh install of Ubuntu 20.04 on your home lab server — see this blog for how-to.
  • SSH has been configured for remote access on your home lab server— see my previous blogs on how to setup on home lab or Linode VM.
  • kubectl, git, and helm are installed on your local machine

Setup MicroK8s

In this section, we will install MicroK8s on our Ubuntu server.

Install MicroK8s

On your server, use snap to install the MicroK8s package.

sudo snap install microk8s --classic --channel=1.21

Add yourself into the group microk8s, gain access to .kube caching directory, and refresh the session so group update takes effect.

sudo usermod -a -G microk8s $USER
sudo chown -f -R $USER ~/.kube
su - $USER

Monitor the cluster provisioning status. This may take a few minutes until the cluster is ready.

microk8s status --wait-ready

Get nodes and services on the cluster.

microk8s kubectl get nodes
microk8s kubectl get services

Enable foundation addons dns and storage

microk8s enable dns storage

Remote Access

To enable remote access to the api-server using kubectl, edit the file /var/snap/microk8s/current/certs/csr.conf.template on your server and add domain name and/or IP address (if static) of your server under alt_names section that reachable from the internet.

[ alt_names ]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster
DNS.5 = kubernetes.default.svc.cluster.local
DNS.6 = your.dnsname.com
IP.1 = 127.0.0.1
IP.2 = 10.152.183.1
IP.3 = 192.168.1.xx
IP.4 = 123.456.789.0

Export kubeconfig to file.

microk8s config > admin.config

On your local machine, copy the kubeconfig file from the server to your machine.

scp username@your.dnsname.com:~/admin.config .

Edit the admin.config file and update clusters.cluster.server by replacing the IP and port with your domain name or public IP (if static). It is recommended to use the port other than 16443 for better security and then configure your router to forward the specified port to your server’s port 16443.

Test the connection

export KUBECONFIG=admin.config
kubectl version
kubectl get nodes
kubectl commands — screen captured from Windows Terminal by the author

Deploy Application

In this section, we will deploy an application named whoami for testing purpose.

Install whoami

First, clone this repository and apply deployment and service on the cluster.

git clone https://github.com/pacroy/whoami.git
cd whoami
kubectl create ns whoami
kubectl apply -f whoami.yml -n whoami

Once the pod is up and running, forward port to its service.

kubectl port-forward service/whoami 8080:80 -n whoami

Go to localhost:8080 on your local machine and you should see the response like this:

whoami via port-forward — screen captured from Microsoft Edge by the author

Expose Service

In this section, we will expose whoami service to the outside world using metallb load balancer and traefik ingress controller.

Enable meltallb

To expose a service with LoadBalancer, we need to enable metallb which a load balancer for bare metal.

microk8s enable metallb

It will ask to input IP address range allocated for load balancers. It is good to assign s small IP address pool within your subnet that are outside your DHCP range to avoid collision.

In this case I choose 192.168.1.85–192.168.1.89.

Enabling MetalLB
Enter each IP address range delimited by comma (e.g. ‘10.64.140.43–10.64.140.49,192.168.0.105–192.168.0.111’): 192.168.1.85–192.168.1.89

Update whoami service type to LoadBalancer.

apply -f service-lb.yml -n whoami

Get service

Test the load balancer by taking note the EXTERNAL-IP and opening it in your browser (in this case it is http://192.168.1.85) and you should see the same result as before.

Restore the service type to ClusterIP as we will expose it via Ingress in the next section.

kubectl apply -f whoami.yml -n whoami

Install Traefik Ingress

On your server, enable traefik ingress controller for external access.

helm repo add traefik https://helm.traefik.io/traefik
helm repo update
kubectl create namespace traefik
helm install traefik traefik/traefik -n traefik

Check component and you should see the traefik service is exposed via LoadBalancer on both port 80 and 443, by default.

Restore whoami service type to ClusterIP and create an ingress which route traffics at any host with URI /whoami to our service.

kubectl apply -f ingress.yml -n whoami

Configure your modem router to forward port 80 and 443 to your LoadBalancer IP address. Instruction varies by router’s brand and model and won’t be covered here.

Open your browser and go to your domain name or public IP and access URI /whoami. In my case, it is http://microk8s.ddns.net/whoami.

whoami via ingress — screen captured from Microsoft Edge by the author

traefik Dashboard

You can access traefik dashboard by forwarding port 9000 from a traefik pod.

kubectl port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -n traefik) -n traefik 9000:9000

Then access the traefik dashboard at http://localhost:9000/dashboard

traefik dashboard — screen captured from Microsoft Edge by the author

Configure HTTPs

In this section, we will enable HTTPS by setup a mechanism to automatically generate SSL certificate for our application using ACME provider from Let’s Encrypt authority (It’s free!).

UPDATE: You may wanna check first if your domain name can work with Let’s Encrypt by follow what I did in this blog.

Introduction

Now, if you try to access https://your.dnsname.com/whoami then you will get this warning page.

Invalid certificate — screen captured from Microsoft Edge by the author

This is because the traefik is using a self-signed certificate which is not trusted by the browser.

traefik self-signed certificate — screen captured by the author

To make our whoami website trusted by the browser, we need to tell traefik to get and use a certificate that issued by a trusted certificate authority (CA) e.g. Let’s Encrypt.

Delete the ingress as we will use IngressRoute instead in the next section.

kubectl delete ingress/whoami -n whoami

IngressRoute

Instead of using standard Ingress object with lots of annotations, traefik also provides their own CRD called IngressRoute that make the configuration more readable and structured.

Let’s try IngressRoute. Edit the file ingressroute.yml and update the hostname microk8s.ddns.net with your domain name then apply.

kubectl apply -f ingressroute.yml -n whoami

Try accessing whoami again at http://your.dnsname.com/whoami and you should see the same result.

Certificate Resolver

Next, we need to upgrade the traefik to include a certificate resolver. Edit the file values.yml and update youremail@domain.com with your email address. This will be used by Let’s Encrypt to send notification email when the certificate nearly expires.

Upgrade the traefik release.

helm upgrade traefik traefik/traefik -n traefik --values values.yml

Check if the arguments are applied correctly by looking the Deployment’s manifest.

helm get manifest traefik -n traefik

Check traefik’s log and you should see no error about the certificate resolver.

kubectl logs $(kubectl get pod -n traefik -o name) -n traefik -f

IngressRoute with TLS

Now, let’s apply a new IngressRoute that using the certificate resolver. Don’t forget to update the hostname to yours!

kubectl apply ingressroute-tls.yml -n whoami

Check if the certificate is created properly.

kubectl exec -it $(kubectl get pod -n traefik -o name) -n traefik -- cat /data/letsencrypt.json

NOTE: It may take a while before the certificate shows up for the configured domain.

Test accessing your application at https://your.dnsname.com/whoami and the browser should show the page with a valid certificate without warning.

whoami with valid certificate — screen captured from Microsoft Edge by the author
valid certificate issued by Let’s Encrypt — screen captured by the author

Security

This section will show you how to enhance security for our application.

TLS version 1.2

HTTPS should no longer support SSL and TLS v1.1 and TLS v1.2 as they are weak in term of security. If we scan our endpoint with SSLLabs, you will get rating at B.

Rate B HTTPS — screen captured from SSLLabs by the author

We can make our traefik ingress to accept connection with minimum TLS v1.2 by creating a new TLSOption in default namespace.

kubectl apply -f tlsoption.yml

This create a default TLSoption that will apply to all traefik routers by default (if not explicitly overridden). Scan the endpoint again with SSLLabs and you will now get rating A.

Rate A HTTPS — screen captured from SSLLabs by the author

Redirect to HTTPS

So far, our whoami application serves HTTP traffic at port 80 and HTTPS traffic at port 443. What if we want to redirect all HTTP traffics to HTTPS. We need to leverage RedirectScheme middleware.

Let’s create a new middleware in default namespace.

kubectl apply -f middleware-https.yml

Then replace IngressRoute with the new one with the middleware.

kubectl apply -f ingressroute-http.yml -n whoami

Test your HTTP endpoint and it should now redirect to HTTPS.

$ curl http://microk8s.ddns.net/whoami -D-
HTTP/1.1 301 Moved Permanently
Location: https://microk8s.ddns.net/whoami
Date: Mon, 12 Apr 2021 14:14:36 GMT
Content-Length: 17
Content-Type: text/plain; charset=utf-8
Moved Permanently

For more information about the traefik’s concept and configuration, please visit its documentation site.

See all available values for traefik Helm chart here.

Cheers!

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store