Setup Single-node Kubernetes Cluster on a Home Lab server using k0s

Reducing your cloud bill and make use of your PC at home

I have been running a small Kubernetes (k8s) cluster on a cloud provider for some time and finding alternative to reduce my bill. I could have run my small workloads on a small server that cost only $5 a month like I did for in this blog but I prefer a k8s cluster as all my existing configurations and scripts are built for it and I can keep learning and catch-up this popular ecosystem.

Recently, I have read this blog by Luc Juggery which show how easy you can setup a k8s cluster on your home lab server with k0s. Fortunately, I have a miniPC box with CPU Atom 1.44Ghz , 4GB RAM that I don’t use much. So here come this blog.

TL;DR

This blog guides you how to setup a single-node Kubernetes cluster with k0s on a miniPC running Ubuntu 20.04 OS and install ingress-nginx and cert-manager and a sample whoami application that can be publicly accessed over the Internet.

Prerequisites

  • Broadband internet with a good download/upload bandwidth — in my case it is 500/500Mbps
  • A home lab PC, at least 2 CPU cores, 2GB of RAM, connected to the your home broadband network
  • A public IP from your internet service provider — this may incur additional cost, check with your ISP

Install Ubuntu Server 20.04 on Home Lab server

Download Ubuntu Server 20.04 ISO file from its official site.

Make a bootable USB stick from the downloaded ISO file follow this official instruction.

Once you get the bootable USB stick, restart your PC and boot with your USB. Just follow on-screen instruction to install Ubuntu server.

Configure Ubuntu

Update System

Once the installation complete, log on and update the system.

apt-get update && apt-get -y upgrade

Configure IP Address

Configure your DHCP server in your router to issue a fixed IP address to your server. Try to assign an IP address that outside of your DHCP range to avoid collision. For example, your router may have DHCP range from 192.168.1.100 to 192.168.1.255. Then you may assign 192.168.1.10 to your server.

Configuration steps are vary by router. If not possible, you may need configure IP manually on Ubuntu. The instruction to do so is beyond this blog.

After DHCP is configured, you may restart the server and check IP address if it is configured properly.

# Reboot
sudo reboot
# Show local IP address
hostname -I | awk ‘{print $1}’

Setup SSH

Add your SSH key to the server by executing this command on another computer/laptop.

ssh-copy-id yourusername@ip_address

Edit the SSH config file

sudo nano /etc/ssh/sshd_config

Change the following lines for better security

PermitRootLogin no
PasswordAuthentication no

Restart SSH

systemctl restart sshd

Test SSH from another computer/laptop and make sure you can connect and logon.

ssh yourusername@ip_address

Configure SSH for Remote Access

If you want to SSH to your server from the internet, you need to configure port forwarding on your router. Choose the external port other than 22 for better security. This vary by router so the instruction is beyond this blog.

Once configured, you may test the connection.

ssh your_username@external_ip -p external_port

You can see your external IP address by executing this command on your server.

curl ipv4.icanhazip.com

If your public IP is static then this is fine. You can use the IP address to connect as it is fixed and will never change.

But most of ISPs won’t usually give you a static one but dynamic one i.e. your external IP keeps changing. In this case, you may need to configure Dynamic DNS (DDNS) settings on your router so you can access your server using a domain name and you don’t need to worry about the IP.

Again, the steps to configure DDNS are vary by router and will not be included in this blog.

To save some keys on SSH command, you can create the file ~/.ssh/config with the following content:

Host youralias
User your_username
Port external_port
Hostname your_domain_name

Then you can easily make SSH connection in short like this

ssh youralias

Unattended Upgrade

You can also configure the system to automatically upgrade and restart by following steps I wrote in this blog. Look for Setup Unattended Upgrades title.

Install k0s

Follow its official instruction by executing this command.

curl -sSLf https://get.k0s.sh | sudo sh

Check if it is install properly

$ k0s version
v0.11.0

Setup the Cluster

Building Config

Generate default config YAML file to a folder that make it accessible by root .

sudo k0s default-config > /root/k0s.yaml

Edit the file and append the following extension. Make sure you update two things:

  1. The IP address range load balancer —Use a small range of available and addressable IP addresses in your local networks that outside DHCP scope to avoid collision. In this example, I use 192.168.1.20–192.168.1.25.
  2. The email address in the cluster-issuer section.

This will install ingress-nginx and cert-manager on your cluster in one shot.

Start the Cluster

Install the cluster from our configuration file and start the server.

sudo k0s install controller -c /root/k0s.yaml — enable-worker
sudo systemctl start k0scontroller.service

Wait a bit and check server status

$ sudo k0s status
Version: v0.11.0
Process ID: 1472
Parent Process ID: 1
Role: controller+worker
Init System: linux-systemd

Check server node

$sudo k0s kubectl get nodes
NAME STATUS ROLES AGE VERSION
urserver1 Ready <none> 6d v1.20.4-k0s1

NOTE: After a few minutes, if you still don’t see the node then try to restart the service.

sudo systemctl restart k0scontroller.service

Reset the Cluster

In case your cluster go wrongly and you want to resetup the cluster then use this command to reset the cluster and restart the server before repeating the steps.

sudo k0s reset
sudo reboot

Remote Access to the Cluster

To access the cluster with kubectl, you need to copy the kubeconfig file.

sudo cp /var/lib/k0s/pki/admin.conf ~/admin.conf

And transfer to your other computer.

scp your_username@your_server:~/admin.conf ~

Edit the file and replace server field with your server IP address:port. If you decide to open this port (6443) to the internet. This could be your server’s external IP or domain name:external port.

apiVersion: v1
clusters:
- cluster:
certificate-authority-data: ***
server: https://your_server:port
name: local
...

Test connection

$ export KUBECONFIG=~/admin.conf
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.4-k0s1", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"clean", BuildDate:"2021-03-03T07:31:16Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}

If you server’s version information then the connection is successful.

Verify Cluster

Check if all pod are up and running. You may need to wait for a while until all pods are up.

kubectl get pods --all-namespaces

Check if all Helm charts are installed as expected

helm list --all-namespaces

Check ingress external IP. It should be your load balancer IP address on your local network that you defined in k0s.yaml file when setting up the server.

kubectl get services --namespaces kube-system

Forward Ports

To make your web application accessible from the internet, you need to forward the port 80 and 443. The configuration steps should be the same as you did for SSH (port 22) and kubectl (port 6443).

Eventually, you may have these port forwarding configuration on your router (Assume 192.168.1.10 is the server’s IP and 192.168.1.20 is the LB’s IP).

Ext.Port   Int.Port   Server IP
-------- -------- ---------
12322 22 192.168.1.10
12343 6443 192.168.1.10
80 80 192.168.1.20
443 443 192.168.1.20

Test Deploying whoami

Prepare Configuration File

Create a new namespace

kubectl create namespace whoami

Create a YAML file whoami.yml with the following content

Deploy Application and Test

Apply the YAML file to the namespace

$ kubectl apply -f whoami.yml --namespace whomai
deployment.apps/whoami-deployment created
service/whoami-service created
ingress.networking.k8s.io/whoami-ingress created

Check the status

kubectl get all --namespace whoami

Try accessing your application at http://load_balancer_ip/whoami

Configure HTTPS

To have a proper SSL certificate for HTTPS connection, you need a valid domain name. Then create a DNS A record pointing to your public IP address. The instruction is vary by DNS service provider and won’t be included here.

Check if DNS record can be resolved properly

$ nslookup whoami.yourdomain.com 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Non-authoritative answer:
Name: whoami.yourdomain.com
Address: xxx.xxx.xxx.xxx

Create a new ingress file ingress.yml with the following context:

Reapply the ingress

$ kubectl apply -f ingress.yml --namespace whoami
ingress.networking.k8s.io/whoami-ingress configured

Watch the pods. You will see the a cert-manager pod is created to auto-provision SSL certificate and then get terminated. This process usually takes less than one minute.

kubectl get pod --watch --namespace whoami

Check the certificate status and you should see True in the Ready column.

$ kubectl get certificates --namespace whoami
NAME READY SECRET AGE
whoami True whoami 2m56s

Now, test accessing your application at http://yourdomain.com/whoami and it should redirects to HTTPS with valid SSL certificate.

Clean up

Once you finished testing, you can delete whoami.

kubectl delete all --all --namespace whoami
kubectl delete namespace/whoami

Caveats

k0s worker node is sometimes not up and running

when you start the cluster first time, sometimes the worker node is not up even after some mount of time. In this case, restarting the service is usually help.

sudo systemctl restart k0scontroller.service

Cannot use CNAME record with cert-manager

In my environment, my public IP is dynamic and mapped to a DNS using Dynamic DNS (DDNS) service. I have my own custom domain that I try to map to DDNS domain using CNAME record. But that doesn’t seem to work for cert-manager.

I end up mapping my custom domain using a A record and thinking about to setup an automated task to regularly check and update DNS A record.

But please let me know if you find a better way to deal with this.