Using MetalLB And Traefik for Load balancing on your Bare Metal Kubernetes Cluster – Part 1

Running a Kubernetes Cluster in your own data center on Bare Metal hardware can be lots of fun but also can be challenging. One of the changeless are exposing your service to an external Load Balancer, Kubernetes does not offer an implementation of an external load-balancer for bare metal cluster implementations. The Kubernetes implementations of Network Load Balancers are only available on specific Cloud Providers like AWS, GCP, Azure, etc.. and is enabled by specifying the –cloud-provider= option. leaving a gap/challenge if you create your own Bare Metal Kubernetes Cluster. While their are many workarounds to address this issue most of them are not perfect and have their own list of challenges. while I was looking for a for a solution, most recently came across a project called MetalLB, at first I was quite sceptical if its going to work, but after testing this for a while I have to say, I am quite happy with the solution. Below is a quote from the MetalLB website.

Kubernetes does not offer an implementation of network load-balancers (Services of type LoadBalancer) for bare metal clusters. The implementations of Network LB that Kubernetes does ship with are all glue code that calls out to various IaaS platforms (GCP, AWS, Azure…). If you’re not running on a supported IaaS platform (GCP, AWS, Azure…), LoadBalancers will remain in the “pending” state indefinitely when created.

Note: On the MetalLB project website they warn you as follows. MetalLB is a young project. You should treat it as a beta system. The project maturity page explains what that implies., however I using this for a while and it seems quite mature. One last note, I am continuing to use Traefik for internal micro services / load balancing which works great, however MetalLB helps exposing your service for external use. Now that we have that out of the way, lets see what it takes to install and configure MetalLB on your Kubernetes cluster.

Installing MetalLB on your Kubernetes cluster

You have two options to install MetalLB on your Kubernetes cluster.

1. Use the a per-packged MetalLBHelm chart.
2. Install / configure manually by applying a YAML configuration file.

Installing MetalLB with Helm

Note: To use Helm you first need to install/configure Helm/Tiller on your Kubernetes cluster. you can see a how-to step by step process here Part 1 and here Part 2. First, lets create a configmap which will be using at helm install time, the below configmap was used with a flannel configuration. cat values.yaml

configInline:
  address-pools:
  - name: default
    protocol: layer2
    addresses:
    - 10.90.10.190-10.90.10.199

Note: There are a number of networking options you can specify likeprotocol: BGP, etc.. please take a look at the MetalLB website for a full list of options. Once helm is available on your cluster, just run the below to install MetalLB (omit the -tls if your are not using a secure configuration). Note: The below configuration installs MetalLB with helm including a configmap. (otherwise you get the errors below)

helm install --name=metallb --namespace=metallb-system -f values.yaml stable/metallb --tls

# If all is ok, you shuld see something like the below message at the end of the helm deploy.
MetalLB is now running in the cluster.
LoadBalancer Services in your cluster are now available on the IPs you
defined in MetalLB's configuration. To see IP assignments,
try `kubectl get services`.

# Without specifying a configmap.
helm install stable/metallb --name=metallb --namespace=metallb-system --tls

Errors you will see if you don’t specify / use a configmap to list your network range, etc..

MetalLB is now running in the cluster.
WARNING: you specified a ConfigMap that isn't managed by
Helm. LoadBalancer services will not function until you add that
ConfigMap to your cluster yourself.

You can also check the logs to make sure all looks good, by running the below

kubectl logs -l component=speaker -n metallb-system

If your cluster is not configured to use RBAC, you might run somthing like the below.

helm install \
--set rbac.create=false \
-f values.yaml \
stable/metallb --name=metallb --namespace=metallb-system --tls

An example configmap if using a BGP overlay system.

# cat values.yaml
configInline:
  peers:
  - peer-address: 10.0.0.1
    peer-asn: 64512
    my-asn: 64512
  address-pools:
  - name: default
    protocol: bgp
    addresses:
    - 10.90.10.190-10.90.10.199

If all works correctly, you should now have a MetalLB working configuration. You can also check what helm report about the installation.

helm list --tls
NAME   	REVISION	UPDATED                 	STATUS  	CHART        	APP VERSION	NAMESPACE     
metallb	1       	Mon Feb 11 19:52:17 2019	DEPLOYED	metallb-0.8.4	0.7.3      	metallb-system

To remove MetalLB from your cluster, just run the below.

helm delete --purge metallb --tls

Installing MetalLB manually

To install MetalLB manually, you would do so in cases ware you don’t have a helm configuration or you would like to install a version not available in the helm repository. To MetalLB from the web just run the below.

kubectl apply -f https://raw.githubusercontent.com/google/metallb/master/manifests/metallb.yaml

You can also create a deploy file, like the one below.

cat deploye_metal.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: metal-lb-ip-space
      protocol: layer2
      addresses:
      - 10.90.10.190-10.90.10.199

Then to install just run the below, this will already include your IP address definitions.

kubectl apply -f deploye_metal.yaml

A quick check on your pods will reveal something like the below, note the metallb-controller and metallb-speaker.

kubectl get po --all-namespaces
metallb-system   metallb-controller-776cdd69dc-4lzcx   1/1     Running   0          9d
metallb-system   metallb-speaker-4xrnm                 1/1     Running   0          9d
metallb-system   metallb-speaker-g59h6                 1/1     Running   0          9d
metallb-system   metallb-speaker-jzh4g                 1/1     Running   0          9d
metallb-system   metallb-speaker-k6spd                 1/1     Running   0          9d

MetalLB in action

To test MetalLB working, you can deploy any pod(s) to you cluster, and specify the Cluster Type as LoadBalancer. An example Nginx deployment is below.

# cat nginx_load_balance.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1
        ports:
        - name: http
          containerPort: 80

---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Now, just run the below to deploy to your cluster.

kubectl apply -f nginx_load_balance.yaml

Now take a look on your Kubernets Cluster services, note the EXTERNAL-IP that got assigned automatically by MetalLB.

kubectl get svc --all-namespaces
NAMESPACE     NAME            TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)     AGE
default       kubernetes      ClusterIP   10.30.0.1             443/TCP     14d
kube-system   tiller-deploy   ClusterIP   10.30.4.154           44134/TCP   13d
default       nginx           LoadBalancer   10.30.2.56    10.90.10.190   80:30827/TCP   2s
...

In the next post Part 2 – I will show you how to combine this with Traefik for internal Micro-services Load Balancing. I hope you enjoyed reading Using MetalLB And Traefik Load Balancing For Your Bare Metal Kubernetes Cluster, give it a thumbs up by rating the article or by just providing feedback. You might also like – realted to Docker Kubernetes / micro-services.