Installing, configuring 3 node Kubernetes(master) cluster on CentOS 7.5 – Installing and configuring flanneld

In Part 2 I described how to install and configure the Kubernetes VM’s, below I am continuing with with the installation and configuration of Flanneld, CNI plugin and Docker.

• Part 1: Initial setup – bear-metal installation, configuration
• Part 2: Installing the Kubernetes VM’s
• Part 3: Installing and configuring Flanneld, CNI plugin and Docker
• Part 4: Installing and configuring kubernetes manifest and kubelet service
• Part 5: Adding CoreDNS as part of the Kubernetes cluster
• Part 6: Adding / Configuring Kubernetes worker nodes
• Part 7: Enabling / Configuring RBAC, TLS Node bootstrapping
• Part 8: Installing / Configuring Helm, Prometheus, Alertmanager, Grafana and Elasticsearch

This is Part 3 – Installing and configuring Flanneld, CNI plugin and Docker. The first thing we are going to do, is grab the latest flanneld binary, you do so by running something like the below.

curl -o flannel-v0.10.0-linux-amd64.tar.gz https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz
tar zxf flannel-v0.10.0-linux-amd64.tar.gz
mv flanneld /usr/bin/flanneld

/usr/bin/flanneld -version
v0.10.0

Note: For a list of the latest flanneld versions click here. Make sure vXlan is enabled on your system, by running the below. Note: Flannel uses vXlan as the encapsulation protocol.

cat /boot/config-`uname -r` | grep CONFIG_VXLAN
CONFIG_VXLAN=m

Next, lets create the flanneld service cat /usr/lib/systemd/system/flanneld.service

[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
Requires=etcd.service
Requires=flanneld.service
After=etcd.service
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS
ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure

[Install]
WantedBy=multi-user.target
WantedBy=docker.service

Next, modify /etc/sysconfig/flanneld something like the below.

cat /etc/sysconfig/flanneld
# Flanneld configuration options 

# etcd url location.  Point this to the server where etcd runs
#FLANNEL_ETCD_ENDPOINTS="http://127.0.0.1:2379"
FLANNEL_ETCD_ENDPOINTS="https://172.20.0.11:2379,https://172.20.0.12:2379,https://172.20.0.13:2379"

# etcd config key.  This is the configuration key that flannel queries
# For address range assignment
#FLANNEL_ETCD_PREFIX="/atomic.io/network"
FLANNEL_ETCD_PREFIX="/coreos.com/network"

# Any additional options that you want to pass
FLANNEL_OPTIONS="-etcd-cafile=/etc/kubernetes/ssl/ca.pem -etcd-certfile=/etc/kubernetes/ssl/etcd-node.pem -etcd-keyfile=/etc/kubernetes/ssl/etcd-node-key.pem -iface=enp0s3 -public-ip=172.20.0.11 -ip-masq=true"

Flannel CNI configuration

We are now going to add the CNI configuration. First lets Download the latest CNI drivers, you do so by running the below.

mkdir -p /opt/cni/bin && cd /opt/cni/bin
curl -o cni-amd64-v0.6.0.tgz https://github.com/containernetworking/cni/releases/download/v0.6.0/cni-amd64-v0.6.0.tgz
tar zxf cni-amd64-v0.6.0.tgz

Note: You can find the latest CNI releases here. Next, lets create the CNI configuration directory

mkdir -p /etc/kubernetes/cni/net.d /etc/cni
/usr/bin/ln -sf /etc/kubernetes/cni/net.d /etc/cni/net.d

Create the the CNI network configuration file.

cat /etc/kubernetes/cni/net.d/10-containernet.conf
{
    "name": "podnet",
    "type": "flannel",
    "delegate": {
        "forceAddress": true,
        "isDefaultGateway": true,
        "hairpinMode": true
    }
}

We are now ready to start flannel, you do so by running the below.

# Show flanneld log/output
journalctl -u flanneld -f &

# Re-load systemd
systemctl daemon-reload

# Enable the service and start the flanneld service
systemctl enable flanneld && systemctl start flanneld

We are now ready to move on to the docker configuration.

Configuring the docker service(s).

Replace in /usr/lib/systemd/system/docker.service service like the below.

# from
After=network-online.target firewalld.service
# to
After=network-online.target flanneld.service

add
[Service]
Type=notify
EnvironmentFile=-/run/flannel/docker <<<---(without the arrows)
...

Create a docker socket service file.

cat /etc/systemd/system/docker.socket
[Unit]
Description=Docker Socket for the API
PartOf=docker.service

[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker

[Install]
WantedBy=sockets.target

Create a docker network file /etc/docker/daemon.json with the below content (replace with the ip of each node). Note: Example below is taken from master1.

cat /etc/docker/daemon.json
{
  "bip": "172.30.0.11/20"
}

Now lets start the docker service.

systemctl daemon-reload
# Pre docker service start
systemctl enable docker.socket && systemctl start docker.socket

journalctl -u docker -f &
systemctl enable docker && systemctl start docker

In Part 4 will continue configuring the Kubernetes manifest and kubelet service. You might also like – Other related articles to Docker and Kubernetes / micro-service. Like what you’re reading? please provide feedback, any feedback is appreciated.