Deploying Helm Charts / Tiller on your kubernetes cluster

Below is a continuation to my previous post(S) part 1-7 on how to configure Kubernetes 3 Master Node cluster. In the post below I am going to show you.

1. How to install / configure – Helm / Tiller on your kubernetes cluster.
2. How to install / configure – Prometheus / AlertManager, Grafana, Elasticsearch on your kubernetes cluster.

Please check out the full series to see how to configure a 3 node Kubernetes master, the links are below.

• Part 1: Initial setup – bear-metal installation, configuration
• Part 2: Installing the Kubernetes VM’s
• Part 3: Installing and configuring Flanneld, CNI plugin and Docker
• Part 4: Installing and configuring kubernetes manifest and kubelet service
• Part 5: Adding CoreDNS as part of the Kubernetes cluster
• Part 6: Adding / Configuring Kubernetes worker nodes
• Part 7: Enabling / Configuring RBAC, TLS Node bootstrapping
• Part 8: Installing / Configuring Helm, Prometheus, Alertmanager, Grafana, Elasticsearch

This is Part 8 – Deploying Helm Tiller, Prometheus, AlertManager, Grafana, Elasticsearch. First we are going to install the helm client, this will help with the Tiller (Helm server) install. Note: Tiller v2.10 has removed checking the environment for an http_proxy/https_proxy, therefore causing many issues behind corporate proxies. The issue is being addressed in a future updated(soon). For the post below I am therefore using Tiller v2.9.1 – the latest stable release prior to v2.10.0. Lets begin with downloading the Helm client. Running the below will download the latest stable Helm client (v.2.10.0).

curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh

Since I will not be using v2.10.0 (because of the proxy bug), I will be downloading a specific version v2.9.1 below. Note: You can get a list of releases here.

curl -o helm-v2.9.1-linux-amd64.tar.gz https://storage.googleapis.com/kubernetes-helm/helm-v2.9.1-linux-amd64.tar.gz
tar zxf helm-v2.9.1-linux-amd64.tar.gz && cp linux-amd64/helm /usr/local/bin/helm
chmod +x /usr/local/bin/helm

Tip: The Helm server is called referred to as tiller. Next, we are going to create a kubernetes tiller(Helm server) service account. this account will be used by the helm/tiller server. Create tiller serviceAccount by running the below.

kubectl create serviceaccount tiller --namespace kube-system

# Assign / grand access to the tiller account (the tiller account requires full cluster access being a pkg manager).
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller

Securing the Helm / Tiller server

Since the Helm / Tiller server has full access to the kubernetes cluster, its strongly recommended to secure tiller access. We are therefore going to configure/use RBAC and TLS/SSL access to strengthen security. Note: By default tiller will have no security constrain, meaning anyone with cluster access would be able to do anything by using the tiller account. Step one will be, generating the tiller server and helm/client certificates. To generate certificates you will need a CA. I will be using the kubernetes CA (the same CA used for all other kubernetes components), feel free to generate another CA and use that instead(I will highlight the steps below). Creating your on CA for tiller (this is only need if not using the kubernetes CA otherwise skip to step 2).

1. Generating certificate CA keys.

   openssl genrsa -out ./ca-key.pem 2048
   openssl req -key ca-key.pem -new -x509 -days 7300 -sha256 -out ca.pem -extensions v3_ca -subj '/C=US/ST=New York/L=New York/O=example.com/CN=tiller-ca'
   

2. Generating tiller keys.

   # Create tiller server key
   openssl genrsa -out ./tiller-key.pem 2048
   # Create tiller client key
   openssl genrsa -out ./helm-key.pem 2048
   

3. Create certificate requests.

   # Create tiller server certificate request
   openssl req -key tiller-key.pem -new -sha256 -out tiller-csr.pem -subj '/C=US/ST=New York/L=New York/O=example.com/CN=tiller-server'
   # Create tiller client certificate request
   openssl req -key helm-key.pem -new -sha256 -out helm-csr.pem -subj '/C=US/ST=New York/L=New York/O=example.com/CN=tiller'
   

4. Sign the certificate request.

   # Sign tiller server certificate
   openssl x509 -req -CA /etc/kubernetes/ssl/ca.pem -CAkey /etc/kubernetes/ssl/ca-key.pem -CAcreateserial -in tiller-csr.pem -out tiller-cert.pem -days 7300
   # Sign tiller client certificate
   openssl x509 -req -CA /etc/kubernetes/ssl/ca.pem -CAkey /etc/kubernetes/ssl/ca-key.pem -CAcreateserial -in helm-csr.pem -out helm-cert.pem  -days 7300
   

5. If using a separate CA for tiller – use the below instead.

   # Sign tiller server certificate
   openssl x509 -req -CA ca.pem -CAkey ca-key.pem -CAcreateserial -in tiller-csr.pem -out tiller-cert.pem -days 7300
   # Sign tiller client certificate
   openssl x509 -req -CA ca.pem -CAkey ca-key.pem -CAcreateserial -in helm-csr.pem -out helm-cert.pem  -days 7300
   

We are now ready to Initialize the tiller server with the newly created tls keys. Note: Export your proxy if your behind a firewall or proxy, by running the below.

export https_proxy=http://10.10.10.10:8080/
export http_proxy=http://10.10.10.10:8080/

Initialize tiller server by running the below.

helm init --debug --tiller-tls --tiller-tls-cert ./tiller-cert.pem --tiller-tls-key ./tiller-key.pem --tls-ca-cert /etc/kubernetes/ssl/ca.pem

If using a seprate CA for tiller – use the below insted.

helm init --debug --tiller-tls --tiller-tls-cert ./tiller-cert.pem --tiller-tls-key ./tiller-key.pem --tls-ca-cert ./ca.pem

Run to below to allow the tiller service account (RBAC) access.

kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'

Finally, lets test Tiller TLS access by using the tiller service account with TLS(RBAC). the below should return no errors i.e. empty.

helm ls --tls --tls-ca-cert /etc/kubernetes/ssl/ca.pem --tls-cert helm-cert.pem --tls-key helm-key.pem

If using a separate CA for tiller – use the below instead.

helm ls --tls --tls-ca-cert ca.pem --tls-cert helm-cert.pem --tls-key helm-key.pem

You can permanent add the certificates to your $HELM_HOME to make life simpler, by running the below.

export HELM_HOME=/root/.helm
cp /etc/kubernetes/ssl/ca.pem $(helm home)/ca.pem
cp helm-cert.pem $(helm home)/cert.pem
cp helm-key.pem $(helm home)/key.pem
unset HELM_HOME

And test by not specifying certificates.

helm ls --tls

Tiller common uses – help

# List pkgs
helm list

# Search pkgs
helm search prometheus
helm search prometheus
NAME                                 	CHART VERSION	APP VERSION	DESCRIPTION                                       
stable/prometheus                    	7.0.2        	2.3.2      	Prometheus is a monitoring system and time seri...
stable/prometheus-adapter            	v0.1.0       	v0.2.1     	A Helm chart for k8s prometheus adapter           
stable/prometheus-blackbox-exporter  	0.1.1        	0.12.0     	Prometheus Blackbox Exporter                      
stable/prometheus-cloudwatch-exporter	0.1.4        	0.1.0      	A Helm chart for prometheus cloudwatch-exporter   
... [snip]

To upgrade tiller just run the below.

helm init --upgrade --service-account tiller

See Part 2 Deploying Prometheus, AlertManager, Grafana, Elasticsearch by clicking here. You might also like – Other related articles to Docker and Kubernetes / micro-service. Like what you’re reading? please provide feedback, any feedback is appreciated.