Ephemeral Kubernetes Clusters With Kind and Make
There are a number of options for persistent local Kubernetes clusters, but when you’re developing tools against the Kubernetes APIs it’s often best to be throwing things away fairly regularly. Enter Kind. Originally designed as a tool for testing Kubernetes itself, Kind runs a working Kubernetes cluster on top of Docker.
At its simplest we can create a new Kubernetes cluster like so:
$ kind create cluster Creating cluster "kind" ... ✓ Ensuring node image (kindest/node:v1.14.2) 🖼 ✓ Preparing nodes 📦 ✓ Creating kubeadm config 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 Cluster creation complete. You can now use the cluster with: export KUBECONFIG="$(kind get kubeconfig-path --name="kind")" kubectl cluster-info
The instructions from running the command show how to connect to the new cluster. Running the Docker commands will show the running container.
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 0e39e49feaed kindest/node:v1.14.2 "/usr/local/bin/entr…" About a minute ago Up About a minute 61227/tcp, 127.0.0.1:61227->6443/tcp kind-control-plan
You can use the Kind CLI tool directly, but it’s also a handy utility for simple automation tasks. The Roadmap promises better support for Kind as a Go library in the future, but for the moment I’ve started using
WAIT := 200s APPLY = kubectl apply --kubeconfig=$$(kind get kubeconfig-path --name $@) --validate=false --filename NAME = $$(echo $@ | cut -d "-" -f 2- | sed "s/%*$$//") tekton tekton%: create-tekton% @$(APPLY) https://storage.googleapis.com/tekton-releases/latest/release.yaml gatekeeper gatekeeper%: create-gatekeeper% @$(APPLY) https://raw.githubusercontent.com/open-policy-agent/gatekeeper/master/deploy/gatekeeper-constraint.yaml create-%: -@kind create cluster --name $(NAME) --wait $(WAIT) delete-%: @kind delete cluster --name $(NAME) env-%: @kind get kubeconfig-path --name $(NAME) clean: @kind get clusters | xargs -L1 -I% kind delete cluster --name % list: @kind get clusters .PHONY: tekton gatekeeper tekton% gatekeeper% create-% delete-% env-% clean list
Makefile provides a few handy shortcuts for Kind commands. Not strictly necessary but useful for consistency. So I can now run commands like:
$ make create-hello # Create a new cluster called hello $ make delete-hello # Delete the hello cluster $ make list # Provide a list of all cluster $ make clean # Delete all of my clusters
More useful are the higher-level commands, in the example above
make tekton and
make gatekeeper. Tekton is a project aiming to provide Kubernetes-style resources for declaring CI/CD pipelines. Gatekeeper provides a policy controller for Kubernetes, using Open Policy Agent. Both projects add new custom resources to Kubernetes. With the
Makefile above I can run one command and spin up an ephemeral
Kubernetes cluster pre-provisioned with Tekton, or Gatekeeper. Because I got carried away the Makefile also supports grabbing multiple Tekton or Gatekeeper clusters with different names, so the following works too:
$ make tekton1 $ make tekton2 $ make tekton-loves-make
I can imagine Kind, or a sub-project, growing a
Kindfile in the future. And an improved CLI which moves beyond the initial cluster testing scenarios it was originally designed for. But for now, and for me at least, Make makes a great prototyping tool. I can grab arbitrary ephemeral Kubernetss clusters until I run out of memory, and adding new projects I’m interested in is trivial. Just as importantly thowing them away is easy too.
Make is a great tool to have in your toolbox in my experience, and learning just enough to be dangerous goes a long way in terms of expressiveness and power. But as mentioned before, I have a bit of a thing for DSLs.