If you've followed me on Twitter or come to any of my talks at CodeMash, Pittsburgh Tech Fest, or coming up at Abstractions (Here and Here), you know that I'm into Kubernetes and deploying it out to a Raspberry Pi Cluster.
In prep for my talks coming up at Abstractions, I wanted to make the demo's a little bit more realistic by using real dns names for the demos instead of using
NodePorts (because who is going to access a site on the public internet like this:
If you're not familiar with Kubernetes,
NodePorts are a simple way to expose containers running in a cluster to the outside world. You access one the nodes in the cluster via this port and magically you are routed to the container. You can read more about NodePorts here.
I tried a few solutions to get this dns routing to work.
First I tried to implement Ingress routing. This is a k8s native solution which lets you route traffic to the cluster, and a router inside the cluster then looks at the request and sends you to the appropriate container inside the cluster.
I struggled to get ingress working in Minikube on my Mac laptop. There was some errors with my certs and the service account. I then tried to deploy to my AWS cluster running on CoreOS. This time I got past the cert errors, but the Nginx controller wouldn't pick up the services.
Being a big fan of the work the Deis team has been doing. They have built / are building some really cool things and I encourage everyone to check it out.
They make a router which works similarly to Ingress but instead of deploying
Ingress resources to your cluster, you just attach labels to the services.
You just need to deploy the router, add a label (
router.deis.io/routable: "true") and an annotation (
router.deis.io/domains: foo,bar,www.foobar.com) to a service and that's it!
Compiling for ARM
I love me some raspberry pi's and think it's totally cool that we can run kubernetes (and lots of other cool stuff) on small, cheap hardware. I think building out a k8s cluster by hand is a great learning path to understand how kubernetes works and how to troubleshoot (because stuff will ultimately go wrong).
Since the Deis router isn't compiled for ARM, that's up to us to do.
Build the Router Binary
- On my mac I cloned the Deis Router repo down:
git clone https://github.com/deis/router.git
make boostrapto get the environment running
- Edit the make file and change
- Build the router binary for arm:
- Copy that file over to your raspberry pi
- Clone the router repo onto your pi and move the
Build the helper containers:
The Deis team uses some base images they built as a common starting point. First, I had to update the base image (
quay.io/deis/base:0.2.0). I did this by changing the root image to be
hypriot/rpi-golang which turned into:
Then just update the base image in
rootfs/Dockerfile to be the base image we created in the previous step.
My final image is here:
Here's my replication controller which I used to deploy the router to my pi:
apiVersion: v1 kind: ReplicationController metadata: name: deis-router namespace: deis labels: heritage: deis spec: replicas: 1 selector: app: deis-router template: metadata: labels: app: deis-router spec: containers: - name: deis-router image: stevesloka/deis-router-arm:0.1.0 imagePullPolicy: Always env: - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - containerPort: 8080 hostPort: 80 - containerPort: 6443 hostPort: 443 - containerPort: 2222 hostPort: 2222 - containerPort: 9090 hostPort: 9090 livenessProbe: httpGet: path: /healthz port: 9090 initialDelaySeconds: 1 timeoutSeconds: 1 readinessProbe: httpGet: path: /healthz port: 9090 initialDelaySeconds: 1 timeoutSeconds: 1
Hope this helps anyone run this on their pi. I probably need to come up with a more automated way to build these images, but for now it should work out fine.
I was pinged on Twitter someone does have Ingress working on ARM here, but haven't yet taken a look yet, but will soon.