If you’re not familiar, I work on an Ingress Controller for Kubernetes called Contour (projectcontour.io. Contour uses Envoy Proxy as its data path component, meaning, Envoy handles all the network traffic for a cluster and Contour serves as the control plane or xDS server for Envoy.

I’ve used RaspberryPi’s in the past for demo’s of Kubernetes. It’s a great way to introduce how Deployments work as well as networking when you can physically unplug cables in a live demo. The problem that always arises is having a proper image to run on the arm architecture.

To note as well, I’m not the first to go down this path. There’s been an open issue in the Envoy repo for some time regarding this same problem. Also, there will be builds from the Envoy repo on its next release, but it’s always fun do try it yourself right? =)

Shout out!

I also wanted to give a shout-out to @moderation who has been helpful with tips/tricks on putting these steps together!

Setup your RaspberryPi

There are a few steps to get up and running on your Pi. I’m using a Raspberry Pi 4 with 4gb of ram. For simplicity, we’ll use Ubuntu as the OS for this build since the envoy repo has the best support for that outside of the Docker based builds.

Go ahead and download Ubuntu Server 18.04 and flash that to your SD Card. I used Etcher from balena.

Install Deps

There are a bunch of dependencies that are required. The Envoy Developer Guide has a good run-through of what’s required to be installed, here’s a quick start of what I did to get rolling:

Get Ubuntu Deps

$ sudo apt-get install \
libtool \
cmake \
automake \
autoconf \
make \
ninja-build \
curl \
unzip \
zip \
virtualenv \
build-essential \
openjdk-8-jdk

Install Go + Deps

# Download Go
$ wget https://dl.google.com/go/go1.14.2.linux-arm64.tar.gz

# Extract go to local system
$ sudo tar -C /usr/local -xzf go1.14.2.linux-arm64.tar.gz

# Add to $PATH
$ export PATH=$PATH:/usr/local/go/bin

# Get golang deps
$ go get -u github.com/bazelbuild/buildtools/buildifier
$ go get -u github.com/bazelbuild/buildtools/buildozer

Compile Bazel

Next we need bazel to compile Envoy. Since there isn’t an arm version available, we’ll compile it ourselves.

## Create a bazel directory
$ mkdir bazel && cd bazel

## Get the distribution
$ wget https://github.com/bazelbuild/bazel/releases/download/3.1.0/bazel-3.1.0-dist.zip

## Unzip
$ unzip bazel-3.1.0-dist.zip

# Build it!
$ env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh

Build Envoy

Finally, we can get to the main show, compiling Envoy!

The first thing we’ll do is clone the main Envoy repo. Next we’ll clone a repo of mine which disables a bunch of extensions that Contour doesn’t require. This step isn’t fully required (since you might want some of the extensions I’ve disabled), however, I did have issues compiling some in the the pi.

First let’s clone the repo

$ git clone https://github.com/envoyproxy/envoy.git
$ git clone https://github.com/stevesloka/envoy-custom-extensions.git

## Move to the directory
$ cd envoy

## Build Envoy
$ bazel build --jobs=3 -c opt //source/exe:envoy-static \
  --override_repository=respository-name${PATH_TO_CLONED_REPO}/envoy_custom/

Docker Image

Now that we’ve got the binary built, let’s make a Docker image out of it, so we can deploy inside our cluster. After that we’ll build Contour for arm as well.

Envoy

The default Dockerfile which ships in the Envoy repo needs the Envoy binary we just built to be in a specific location. We’ll set that up and then build the image:

### Created a directory for the binary
$ mkdir -p build_release_stripped

### Copy the binary to our new directory

$ cp bazel-bin/source/exe/envoy-static build_release_stripped/envoy

### Build the image
$ docker build -f ci/Dockerfile-envoy -t stevesloka/envoy-contour-arm .

Contour

## Clone the repo
$ git clone https://github.com/projectcontour/contour.git

## Move to the directory
$ GO_ARCH=arm64 REGISTRY=stevesloka VERSION=arm make container push

Deploy!

Now that we have our images we can deploy them to our cluster. Just update your image tags for Contour & Envoy to match the one’s you built.

In my examples above they are:

stevesloka/envoy-contour-arm:latest
stevesloka/contour:arm