Background

Deploying applications to Kubernetes has been getting better and better. Things like Deployments, Dynamic Volume Provisioning and other features are allowing folks to focus on the apps more-so than the infrastructure which is great.

However, in a development model, you want more access to the cluster to allow for easier debugging and access.

I use Minikube all the time in my day-to-day job to test out new Operators I’m working on or to deploy an application locally to test out some features.

The one component that was always difficult was how to connect to services on Minikube. The only real way today has been to expose the service via a NodePort which works, but it’s difficult if since now there’s a different configuration to deal with. The service type now has to be NodePort when I only want it to be ClusterIP (meaning no outside cluster access). Additionally, that port is dynamic, so now I’ve got to find out the new port each time or set it statically.

Solution

In slack the other day, I saw a post from Dale Hamel (@dalehamel-shopify on Slack), which outlined a way to route traffic from your host machine to the minikube cluster allowing you to curl Kubernetes services via ClusterIP or DNS name.

There are just a few simple commands that you need to do to set this up, and a big thanks to Dale for sharing with everyone!

Implementation

NOTE: These instructions are build for an OSX system, but can be translated to Linux as well pretty easily.

Create a route to allow traffic from the host machine to route to the minikube IP:

# Remove any existing routes
$ sudo route -n delete 10/24 > /dev/null 2>&1

# Create route
$ sudo route -n add 10.0.0.0/24 $(minikube ip)

Create a DNS resolver to allow traffic to use the minikube internal DNS server for resolution:

$ sudo cat <<EOF >/etc/resolver/svc.cluster.local
nameserver 10.0.0.10
domain svc.cluster.local
search svc.cluster.local default.svc.cluster.local
options ndots:5
EOF

Update mac dns resolver, then REBOOT:

sudo defaults write /Library/Preferences/com.apple.mDNSResponder.plist AlwaysAppendSearchDomains -bool YES

Get the list of interfaces:

# xhyve uses "bridge100" / virtualbox uses "bridge0"
$ ifconfig 'bridge100' | grep member | awk '{print $2}'

Open up the firewall to allow access:

# Take all interfaces from the previous command and apply
$ sudo ifconfig bridge100 -hostfilter en5

Test it out!

# --- Delete current route:
$ sudo route -n delete 10/24 > /dev/null 2>&1                                 
Password:

# --- Add new route:
$ sudo route -n add 10.0.0.0/24 $(minikube ip)
add net 10.0.0.0: gateway 192.168.64.30

# --- Get interfaces:
$ ifconfig 'bridge100' | grep member | awk '{print $2}'
en5

# --- Set firewall:
$ sudo ifconfig bridge100 -hostfilter en5

# --- Get services:
$ kubectl get svc
NAME                      CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
elasticsearch             10.0.0.181   <none>        9200/TCP   2m
elasticsearch-discovery   10.0.0.88    <none>        9300/TCP   2m
es-data-svc               10.0.0.55    <none>        9300/TCP   2m
kubernetes                10.0.0.1     <none>        443/TCP    8m

# --- Curl service by IP:
$ curl -k https://10.0.0.181:9200
{
    "name" : "71b5524c-524c-4f4c-9621-e45c5c34f22c",
    "cluster_name" : "myesdb",
    "cluster_uuid" : "Y0D14nKKRTuPH8x2Yq3kuQ",
    "version" : {
    "number" : "5.3.1",
    "build_hash" : "5f9cf58",
    "build_date" : "2017-04-17T15:52:53.846Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.2"
},
"tagline" : "You Know, for Search"
}

# --- Curl service by DNS:
$ curl -k https://elasticsearch:9200
{
    "name" : "71b5524c-524c-4f4c-9621-e45c5c34f22c",
    "cluster_name" : "myesdb",
    "cluster_uuid" : "Y0D14nKKRTuPH8x2Yq3kuQ",
    "version" : {
    "number" : "5.3.1",
    "build_hash" : "5f9cf58",
    "build_date" : "2017-04-17T15:52:53.846Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.2"
},
"tagline" : "You Know, for Search"
}