Kubernetes Cluster Federation, A.K.A KubeFed. When can it be useful? and 3 things to be aware of.

Disclaimer: This post does not intend to discuss WHAT K8 Federation is, rather attempts to answer the question of When Federation can be useful.

So, lately I have came across an interesting topic in K8 which is called Cluster Federation A.K.A KubeFed. In it's core Cluster Federation enables Coordination and Configuration of two or more independent cluster. In another words you can combine the power of two or more clusters and let them work in Parallel for running a common task. In this model, one Cluster will act as the coordinator server also known as Host Cluster, and manages the configuration of several independent cluster under it's hook. 

To understand it's concept better, I followed this awesome walkthrough I found on internet, and then I created 3 Clusters, one running on Azure (AKS), another running on AWS (EKS), and third one running on GCloud (GKE). In my setup I choose the GKE cluster as the HOST cluster, and registered the other two cluster underneath of it - if you follow the walk-through link you will know how to do so.

Afterwards, I created a namespace in the HOST cluster, and it immediately propagated in the other two clusters which resulted the other two clusters to create that namespace, I then deployed a POD and a Service in the HOST cluster, and same magic happened. I could see the POD and Service has been propagated in the underlying clusters.

When to use Federation? A.KA the million dollar question...

1. After running this experiment and did more research on it, I asked myself, so when exactly do we turn to Federation? From my perspective, federation can be helpful when you really really need to operate a mission critical, highly available workload. In another words, Federation can be helpful when Availability is your top priority, and your current provider does not meet the SLA you are targeting to achieve. For example Azure AKS offers an SLA of  99.9 % which means about 9 hours down-time per year. If you need SLA higher than what your Cloud provider offers you, then you can provision additional cluster in a different region of the same provider, or even go outside and use a different cloud provider and join the clusters using federation. Personally, I have never had a chance so far to work in such environment where Availability is so critical that the existing cloud provider SLA is insufficient. 
2. Another reason, you might want to turn into federation is to avoid vendor-lock in. I myself don't believe in the idea of "vendor lock-in" but that is a whole different topic that I don't want to discuss in this post. if Vendor lock-in is your concern, you can run run different K8 cluster across different providers and your On-Prem infrastructure and have them run the same workload for you using Federation. In this manner you can switch-off the cluster that is running on the provider you want to bid a farewell with, without causing any operational disruption in your workload.
3. You might want to Use Federation when sub-millisecond latency matters. So, let's say you have a global application that servers million of request per-day from different geo location. Federation can help you to run multiple cluster across different geo-location and have them serve the load according to the origin of the request by setting up a Load Balancer that distribute the load accordingly. Having said that these days, all the cloud providers such as AWS, Azure, and GCP support multi region K8 setup, that does almost the same thing, but there is a caveat to it: if you use multi region K8 setup that is offered by your Cloud provider you need to operate them separately. for example if you update a Service or POD in Region U.S, your E.U region will not be updated magically, unless you do it by yourself or update your CI | CD pipeline to do it for you, while in Federation case, as soon you ask the HOST cluster to do something, it will immediately propagate that operation to the underlying clusters.

3 things to be Aware of when deciding to go with Federation:

• Federation is Complicated: if you are not using K8 managed services offered by cloud provider, you know how complex it is to operate and manage a K8 cluster. Now, Federating takes that complexity to a whole new level. Even if you have all your clusters as managed cluster running in the cloud, using Federation is complex since Cloud providers do not support Federation natively.
• KubeFed is in Beta: As of today, according to it's official github page, KubeFed is in Alpha stage and it's going toward the Beta version very quickly. So be sure to think twice before introducing KubeFed into your Production workflow.

• The HOST cluster is only for manipulating underlying clusters. when using Federation, you can ask the Host Cluster to Create, Update, or Delete resource in the underlying clusters, but you can NOT use the HOST cluster to Query the underlying clusters. For example, you can use the Host Cluster to Create or Update a POD using "kubectl apply -f *.yaml" command  for you, but you can't use the Host Cluster to run "kubectl get pod" command. The good news is, you can take advantage of Bash for loop to retrieve all the PODS across all clusters. For example, the following command returns all the pods in all my clusters: 

> for c in `kubectl config get-contexts --no-headers=true -o name|grep -v k8s-cluster-kubefed `
do
echo "Getting pods in context $c"
kubectl get pods -n kubefed-poc --context=$c
done