Empowering Kubernetes Operator Development with Kopf: Deploying to Kubernetes EKS(Part-3)

In my previous article on Getting Started with Writing Operator, I built a simple operator which launched a Kubernetes Deployment and exposed it via Kubernetes Service for the custom resource Microservice.

In this part of the article, I’ll be moving on from development mode to deployment, and will be running the operator as a Kubernetes Deployment in the Kubernetes itself. I’ll be using AWS EKS as the Kubernetes environment.

Kubernetes Operator operates in the same method irrespective of the Kubernetes Environment.

The following steps will be involved to deploy the operator to Kubernetes:

1. Adding authentication logic for Operator
2. Creating Dockerfile for the Operator.
3. Building and pushing the image to Docker Hub.
4. Launching an EKS Cluster.
5. Creating necessary resources, CRD and the Operator.
6. Deploying Microservice to test the operator.

Authentication Logic for Operator

@kopf.on.login()
def custom_login_fn(**kwargs):
    if EnvConfig.ENV == "dev":
        return kopf.login_with_kubeconfig(**kwargs)
    else:
        return kopf.login_with_service_account(**kwargs)

I’ve modified the code and folder structures to make it more readable, you may find the code here: Microservice Operator.

Every time the operator starts, it performs an initial authentication process with the Kubernetes API server, to determine the whereabouts of the Kubernetes to be used. This above function overrides the default authentication flow, which in my case is environment driven.

Depending upon the value passed for the environment variable, operator will be able to authenticate from either using kubeconfig (for development purpose) or from the service account (for production).

Dockerfile for Operator

The following Dockerfile is used for creating the image of the operator.

FROM python:3.11-slim-buster

WORKDIR /app

COPY requirements.txt .

RUN pip install --no-cache-dir -r requirements.txt

COPY . .

ENTRYPOINT ["kopf", "run", "main.py"]

The requiremets.txt file contains the python packages used for the operator.

Building and Pushing Image

I’ve used my personal docker image registry to push the image.

Command to build the image:

docker build -t sayedimran/microservice-operator:v1.0.0 .

Image Build Output

Command to push the image to the registry:

docker push sayedimran/microservice-operator:v1.0.0

Pushing image to the registry

Launching an EKS Cluster

I’ve created an EKS cluster with all the default options as shown in the below image.

Default EKS configuration

Created a Node Group with all default options with Amazon Linux 2 Image with 2 nodes of type t2.medium and 20 GiB of storage for each:

Default Node Group Configuration

Once the cluster is ready, to access the EKS via kubectl, I’ll be the using the aws CLI in AWS CloudShell.

Command to generate kubeconfig:

aws eks --region ap-south-1 update-kubeconfig --name aws-cluster-1

Generating Kubeconfig for the created cluster

Checking for the kube-system namespace pods

Pods of kube-system

Cluster is now ready for the operator to be deployed.

Creating resources & CRD for Operator

As of now the operator has been bundled as image and pushed to the registry which now can be pulled into the Kubernetes when deployed.

Before deploying the operator, the following resources are the pre-requisites for the operator to function in the cluster:

• ClusterRole
• ClusterRoleBinding
• Role
• RoleBinding
• ServiceAccount

All the above resources are basically the RBAC permissions to be provided to the operator so as to provision resources on launch of the custom resources.

Link of the CRD, resources & operator deployment can be found here:

• Microservice CRD [crd.yaml]
• Microservice Operator Resources and Deployment [deploy.yaml]

Command to deploy the CRD, resources and operator:

kubectl apply -f https://raw.githubusercontent.com/sayed-imran/microservice-operator/v1.0.0/crd.yml
kubectl apply -f https://raw.githubusercontent.com/sayed-imran/microservice-operator/v1.0.0/deploy.yml

CRD, resources & Operator deployed

Testing the Microservice Resource

Once the pod for operator goes to running state, we can deploy our custom resource i.e. Microservice.

Deploying a sample microservice application, link for the same can be found here: [sample-microservice.yaml]

Command to deploy the custom resource:

kubectl apply -f https://raw.githubusercontent.com/sayed-imran/microservice-operator/v1.0.0/cr.yml

Deploying the custom resouce microservice

As we can see the resource has been deployed successfully, we can also check for the child resources i.e., pods of the deployment and the service.

Child resources for the custom resource

This marks the completion of the deployment of the Kubernetes Operator in the EKS.

Looking forward to add further more advanced logic to the Operator, do let me know if you want me to add anything relevant and please do give your feedbacks in the comment, would love to hear from you.

#kubernetes #operator #aws #eks #automation #python #kopf #docker #devops #cloud #containerization #sdk #python #go #community #learning #dev