February 11, 2022
This post is a work in progress and hasn't been published yet.
This article is the fourth part of the A Vercel-like PaaS beyond Jamstack with Kubernetes and GitOps series.
Previously, I've modeled how traffic flows from client to application with the following diagram, in this part I'm setting up components 4, 5 and 6:
✓ 1.client DNS, 443/TCP - part I↓✓ 2.host k0s installed - part I↓✓ 3.ingress ingress-nginx installed - part I↓4.service↓5.pod↓6.container↓✓ 7.application Dockerfile - part III
The Kubernetes documentation illustrates this flow in a similar way, and these components sit in the grey rectangle with the cluster label in the following picture:
Traffic flow according to Kubernetes documentation
Instead, I'm doing a simple find and replace with
__TOKEN__ strings in manifests files,
I've explained this task in part II.
An Ingress is not a component in itself, in the sense that it doesn't translates to a pod deployed to the cluster, with its own IP address on the cluster's internal network.
It's a routing rule that instructs the cluster's ingress controller to forward incoming connections to a certain Service according to a rule. Next section will explain what a Service is.
In my case, since I've installed ingress-nginx, this new rule will tell to the
ingress-nginx-162... pod I was talking about in part I,
to forward traffic received form the host to the Service I'll defined in the
apiVersion: networking.k8s.io/v1kind: Ingress---spec:ingressClassName: nginx <-- when this ingress-controller receives requestsrules:- host: __KUBE_INGRESS_HOST__ <-- route requests to this domain namehttp:paths:- path: / <-- regardless of the requested pathbackend:service:name: api <-- to this service (in the same namespace)port:number: 3000 <-- using this TCP port
So, adding an Ingress means , what does On the other hand, the
ingress-nginx controller transletes to a pod with its own
IP address, running
$ kubectl describe pod -A -l app.kubernetes.io/name=ingress-nginxName: ingress-nginx-1644337728-controller-5cdf6b55b9-55qj4...Status: RunningIP: 172.31.12.80Containers:controller:Ports: 80/TCP, 443/TCP, 8443/TCPHost Ports: 80/TCP, 443/TCP, 8443/TCP
A simple way to see traffic being forwarded from this pod to a service is to show this pod's
log stream while querying a deployed application. Here is the pod's output while running
curl command form another terminal:
# run this command in one terminal:$ kubectl logs -f $(kubectl get pod -A \-l app.kubernetes.io/name=ingress-nginx -o name)---# run this command in a second terminal:$ curl https://7c77eb36.nodejs.k0s.gaudi.s---# then the first terminal shows this new line:184.108.40.206 - - [07/Mar/2022:20:00:10 +0000] "GET / HTTP/2.0" 200 15 "-" "curl/7.68.0" 34 0.001 [nodejs-7c77eb36-api-3000]  10.244.0.24:3000 25 0.004 200 ee266baf44f3cfd0838e28fa1ff79785
This log line shows the
curl/7.68.0 client request to be forwared to
Since the URL I'm requesting points to
api service, on port
3000 and deployed in the
which does exist:
$ kubectl get service api -n nodejs-7c77eb36 \-o custom-columns=NAMESPACE:metadata.namespace,SERVICE:metadata.name,PORT:spec.ports[*].targetPortNAMESPACE SERVICE PORTnodejs-7c77eb36 api 3000
On the other hand, a Service is an abstract component that has an IP address and act as a load balancer in front of a pool of pods.
Because Pods can be created and deleted at will, the Service act as a single immutable endpoint,
meaning it has a fixed IP address. Then traffic is balanced between members of a Pod pool.
file defines a Service resource that
To enter the pool, pods must use a specific selector as it's declared in the application's'
apiVersion: v1kind: Service---spec:type: ClusterIPports:- name: httpport: 80 <-- service listens to this porttargetPort: 3000 <-- and routes traffic to this pod's portprotocol: TCPselector:app: api <-- pods must use this value to enter the pool
Pods are defined in the
apiVersion: apps/v1kind: Deployment
I set how many pods I want to start:
spec:replicas: 1 <-- the size of the pool
Which service pods belongs to:
selector:matchLabels:app: api <-- selector must match service
The credentials to pull the Docker image from the GitLab Container Registry:
spec:imagePullSecrets:- name: gitlab-registry
I need only one container running the image built in section 1:
containers:- name: nodeimage: __CI_REGISTRY_IMAGE__:__CI_COMMIT_SHORT_SHA__
I don't want to lock any host's resources for these pods. This is where I save a lot compared to other managed PaaS because I can wisely overcommit resources:
resources:requests:cpu: '0'memory: '0'
I don't want to allow a pod to eat all the host's resources so I limit how much a pod can use 2:
limits:cpu: '0.125'memory: 64M
And I expose the correct port:
ports:- containerPort: 3000
It is critical to use the port the application is listening to in deployment.yml configuration (containerPort) and therefore in the service.yml (targetPort) and ingress.yml configuration (service.port.number).
In every application, I'm using the
COMMIT_SHORT_HASH variable (exists only in during pipeline runtime) to assign a value to the
COMMIT environmental variable (stored in the Docker image). Then, I can read this value from code:
1 Be aware that if you want to overcommit resources, as thedocumentation notes: if a container specifies its own memory limit, but does not specify a memory request, Kubernetes automatically assigns a memory request that matches the limit. ↑
2 Remember that Kubernetes doesn't gently refuse access to more resource, it usually terminates(kills) hungry pods. A good reason not to run shell commands, such as migration or cron jobs, in running pods that serves web trafic. ↑