Services and Ingress

Services are the k8s resource one uses to expose HTTP APIs, databases and other components that communicate on a network to other k8s pods and, ultimately, to the outside world. After working through this module, students should be able to:

  • Set up port forwarding from pods on a Kubernetes cluster

  • Use a debug deployment to test access

  • Create and attach a service to a deployment

  • Use a NodePort service to expose a FastAPI application on the public internet

  • Use an Ingress object to map a NodePort port to a subdomain on the host

k8s Networking Overview

To understand services we need to first discuss how networking works in k8s.

Note

We will be covering just the basics of k8s networking, enough for you to become proficient with the main concepts involved in deploying your application. Many details and advanced concepts will be omitted.

k8s creates internal networks and attaches pod containers to them to facilitate communication between pods. For a number of reasons, including security, these networks are not reachable from outside k8s.

Recall that we can learn the private network IP address for a specific pod with the following command:

[coe332-vm]$ kubectl get pods <pod_name> -o wide

For example:

[coe332-vm]$ kubectl get pods hello-deployment-9794b4889-mk6qw -o wide
NAME                                    READY   STATUS        RESTARTS       AGE   IP              NODE            NOMINATED NODE   READINESS GATES
hello-deployment-6949f8ddbc-znx75       1/1     Running       21 (31m ago)   21h   10.233.116.45   kube-worker-1   <none>           <none>

This tells us k8s assigned an IP address of 10.233.116.45 to our hello-deployment pod. IP addresses starting with 10. are private IP addresses which are part of a private network, isolated to this k8s cluster. k8s assigns every pod on this private network an IP address. Pods that are on the same network can communicate with other pods using their IP address.

Ports

To communicate with a program running on a network, we use ports. We saw how our FastAPI program used port 8000 to communicate HTTP requests from clients. We can expose ports in our k8s deployments by defining a ports stanza in our template.spec.containers object. Let’s try that now.

Create a file called deployment-hello-fastapi.yml and copy the following contents

 1---
 2apiVersion: apps/v1
 3kind: Deployment
 4metadata:
 5  name: hello-fastapi
 6  labels:
 7    app: hello-fastapi
 8spec:
 9  replicas: 1
10  selector:
11    matchLabels:
12      app: hello-fastapi
13  template:
14    metadata:
15      labels:
16        app: hello-fastapi
17    spec:
18      containers:
19        - name: hello-fastapi
20          imagePullPolicy: Always
21          image: nathandf/hello-fastapi:1.0
22          ports:
23          - name: http
24            containerPort: 8000

Much of this will look familiar. We are creating a deployment that matches the pod description given in the template.spec stanza. The pod description uses an image, nathandf/hello-fastapi:1.0. This image runs a very simple FastAPI server that responds with simple text message at the / endpoint.

The ports attribute is a list of k8s port descriptions. Each port in the list includes:

  • name – the name of the port, in this case, http. This could be anything we want really.

  • containerPort – the port inside the container to expose, in this case 8000. This needs to match the port that the containerized program (in this case, FastAPI server) is binding to.

Next create the hello-fastapi deployment using kubectl apply

[coe332-vm]$ kubectl apply -f deployment-hello-fastapi.yml
deployment.apps/deployment-hello-fastapi configured

With our deployment created, we should see a new pod.

EXERCISE

Determine the IP address of the new pod for the deployment-hello-fastapi.

SOLUTION

 [coe332-vm]$ kubectl get pods
 NAME                                READY   STATUS    RESTARTS       AGE
 hello-deployment-6949f8ddbc-znx75   1/1     Running   21 (36m ago)   21h
 hello-label                         1/1     Running   21 (57m ago)   21h
 hello-label2                        1/1     Running   21             21h
 hello-fastapi-7bf64cc577-l7f52        1/1     Running   0              2m34s


 [coe332-vm]$ kubectl get pods hello-fastapi-7bf64cc577-l7f52 -o wide
 NAME                          READY   STATUS    RESTARTS   AGE     IP              NODE            NOMINATED NODE   READINESS GATES
 hello-fastapi-7bf64cc577-l7f52  1/1     Running   0          3m46s   10.233.116.59   kube-worker-1   <none>           <none>


# Therefore, the IP address is 10.233.116.59

We found the IP address for our FastAPI container, but if we try to communicate with it from our Jetstream VMs, we will either find that it hangs indefinitely or possible gives an error:

[coe332-vm]$ curl 10.233.116.59:8000/
curl: (7) Failed connect to 10.233.116.59:8000; Network is unreachable

This is because the 10.233.*.* private k8s network is not available from the outside. However, it is available from other pods in the namespace.

A Debug Deployment

For exploring and debugging k8s deployments, it can be helpful to have a basic container on the network. We can create a deployment for this purpose.

For example, let’s create a deployment using the official Python 3.10 image. We can run a sleep command inside the container as the primary command, and then, once the container pod is running, we can use exec to launch a shell inside the container.

EXERCISE

Create a new “debug” deployment using the following definition:

 1---
 2apiVersion: apps/v1
 3kind: Deployment
 4metadata:
 5  name: py-debug
 6  labels:
 7    app: py-debug
 8spec:
 9  replicas: 1
10  selector:
11    matchLabels:
12      app: py-debug
13  template:
14    metadata:
15      labels:
16        app: py-debug
17    spec:
18      containers:
19        - name: py-debug
20          image: python:3.10
21          command: ['sleep', '999999999']

Once it is ready, exec into the running pod for this deployment. Once we have a shell running inside our debug deployment pod, we can try to access our FastAPI server. Recall that the IP and port for the FastAPI server were determined above to be 10.244.7.95:8000 (yours will be different).

If we try to access it using curl from within the debug container, we get:

root@py-debug-deployment-5cc8cdd65f-xzhzq: $ curl 10.233.116.59:8000
Hello, world!

Great! k8s networking from within the private network is working as expected!

Services

We saw above how pods can use the IP address of other pods to communicate. However, that is not a great solution because we know the pods making up a deployment come and go. Each time a pod is destroyed and a new one created it gets a new IP address. Moreover, we can scale the number of replica pods for a deployment up and down to handle more or less load.

How would an application that needs to communicate with a pod know which IP address to use? If there are 3 pods comprising a deployment, which one should it use? This problem is referred to as the service discovery problem in distributed systems, and k8s has a solution for it.. the Service abstraction.

A k8s service provides an abstract way of exposing an application running as a collection of pods on a single IP address and port. Let’s define a service for our hello-fastapi deployment.

Copy and paste the following code into a file called service-hello-fastapi.yml:

 1---
 2apiVersion: v1
 3kind: Service
 4metadata:
 5  name: hello-fastapi-service
 6spec:
 7  type: ClusterIP
 8  selector:
 9    app: hello-fastapi
10  ports:
11  - name: hello-fastapi
12    port: 8000
13    targetPort: 8000

Let’s look at the spec description for this service.

  • type – There are different types of k8s services. Here we are creating a ClusterIP service. This creates an IP address on the private k8s network for the service. We may see other types of k8s services later.

  • selector – This tells k8s what pod containers to match for the service. Here we are using a label, app: hello-fastapi, which means k8s will link all pods with this label to our service. Note that it is important that this label match the label applied to our pods in the deployment, so that k8s links the service up to the correct pods.

  • ports - This is a list of ports to expose in the service.

  • ports.port – This is the port to expose on the service’s IP. This is the port clients will use when communicating via the service’s IP address.

  • ports.targetPort – This is the port on the pods to target. This needs to match the port specified in the pod description (and the port the containerized program is binding to).

We create this service using the kubectl apply command, as usual:

[coe332-vm]$ kubectl apply -f hello-fastapi-service.yml
service/hello-service configured

We can list the services:

[coe332-vm]$ kubectl get services
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
hello-service   ClusterIP   10.233.12.76   <none>        8000/TCP   11s

We see k8s created a new service with IP 10.233.12.76. We should be able to use this IP address (and port 8000) to communicate with our FastAPI server. Let’s try it. Remember that we have to be on the k8s private network, so we need to exec into our debug deployment pod first.

[coe332-vm]$ kubectl exec -it py-debug-deployment-5cc8cdd65f-xzhzq -- /bin/bash

# from inside the container ---
root@py-debug-deployment-5cc8cdd65f-xzhzq:/ $ curl 10.233.12.76:8000/
Hello, world!

It worked! Now, if we remove our hello-fastapi pod, k8s will start a new one with a new IP address, but our service will automatically route requests to the new pod. Let’s try it.

# remove the pod ---
[coe332-vm]$ kubectl delete pods hello-fastapi-86d4c7d8db-2rkg5
pod "hello-fastapi-86d4c7d8db-2rkg5" deleted

# see that a new one was created ---
[coe332-vm]$ kubectl get pods
NAME                                    READY   STATUS    RESTARTS   AGE
hello-deployment-9794b4889-w4jlq        1/1     Running   2          175m
hello-pvc-deployment-6dbbfdc4b4-sxk78   1/1     Running   233        9d
hello-fastapi-86d4c7d8db-vbn4g             1/1     Running   0          62s

# it has a new IP ---
[coe332-vm]$ kubectl get pods hello-fastapi-86d4c7d8db-vbn4g -o wide
NAME                          READY   STATUS    RESTARTS   AGE    IP            NODE   NOMINATED NODE   READINESS GATES
hello-fastapi-86d4c7d8db-vbn4g   1/1     Running   0          112s   10.233.12.96   c05    <none>           <none>
# Yep, 10.233.12.96 -- that's different; the first pod had IP 10.233.116.59

# but back in the debug deployment pod, check that we can still use the service IP --
root@py-debug-deployment-5cc8cdd65f-xzhzq:/ $ curl 10.233.12.76:8000/
Hello, world!

Note that k8s is doing something non-trivial here. Each pod could be running on one of any number of worker computers in the TACC k8s cluster. When the first pod was deleted and k8s created the second one, it is quite possible it started it on a different machine. So k8s had to take care of rerouting requests from the service to the new machine.

k8s can be configured to do this “networking magic” in different ways. While the details are beyond the scope of this course, keep in mind that the virtual networking that k8s uses does come at a small cost. For most applications, including long-running web APIs and databases, this cost is negligible and isn’t a concern. But for high-performance applications, and in particular, applications whose performance is bounded by the performance of the underlying network, the overhead can be significant.

EXERCISE

Now, you have enough tools in your k8s toolbox to deploy your entire web app on the k8s cluster. Follow the steps below to try to launch your web app. In each step, you will be creating a new k8s resource described by its own k8s yaml file. We recommend carefully naming the files following a pattern like <name>-<env>-<resource>-<item>.yml. In this case <name> is the name of your app, <env> is the environment - test or prod, <resource> is the k8s resource used - e.g. deployment or service or pvc, and <item> is the identity of the service - e.g. fastapi or redis or worker. For example, imagine you are creating a deployment for the FastAPI front end for the test copy of an web app for analyzing the HGNC data. You might name this yaml file HGNC-test-deployment-fastapi.yml. Other naming schemes for these files are perfectly valid as long as there is consistency among them.

  • Step 1: Create a PVC for Redis to write dump files

  • Step 2: Create a deployment for your Redis database which binds the PVC from Step 1

  • Step 3: Create a service for your Redis deployment so you have a persistent IP address to use to communicate with Redis

  • Step 4: Create a deployment for your FastAPI API

  • Step 5: Create a service for your FastAPI API so you have a persistent IP address to use to communicate with FastAPI

  • Step 6: Create a deployment for your Worker which is scaled to three replicas

As you apply files, be sure to check your work in between each step. Make sure deployments and pods are ready and available, make sure PVCs are bound, make sure services are correctly associated with deployments, etc. Use a debug deployment to test things as much as possible along the way.

Public Access to Your Deployment

The final objective is to make your FastAPI API available on the public internet. This process assumes you have already created a deployment and a service (of type ClusterIP) for your FastAPI API. There are two new k8s objects required:

  1. A second Service object of type NodePort which selects your deployment using the deployment label and exposes your FastAPI on a public port.

  2. An Ingress object which specifies a subdomain to make your FastAPI available on and maps this domain to the public port created in Step 1.

Create a NodePort Service

The first step is to create a NodePort Service object pointing at your FastAPI deployment. Copy the following code into a new file called service-nodeport-hello-fastapi.yml or something similar:

 1---
 2kind: Service
 3apiVersion: v1
 4metadata:
 5    name: hello-fastapi-nodeport-service
 6spec:
 7    type: NodePort
 8    selector:
 9        app: hello-fastapi
10    ports:
11        - port: 8000
12          targetPort: 8000

Update the highlighted lines:

  1. The name of the Service object can be anything you want, so long as it is unique among the Services you have defined in your namespace. In particular, it needs to be a different name from your ClusterIP service defined previously.

  2. The value of app in the selector stanza needs to match the app label in your deployment. This should be exactly the same as what you did in the ClusterIP service created previously. As mentioned before, be sure the selector targets the label in your FastAPI deployment, not the deployment name.

As usual, create the NodePort using kubectl:

[coe332-vm]$ kubectl apply -f service-nodeport-hello-fastapi.yml
service/hello-fastapi-nodeport-service created

Change the command to reference the file name you used. Check that the service was created successfully and determine the port that was created for it:

[coe332-vm]$ kubectl get services
NAME                           TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
hello-fastapi-nodeport-service   NodePort    10.233.1.87    <none>        8000:32627/TCP   24s
hello-fastapi-service            ClusterIP   10.233.58.60   <none>        8000/TCP         4d10h

Here we see that port 32627 was created for this service. Your port will be different.

You can test that the NodePort service is working by using the special domain coe332.tacc.cloud to exercise your FastAPI from the kube-access VM:

[coe332-vm]$ curl coe332.tacc.cloud:32627/
Hello, world!

Change the port (32627) to the port associated with your nodeport service, and the URL path (/) to a path your FastAPI recognizes.

Note

The curl above only works on the public internet - in the next section, we will map the port to a subdomain of the host.

Create an Ingress

Next we will create an Ingress object which will map the NodePort port defined previously (in the above example, 32627) to a specific domain on the public internet.

Copy the following code into a new file called ingress-hello-fastapi.yml or similar:

 1---
 2kind: Ingress
 3apiVersion: networking.k8s.io/v1
 4metadata:
 5  name: hello-fastapi-ingress
 6  annotations:
 7    nginx.ingress.kubernetes.io/ssl-redirect: "false"
 8spec:
 9  ingressClassName: nginx
10  rules:
11  - host: "username-fastapi.coe332.tacc.cloud"
12    http:
13        paths:
14        - pathType: Prefix
15          path: "/"
16          backend:
17            service:
18              name: hello-fastapi-nodeport-service
19              port:
20                  number: 8000

Be sure to update the highlighted lines:

  1. Specify a meaningful name for the ingress. Keep in mind it should be unique among all Ingress obejcts within your namespace.

  2. Update the host value to include your username in the subdomain, i.e., use the format - host: "<username>.coe332.tacc.cloud".

Create the Ingress object:

[coe332-vm]$ kubectl apply -f ingress-hello-fastapi.yml

Double check that the object was successfully created:

[coe332-vm]$ kubectl get ingress
NAME                  CLASS    HOSTS                              ADDRESS   PORTS   AGE
hello-fastapi-ingress   <none>   username-fastapi.coe332.tacc.cloud             80      102s

At this point our FastAPI should be available on the public internet from the domain we specified in the host field. We can test by running the following curl command from anywhere, including our laptops.

[local]$ curl username-fastapi.coe332.tacc.cloud/
Hello, world!

Additional Resources