As a part of the Kubernetes tutorials for beginners series, my goal is to offer a full and solid list of articles that goes through the very basics definitions, the history and the need to use Kubernetes and containers until i reach the deep parts, so regardless of your technical background, i will offer you everything you need here to master Kubernetes step by step.
On this article we are going to discuss The lifecycle of a Kubernetes Pod, so let’s start first by the definition of the Pod, so a Pod is the smallest unit of organization in Kubernetes. It will contain one or more containers, all of which share storage, network, and deployment specifications. Everything in a Pod will be deployed together, at the same time, in the same location, and will share a schedule along with the deployment parameters.
Pod model types
We have two model types of pod you can create. “one-container-per-pod” and “multi-container-pod.
- One-container-per-pod: This one is the most popular model. The Pod here acts as a wrapper for a single container and since Pod is the smallest object Kubernetes knows, it manages the Pods rather than the containers directly.
- Multi-container-pod: For this model a pod might hold multiple co-located containers that are tightly coupled and need to share resources. These containers work as a single cohesive unit of service. The Pod wraps these containers and storage resources together as a single unit. Some example use cases are sidecars, proxies, logging.
The idea is that each pod is meant to run a single instance of a given application. If you want to scale your application horizontally (e.g., run multiple replicas), you should use multiple Pods, one for each instance. Note that this is not the same thing as running multiple containers of the same application in a single pod.
It is worth mentioning that Pods aren’t meant to be durable entities so if a node fails, or in the case of node maintenance etc, it won’t survive. There is something called Controllers in Kubernetes that solves this issue for us. In general pods created with some type of controller.
The States of a Pod
Over the pod lifecycle, it can attain the following states:
- The Pending State: This means that the pod is accepted by the Kubernetes system but its container(s) is/are not created yet.
- The Running State: Here the pod is scheduled on a node and all its containers are created and at-least one container is in Running state.
- The Succeeded State: At this point, all the container(s) in the Pod have exited with status 0 and will not be restarted.
- The Failed State: On this state, all the container(s) of the Pod have exited and at least one container has returned a non-zero status.
- The CrashLoopBackoff State: Here the container fails to start and is tried again and again.
The Birth of a Pod
Now let’s look at the events or the steps that lead to the creation of a Pod:
Source: Joe Beda’s Blog.
- The kubectl API client or any other one submits the Pod specifications to the API server.
- Then, the API server writes the Pod object to the etcd data store. Once the write is successful, an acknowledgment is sent back to API server and to the client.
- The API server now reflects the change in state of etcd.
- All Kubernetes components use watches to keep checking API server for relevant changes.
- Here, the kube-scheduler (via its watcher) finds that a new Pod object is created on API server but is not bound to any node.
- kube-scheduler assigns a node to the pod and updates the API server.
- This change is then propagated to the etcd data store. The API server also reflects this node assignment on its Pod object.
- Kubelet on every node also runs watchers who keep watching API server. Kubelet on the destination node sees that a new Pod is assigned to it.
- Kubelet starts the pod on its node by calling Docker and updates the container state back to the API server.
- The API server persists the pod state into etcd.
- Once etcd sends the acknowledgment of a successful write, the API server sends an acknowledgment back to kubelet indicating that the event is accepted.
Tasks During a Pod’s Life
The Init containers are containers which are run before the main application container gets started. They have two important characteristics:
- They always run to completion.
- Each init container must complete before the next one is started.
They can be useful when some initial actions need to be run before the main container in the pod starts.
Example: copying config files and updating config values. Init containers use different Linux namespaces, so they have a different filesystem view so they can be given access to secrets which may not be desirable for sharing within the app container.
kubelet can run code triggered by Container Lifecycle Hooks. This allows the user to run specific code during specific events of a containers lifecycle.
For example: running a graceful shutdown script before a container is terminated.
There are two hooks which are exposed:
PostStart : This hook gets executed upon container creation but there is no guarantee that it will run after the container ENTRYPOINT.
PreStop : This hook gets executed just before a container is terminated. This is a blocking call which means the hook execution must complete before the call to delete a container can be sent.
Both hooks mentioned above do not take any parameters. There are two types of handlers which can be implemented in the hook implementation:
Exec : runs a specific command inside the container and the resources consumed by the command are counted against the container.
HTTP : executes an HTTP request against a specific endpoint on the container.
Apart from lifecycle hooks, another important thing which happens during a pods lifetime is the execution of container probes.
Container probes are diagnostics performed by kubelet on the container. There are two kinds of probes which kubelet can run on running containers:
livenessProbe : Indicates whether the container is running. If the liveness probe fails, kubelet kills the container and the container is subjected to its Restart Policy.
readinessProbe : Indicates whether the container is ready to service requests. If this probe fails, the endpoints controller removes the container IP from list of endpoints of all services that match the Pod.
There are three ways to implement a probe:
ExecAction : Executes a command inside the container. The diagnostic is considered successful if the command returns 0.
TCPSocketAction : Performs a TCP socket check against the container IP and specified port. The diagnostic is considered successful if the port is open.
HTTPGetAction : Runs an HTTP GET action against the container IP with the specified port and path. The diagnostic is considered successful if the response has a status code between 200 and 400.
The termination of a Pod
- The user sends a command to delete a Pod.
- The Pod object in the API server is updated with the time beyond which the Pod is considered “dead” (default of 30 seconds) along with the grace period.
- THe below actions happen in parallel:
- The pod shows up as “Terminating” when listed in client commands.
- When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the pod shutdown process.
- The endpoint controller watches the pod is about to be deleted and hence removes the pod from all the endpoints which were serviced by the pod.
- If the pod has defined a preStop hook, it is invoked inside of the pod. If the
preStophook is still running after the grace period expires, step 2 is then invoked with a small (2 second) extended grace period.
- The processes in the Pod are sent the TERM signal.
- When the grace period expires, any processes still running in the Pod are killed with SIGKILL.
- The Kubelet will finish deleting the Pod on the API server by setting grace period 0 (immediate deletion). The Pod disappears from the API and is no longer visible from the client.
We can see that there are multiple ways to control the events that happen within the duration of a Pod’s lifetime. Init container(s) can help remove a lot of complexity related to bootstrapping of containers and thus help keep logic within the main containers simple. Similarly, a post start lifecycle hook can help run any code (such as registering to a monitoring system or a service mesh) which needs to run once the container starts running. Liveness and readiness probes help remove bad pods before they start disrupting any customers. Graceful shutdowns can be run as a pre-stop lifecycle hook allowing for a lot more elegant exit. Knowing the above control mechanisms can help in better designing a pod and the supporting use cases.