GKE Series: GKE Nodepool

This article explains how node pools work in Google Kubernetes Engine.

Key Points

• A node pool is a group of nodes within a cluster that all have the same configuration.
• Each node in the pool has a Kubernetes node label, cloud.google.com/gke-nodepool, which has the node pool's name as its value.
• Default node pool is created when you create a GKE cluster
• You can add additional node pools of different sizes and types to your cluster. All nodes in any given node pool are identical to one another.
• Custom (additional) node pools are useful when you require more resources than others, such as more memory or more local disk space.
• You can create, delete or update a node pool individually without affecting the whole cluster.
• You cannot configure a single node in a node pool; any configuration changes affect all nodes in the node pool.
• You can also run multiple Kubernetes node versions on each node pool in your cluster, update each node pool independently, and target different node pools for specific deployments.
• When you delete a node pool, GKE drains all the nodes in the node pool. The draining process involves GKE evicting Pods on each node in the node pool.
• Add a node pool to GKE Standard Cluster using Google GCLOUD Command

gcloud beta container node-pools create POOL_NAME \
    --cluster CLUSTER_NAME \
    --project GCP_PROJECT_ID \
    --region GCP_REGION \
    --node-version GKE_VERSION \
    --machine-type INSTANCE_TYPE \
    --image-type "COS_CONTAINERD" \
    --disk-type PERSISTENT_DISK_TYPE \
    --disk-size PD_DISK_SIZE_IN_GB \
    --node-labels KEY1=VALUE1,KEY2=VALUE2,.... \
    --scopes "SCOPE_API1, SCOPE_API2,....." \
    --num-nodes MIN_NODES_PER_ZONE \
    --max-nodes MAX_NODES_PER_ZONE \
    --enable-autoupgrade \
    --enable-autorepair \
    --max-surge-upgrade INT_VALIE \
    --max-unavailable-upgrade INT_VALUE \
    --max-nodes-per-node PODS_COUNT \
    --shielded-integrity-monitoring \
    --shielded-secure-boot \
    --tags "TAG1,TAG2,...." \
    --node-taints KEY=VALUE:ACTION

• View all the node pools in a cluster

gcloud container node-pools list --cluster CLUSTER_NAME

• View details about specific node pool in a cluster

gcloud container node-pools describe POOL_NAME \
    --cluster CLUSTER_NAME

• Resize a node pool

gcloud container clusters resize CLUSTER_NAME \
    --node-pool POOL_NAME \
    --num-nodes NUM_NODES

NUM_NODES: the number of nodes in the pool in a zonal cluster. If you use multi-zonal or regional clusters, NUM_NODES is the number of nodes for each zone the node pool is in.

• Delete a node pool

gcloud container node-pools delete POOL_NAME \
    --cluster CLUSTER_NAME

• By default, GKE upgrades one node at a time.
• While a node is being upgraded, GKE stops scheduling new Pods onto it, and attempts to schedule its running Pods onto other nodes.
• The upgrade is only complete when all nodes have been recreated and the cluster is in the desired state. When a newly-upgraded node registers with the control plane, GKE marks the node as schedulable.
• During automatic or manual node upgrades, PodDisruptionBudgets (PDBs) are respected for a maximum of 1 hour. If Pods running on a node cannot be scheduled onto new nodes within 1 hour, the upgrade is initiated, regardless.
• GKE control planes are compatible with nodes up to two minor versions older than the control plane. For example, Kubernetes 1.13 control planes are compatible with Kubernetes 1.11 nodes.
• Upgrading a node pool may disrupt workloads running in that node pool. To avoid this, you can create a new node pool with the desired version and migrate the workload. After migration, you can delete the old node pool.
• Manually Upgrade a node pool with different cluster version

gcloud container clusters upgrade CLUSTER_NAME \
  --node-pool=NODE_POOL_NAME \
  --cluster-version VERSION

Options for Pod Scheduling/Deployment

• You can explicitly deploy a Pod to a specific node pool by setting a nodeSelector in the Pod manifest. This forces a Pod to run only on nodes in that node pool.
• You can specify resource requests for the containers. The Pod only runs on nodes that satisfy the resource requests.
• Node Affinity and Node Anti-Affinity
• Pod Affinity and Pod Anti-Affinity

References

• Add a node pool to GKE Standard Cluster
• Resize a node pool
• Upgrade a node pool
• Delete a node pool
• Deploying a Pod to a specific node pool