Open in app

Sign in

Write

Sign in

AWS ALB Ingress Controller with Istio Service Mesh

Manish Sharma
9 min readOct 4, 2022

--

[0] Provision AWS EKS from AWS Console

First step is to provision AWS EKS to install Istio. Skip this step if you already have AWS EKS provisioned.

Provision AWS EKS from the AWS Console

This page have all the steps documented to setup AWS EKS from the AWS UI Console

mycloudjourney.medium.com

[A] Install Istio & Istio Ingress Gateway in EKS

Using Istio Helm Chart

#Install Helm Client -> https://github.com/helm/helm/releases
$ mkdir -p "/home/cloudshell-user/21-09-2022/helm-cli"
$ cd "/home/cloudshell-user/21-09-2022/helm-cli"
$ wget https://get.helm.sh/helm-v3.10.0-rc.1-linux-amd64.tar.gz
$ tar -xvzf helm-v3.10.0-rc.1-linux-amd64.tar.gz
$ sudo mv linux-amd64/helm /usr/local/bin/helm
$ rm -rf helm-v3.10.0-rc.1-linux-amd64.tar.gz
$ rm -rf linux-amd64
$ helm --help#Configure Helm Repository
$ helm repo add istio https://istio-release.storage.googleapis.com/charts
$ helm repo update# Create a namespace
$ kubectl create namespace istio-system
$ kubectl get ns# Install the Istio base chart which contains cluster-wide resources used by the Istio control plane
$ helm install istio-base istio/base -n istio-system
$ helm status istio-base -n istio-system
$ helm get all istio-base -n istio-system
$ kubectl get crd# Install the Istio discovery chart which deploys the istiod service
$ helm install istiod istio/istiod -n istio-system --wait
$ kubectl get all -n istio-system##Steps for istio ingress gateway installation
# create a namespace for istio-gateway
$ kubectl create namespace istio-gateway# Enable istio-injection enabled for the istio-gateway kubernetes namespace
$ kubectl label namespace istio-gateway istio-injection=enabled
$ kubectl get namespace -L istio-injection# Install istio ingress gateway
$ mkdir -p "/home/cloudshell-user/21-09-2022/istio-ingress-gw-helm-chart"
$ cd "/home/cloudshell-user/21-09-2022/istio-ingress-gw-helm-chart"
$ helm pull istio/gateway
$ ls -la
$ tar -xvzf gateway-1.15.0.tgz
$ rm -rf gateway-1.15.0.tgz
$ cd gateway
$ helm install istio-ingressgateway . -n istio-gateway
OR
$ helm install istio-ingressgateway istio/gateway -n istio-gateway$ helm get all istio-ingressgateway -n istio-gateway
$ kubectl get all -n istio-gateway

[B] Deploy Sample Book Store Applications

# Create namespace to deploy sample applications
$ kubectl create ns sample-applications# Enable istio-injection for any new pods that are created in that namespace will automatically have a sidecar added to them.
$ kubectl label namespace sample-applications istio-injection=enabled# Download Sample Applications
$ mkdir -p "/home/cloudshell-user/21-09-2022/sample-applications"
$ cd "/home/cloudshell-user/21-09-2022/sample-applications/"
$ wget https://raw.githubusercontent.com/istio/istio/release-1.15/samples/bookinfo/platform/kube/bookinfo.yaml# Deploy Sample Applications
$ kubectl apply -f bookinfo.yaml -n sample-applications# Get deployment status of all the sample applications
$ kubectl get all -n sample-applications# To confirm that the Bookinfo application is running
$ kubectl exec "$(kubectl get pod -n sample-applications -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -n sample-applications -- curl -sS productpage:9080/productpage# Create istio gateway and Virtual Service for sample applications
$ wget https://raw.githubusercontent.com/istio/istio/release-1.15/samples/bookinfo/networking/bookinfo-gateway.yaml
$ kubectl apply -f bookinfo-gateway.yaml -n sample-applications
$ kubectl get gateway -n sample-applications
$ kubectl get vs -n sample-applications# Export Istio Ingress Gateway Service LB hostname and port
$ kubectl get svc istio-ingressgateway -n istio-gateway
$ export INGRESS_HOST=$(kubectl -n istio-gateway get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
$ export INGRESS_PORT=$(kubectl -n istio-gateway get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-gateway get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
$ export TCP_INGRESS_PORT=$(kubectl -n istio-gateway get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].port}')
$ export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT# Access Application from local machine
$ curl -s "http://${GATEWAY_URL}/productpage"# Access Application from local machine
$ echo $GATEWAY_URL/productpage #copy output and paste in the browser

[C] Install and Configure AWS ALB Ingress Controller

$ mkdir -p "/home/cloudshell-user/21-09-2022/aws-alb-ingress-controller"
$ cd "/home/cloudshell-user/21-09-2022/aws-alb-ingress-controller"

Prerequisites

  1. Create an IAM OIDC Identity Provider (Terraform / Manual ?)

Following this link or execute below commands to create an IAM OIDC identity provider for our AWS Managed EKS cluster with the AWS Management Console (https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html)

INFO: Our AWS Managed EKS cluster has an OpenID Connect (OIDC) issuer URL associated with it. To use AWS Identity and Access Management (IAM) roles for service accounts, an IAM OIDC provider must exist for our EKS cluster. Check section 2.4.

2. Open the Amazon EKS console at https://console.aws.amazon.com/eks/home#/clusters.

3. In the left pane, select Clusters, and then select the name of your cluster on the Clusters page.

4.In the Details section on the Overview tab, note the value of the OpenID Connect provider URL.

5. Open the IAM console at https://console.aws.amazon.com/iam/.

6. In the left navigation pane, choose Identity Providers under Access management. If a Provider is listed that matches the URL for your cluster, then you already have a provider for your cluster. If a provider isn’t listed that matches the URL for your cluster, then you must create one.

7. To create a provider, choose Add provider.

8. For Provider type, select OpenID Connect.

9. For Provider URL, enter the OIDC provider URL for your cluster, and then choose Get thumbprint.

10. For Audience, enter sts.amazonaws.com and choose Add provider.

Deploy the AWS Load Balancer Controller

To deploy the AWS Load Balancer Controller to an Amazon EKS cluster (https://docs.aws.amazon.com/eks/latest/userguide/aws-load-balancer-controller.html)

2.1 Download an IAM Policy

Download an IAM policy for the AWS Load Balancer Controller that allows it to make calls to AWS APIs on your behalf.

$ curl -o /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/iam_policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.3/docs/install/iam_policy.json

2.2 Create an IAM policy using the policy downloaded in the step 2.1

$ aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file:///home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/iam_policy.json

We have an option to create AWS IAM Policy through AWS Management console as well. Prefer any option.

2.3 Create an IAM role.

  • View test-alb EKS cluster OIDC provider URL
$ aws eks describe-cluster --name test-alb --query "cluster.identity.oidc.issuer" --output textSample Output:
https://oidc.eks.ap-south-1.amazonaws.com/id/83F071580BE9D7783C31D7E58EB023E4
Just copy the text just after /id/
  • Create LB Trust Policy

INFO: The following trust policy limits access to the defined service account (SA) aws-app-load-balancer-controller in istio-gateway namespace.

cat > "/home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/load-balancer-role-trust-policy.json" <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::799361485816:oidc-provider/oidc.eks.ap-south-1.amazonaws.com/id/8DCB4B0BCC7118A1AB1B37ED21E1487B"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    "oidc.eks.ap-south-1.amazonaws.com/id/8DCB4B0BCC7118A1AB1B37ED21E1487B:aud":"sts.amazonaws.com",
                    "oidc.eks.ap-south-1.amazonaws.com/id/8DCB4B0BCC7118A1AB1B37ED21E1487B:sub":"system:serviceaccount:kube-system:aws-load-balancer-controller"
                }
            }
        }
    ]
}
EOF
  • Create the AWS IAM Role applying above mentioned policy document
$ aws iam create-role \
  --role-name AmazonEKSLoadBalancerControllerRole \
  --assume-role-policy-document file://"/home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/load-balancer-role-trust-policy.json"

Output will look something like this

{
    "Role": {
        "Path": "/",
        "RoleName": "AmazonEKSLoadBalancerControllerRole",
        "RoleId": "AROA3UHNIJP4PFRKFE6ME",
        "Arn": "arn:aws:iam::799361485816:role/AmazonEKSLoadBalancerControllerRole",
        "CreateDate": "2022-09-21T08:54:07+00:00",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Effect": "Allow",
                    "Principal": {
                        "Federated": "arn:aws:iam::799361485816:oidc-provider/oidc.eks.ap-south-1.amazonaws.com/id/8DCB4B0BCC7118A1AB1B37ED21E1487B"
                    },
                    "Action": "sts:AssumeRoleWithWebIdentity",
                    "Condition": {
                        "StringEquals": {
                            "oidc.eks.ap-south-1.amazonaws.com/id/8DCB4B0BCC7118A1AB1B37ED21E1487B:aud": "sts.amazonaws.com",
                            "oidc.eks.ap-south-1.amazonaws.com/id/8DCB4B0BCC7118A1AB1B37ED21E1487B:sub": "system:serviceaccount:kube-system:aws-load-balancer-controller"
                        }
                    }
                }
            ]
        }
    }
}

Attach the Amazon EKS managed IAM policy to the AmazonEKSLoadBalancerControllerRole IAM role

$ aws iam attach-role-policy \
  --policy-arn arn:aws:iam::799361485816:policy/AWSLoadBalancerControllerIAMPolicy \
  --role-name AmazonEKSLoadBalancerControllerRole

2.4 Create AWS Application Load Balancer Controller Service Account file

  • Create Kubernetes Service Account file
cat >/home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws-load-balancer-controller-service-account.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/name: aws-load-balancer-controller
  name: aws-load-balancer-controller
  namespace: kube-system
  annotations: 
    eks.amazonaws.com/role-arn: arn:aws:iam::799361485816:role/AmazonEKSLoadBalancerControllerRole
EOF
  • Create the Kubernetes Service account (aws-app-load-balancer-controller) in our AWS Managed EKS cluster test-alb
$ kubectl apply -f /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws-load-balancer-controller-service-account.yaml
$ kubectl get sa/aws-app-load-balancer-controller -n kube-system

[D] Download the AWS ALB Controller Specification

$ curl -Lo /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_controller_spec_v2_4_3_full.yaml https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/download/v2.4.3/v2_4_3_full.yaml

If we downloaded the AWS ALB Controller Spec v2_4_3 file, then run the following commands

  • To remove the ServiceAccount section in the manifest.
$ sed -i.bak -e '480,488d' /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_controller_spec_v2_4_3_full.yaml
  • Replace the placeholder cluster-name text with your test-alb AWS Managed EKS Cluster-name in the Deployment spec section of the file
$ sed -i.bak -e 's|your-cluster-name|test-alb|' /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_controller_spec_v2_4_3_full.yaml
  • Replace the placeholder cluster-name text with your test-alb AWS Managed EKS Cluster-name in the Deployment spec section of the file
$ sed -i.bak -e 's|your-cluster-name|test-alb|' /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_controller_spec_v2_4_3_full.yaml
  • Apply AWS ALB Controller Specification file
$ $ kubectl apply -f /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_controller_spec_v2_4_3_full.yaml

This command will create aws load balancer webhook service in kube-system namespace

Apply IngressClass & IngressClassParams to EKS Cluster

  • Download Ingress Class file
$ curl -Lo /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_ingclass_v2_4_3.yaml https://github.com/kubernetes-sigs/aws-load-balancer-controller/releases/download/v2.4.3/v2_4_3_ingclass.yaml

Content of the file ingclass.yml file will look something like this. Change the name from alb to something else , if required.

apiVersion: elbv2.k8s.aws/v1beta1
kind: IngressClassParams
metadata:
  labels:
    app.kubernetes.io/name: aws-load-balancer-controller
  name: alb
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/name: aws-load-balancer-controller
  name: alb
spec:
  controller: ingress.k8s.aws/alb
  parameters:
    apiGroup: elbv2.k8s.aws
    kind: IngressClassParams
    name: alb

The .spec.parameters field of an IngressClass lets us reference another resource that provides configuration related to that IngressClass. In our case reference resource is IngressClassParams

INFO: IngressClassParams is a CRD specific to the AWS Load Balancer Controller, which can be used along with IngressClass’s parameter field. We can use IngressClassParams to enforce settings for a set of Ingresses.

  • Apply the IngressClass and IngressClassParams manifest to our AWS Managed EKS cluster
l$ kubectl apply -f /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/aws_alb_ingclass_v2_4_3.yaml

[E] ALB Ingress Controller Deployment Validation

  • Verify that the AWS ALB Controller is installed
$ kubectl get deployments/aws-app-load-balancer-controller -n kube-system 
OR
$ kubectl get deployments -n kube-system

[F] Create ALB Ingress

cat >/home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/alb-ingress.yaml <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: alb-external-ingress
  namespace: istio-gateway
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
    # alb.ingress.kubernetes.io/listen-ports: '[{HTTP: 80}, {HTTPS: 443}]'
    # alb.ingress.kubernetes.io/actions.ssl-redirect: '443'
    alb.ingress.kubernetes.io/load-balancer-name: alb-external-ingress
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/subnets: subnet-0c11109d0b67a60b7, subnet-0d42b2dbf61968c87, subnet-058587473f3c14f4e
    alb.ingress.kubernetes.io/scheme: internet-facing
    # alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:123456789011:certificate/26cec536-5d01-4140-890d-XXXXXXXXXXXX
    # alb.ingress.kubernetes.io/wafv2-acl-arn: arn:aws:wafv2:us-west-2:123456789011:regional/webacl/prod-common-alb-acl/ae2e63f9-02f4-44b8-9d25-XXXXXXXXXXXX
  labels:
    app: alb-external-ingress
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: istio-ingressgateway
                port:
                  number: 80
EOF

Execute below command to apply this file

$ kubectl apply -f /home/cloudshell-user/21-09-2022/aws-alb-ingress-controller/alb-ingress.yaml

[G] Access Applications

Option 1: Copy the AWS application load balancer DNS name from the AWS Load Balancer page and paste in your browser window

Option 2:

  • Copy the AWS application load balancer DNS name from the AWS Load Balancer page
  • Navigate to AWS Route 53 Hosted zone
  • Edit <xx-domain-name.com> external domain mapping to AWS Application Load Balancer dns name

[H] Remove Classic Load Balancer

If you do not want AWS Classic Load Balancer for then istio Ingress gateway then update helm chart values.yaml file and change the service type from LoadBalancer to NodePort.

Before

After

After change upgrade istio-ingressgateway helm release

$ helm upgrade istio-ingressgateway . -n istio-gateway
$ kubectl get svc -n istio-gateway

Check in AWS Console as well . You will find only Application Load Balancer (ALB) there.

Istio
Istio Service Mesh
Kubernetes
Ingresscontroller
Alb Ingress Controller

--

--

Manish Sharma
Manish Sharma

Written by Manish Sharma

162 Followers
8 Following

I am a technology enthusiast with a passion for continuous learning & innovation. Certified as an AWS Soln Architect Associate & HashiCorp Terraform Associate.

No responses yet

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams