Mutual TLS

Enabling MTLS on our Nginx Ingress Controller is quite simple.

1. Create the Certificate Secret and a policy to use on the Virtual Server resource

cat << EOF | kubectl apply -f -
kind: Secret
metadata:
  name: ingress-mtls-secret
apiVersion: v1
type: nginx.org/ca
data:
  ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQvVENDQXVXZ0F3SUJBZ0lVSzdhbU14OFlLWG1BVG51SkZETDlWS2ZUR2ZNd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2dZMHhDekFKQmdOVkJBWVRBbFZUTVFzd0NRWURWUVFJREFKRFFURVdNQlFHQTFVRUJ3d05VMkZ1SUVaeQpZVzVqYVhOamJ6RU9NQXdHQTFVRUNnd0ZUa2RKVGxneEREQUtCZ05WQkFzTUEwdEpRekVXTUJRR0ExVUVBd3dOCmEybGpMbTVuYVc1NExtTnZiVEVqTUNFR0NTcUdTSWIzRFFFSkFSWVVhM1ZpWlhKdVpYUmxjMEJ1WjJsdWVDNWoKYjIwd0hoY05NakF3T1RFNE1qQXlOVEkyV2hjTk16QXdPVEUyTWpBeU5USTJXakNCalRFTE1Ba0dBMVVFQmhNQwpWVk14Q3pBSkJnTlZCQWdNQWtOQk1SWXdGQVlEVlFRSERBMVRZVzRnUm5KaGJtTnBjMk52TVE0d0RBWURWUVFLCkRBVk9SMGxPV0RFTU1Bb0dBMVVFQ3d3RFMwbERNUll3RkFZRFZRUUREQTFyYVdNdWJtZHBibmd1WTI5dE1TTXcKSVFZSktvWklodmNOQVFrQkZoUnJkV0psY201bGRHVnpRRzVuYVc1NExtTnZiVENDQVNJd0RRWUpLb1pJaHZjTgpBUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTmFINVRzaTZzaUFsU085dEJnYmY3VVRwcWowMUhRTlQ2UjhtQy9pCjhLYXFaSW9XSUdvN2xhTW9xTDYydTc4ay9WOHM2Z0FJaU1DSzBjekFvTFhNSnlJQkxQeTg4Yzdtc2xwZXgxTkEKVmRtMkVTVkN6bVlERE1TT3FpVmszWmpYeC9URmo2QzhNRFhhRkZUWFg1dWdtbWdscnFCWlh0OVI5VVBwVTJMNwo1bEZ0NlJ2R3VGczgvbVZORVR5c1A0SFhCWlh2ZE9mdG1YWUkvK01hOW5CMzIzNjdmcTI0L0RKZ2YvK2xRbUsxCkJLR3poSTZSc1pSSmdWOXdpK1VuZTBYNjlaS2lLOFdXU3lZS252YnRrcHZuTDA2dGNJaXJZNi80UzZ4Sm1HRVQKZEJUNmVxc0NoSUpQUStWSEp5dTROdnV6WmVCUXpGdmMwNytnUGZkVWZra1FXODhDQXdFQUFhTlRNRkV3SFFZRApWUjBPQkJZRUZKUGdhcnFYa00rdEJ0djVhdndTUWhUQmpTU2VNQjhHQTFVZEl3UVlNQmFBRkpQZ2FycVhrTSt0CkJ0djVhdndTUWhUQmpTU2VNQThHQTFVZEV3RUIvd1FGTUFNQkFmOHdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUIKQUl3WXpoY0s4OWtRL0xGWjZFRHgrQWp2bnJTVSs1cmdwQkgrRjVTNUUyY3pXOE5rNXhySnl0Y0ZUbUtlKzZScwpENHlxeTZSVVFEeWNYaDlPelBjbzgzYTBoeFlCZ1M5MWtJa25wYWF4dndLRDJleWc3UGNnK1lkS1FhZFlMcUY0CmI3cWVtc1FVVkpOWHdkZS9VanRBejlEOTh4dngwM2hQY2Qwb2dzUUhWZ21BZVpFd2l3UzFmTy9WNUE4dTl3MEkKcHlJRTVReXlHcHNpS2dpalpiMmhrS05RVHVJcEhiVnFydVA4eEV6TlFnamhkdS9uUW5OYy9lRUltVUlrQkFUVQpiSHdQc2xwYzVhdVV1TXJxR3lEQ0p2QUJpV3J2SmE3Yi9XcmtDT3FUWVhtR2NGM0w1ZU9FeTBhYkp0M2NNcSs5CnJLTUNVQWlkNG0yNEthWnc3OUk2anNBPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
---
apiVersion: k8s.nginx.org/v1
kind: Policy
metadata:
  name: ingress-mtls-policy
spec:
  ingressMTLS:
    clientCertSecret: ingress-mtls-secret
    verifyClient: "on"
    verifyDepth: 1
EOF

2. Attach the MTLS policy to the VS.

cat << EOF | kubectl apply -f -
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
  name: arcadia
spec:
  host: $nginx_ingress  
  tls:
    secret: arcadia-wildcard # Represents the server certificate
    redirect:
      enable: true # Always redirect to https if incoming request is http
  policies:
    - name: ingress-mtls-policy
  upstreams:
    - name: arcadia-users
      service: arcadia-users
      port: 80
      healthCheck: # This is the most basic healthcheck config for more info follow this link https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/#upstream-healthcheck
        enable: true
        path: /healthz
    - name: arcadia-login
      service: arcadia-login
      port: 80
      healthCheck:
        enable: true
        path: /healthz
    - name: arcadia-stocks
      service: arcadia-stocks
      port: 80
      healthCheck:
        enable: true
        path: /healthz
    - name: arcadia-stock-transaction
      service: arcadia-stock-transaction
      port: 80
      healthCheck: 
        enable: true
        path: /healthz
    - name: arcadia-frontend
      service: arcadia-frontend
      port: 80
      healthCheck:
        enable: true
        path: /healthz
  routes:
    - path: /v1/user      
# These directives attach the JWT policy to the route that needs authentication extract the username/email address and add it as a header 
      policies:
      - name: jwt-policy
      action:
        proxy:
          upstream: arcadia-users
          requestHeaders:
            set:
            - name: okta-user
              value: \${jwt_claim_email}
    - path: /v1/login      
      action:
        pass: arcadia-login
    - path: /v1/stock      
      action:
        pass: arcadia-stocks
    - path: /v1/stockt      
      policies:
      - name: jwt-policy
      action:
        proxy:
          upstream: arcadia-stock-transaction
          requestHeaders:
            set:
            - name: okta-user
              value: \${jwt_claim_email}
    - path: /      
      action:
        pass: arcadia-frontend
EOF

3. Verify this is actually working by running the bellow command, it will use the client cert/key pair on the Cloud9 instance to authenticate:

curl -k https://$nginx_ingress/ --cert ./files/4ingress/client-cert.pem --key ./files/4ingress/client-key.pem

4. We are finished with this part of our experience .
   Before moving forward reapply the ingress configuration without the mutual tls configuration

cat << EOF | kubectl apply -f -
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
  name: arcadia
spec:
  host: $nginx_ingress  
  tls:
    secret: arcadia-wildcard # Represents the server certificate
    redirect:
      enable: true # Always redirect to https if incoming request is http
  upstreams:
    - name: arcadia-users
      service: arcadia-users
      port: 80
      healthCheck: # This is the most basic healthcheck config for more info follow this link https://docs.nginx.com/nginx-ingress-controller/configuration/virtualserver-and-virtualserverroute-resources/#upstream-healthcheck
        enable: true
        path: /healthz
    - name: arcadia-login
      service: arcadia-login
      port: 80
      healthCheck:
        enable: true
        path: /healthz
    - name: arcadia-stocks
      service: arcadia-stocks
      port: 80
      healthCheck:
        enable: true
        path: /healthz
    - name: arcadia-stock-transaction
      service: arcadia-stock-transaction
      port: 80
      healthCheck: 
        enable: true
        path: /healthz
    - name: arcadia-frontend
      service: arcadia-frontend
      port: 80
      healthCheck:
        enable: true
        path: /healthz
  routes:
    - path: /v1/user      
# These directives attach the JWT policy to the route that needs authentication extract the username/email address and add it as a header 
      policies:
      - name: jwt-policy
      action:
        proxy:
          upstream: arcadia-users
          requestHeaders:
            set:
            - name: okta-user
              value: \${jwt_claim_email}
    - path: /v1/login      
      action:
        pass: arcadia-login
    - path: /v1/stock      
      action:
        pass: arcadia-stocks
    - path: /v1/stockt      
      policies:
      - name: jwt-policy
      action:
        proxy:
          upstream: arcadia-stock-transaction
          requestHeaders:
            set:
            - name: okta-user
              value: \${jwt_claim_email}
    - path: /      
      action:
        pass: arcadia-frontend
EOF