Helm chart nginx-ingress controller TCP redirect http to https

Question

My environment is hosted in AWS. I had deployed the helm chart stable/nginx-ingress in TCP mode (I need an AWS ELB in TCP mode for WebSocket. I need exactly ELB classic!), also I need to use AWS ACM certificate and my backend without it. So, it should be like:

TCP:80 -> TCP:30452 (Kubernetes)
SSL(TCP):443 -> TCP:31453 (Kubernetes) 

My "nginx-ingress" is:

controller:
  name: controller
  image:
    repository: quay.io/kubernetes-ingress-controller/nginx-ingress-controller
    tag: "0.27.0"
    pullPolicy: IfNotPresent
    # www-data -> uid 101
    runAsUser: 101
    allowPrivilegeEscalation: true

  # Configures the ports the nginx-controller listens on
  containerPort:
    http: 80
    https: 80

  # Will add custom configuration options to Nginx https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/
  config:
    allow-backend-server-header: "true"
    use-proxy-protocol: "true"
    server-tokens: "false"
    use-forwarded-headers: "true"
    ssl-redirect: "true"
    http-redirect-code: "301"
    proxy_redirect: "off"

  ingressClass: websocket

  service:
    enabled: true

    annotations:
      # Enable PROXY protocol
      service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '120'
      # SSL
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:us-west-1:123123123123:certificate/123123123123123123123"
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: 'https'
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: 'tcp'

    ports:
      http: 80
      https: 443

    targetPorts:
      http: http
      https: http

    type: LoadBalancer
    nodePorts:
      http: ""
      https: ""
      tcp: {}
      udp: {}

My INGRESS configuration for service:

ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: websocket

  hosts:
    - host: example.com
      paths:
        - /

How can I set up a redirect from HTTP to HTTPS? When I use nginx.ingress.kubernetes.io/force-ssl-redirect: "true" in ingress, I get redirect loop :(

Answer 1

This is a known issue when using SSL termination combined with ELB Classic in TCP mode (Layer 4).

You can use the following workaround, which consists in:

1 - Create a custom ConfigMap for nginx-ingress, using the following http-snippet:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: ingress-nginx
  name: nginx-configuration
  namespace: <ingress-namespace>
data:
  ssl-redirect: "false"
  hsts: "true"
  server-tokens: "false"
  # You may not want to use proxy_protocol here
  http-snippet: |
    server {
      listen 8000 proxy_protocol;
      server_tokens off;
      return 301 https://$host$request_uri;
    }

2 - Make sure that your Ingress is configured to use the above configMap, like this.

3 - Create a new NodePort and add the definitions to nginx ingress container and Service. Example:

Container:

  - name: http
    containerPort: 80
  - name: https
    containerPort: 443
  - name: http-redirect
    containerPort: 8000

Service:

  ports:
  # As you are using SSL termination, note that this does Service(443)-->Container(80)
  - name: https
    port: 443
    protocol: TCP
    targetPort: http # <-- Container port 80
    nodePort: 31453
  - name: http
    port: 8000
    protocol: TCP
    targetPort: http-redirect
    nodePort: 32000

4 - Point the port 80 of your ELB to port 8000 of the nginx ingress Service.

TCP:80 -> TCP:32000 (Kubernetes)
SSL(TCP):443 -> TCP:31453 (Kubernetes) 

This will create an additional listener whose only function is to make the redirect. As it is reached only once, the redirect loop will be solved. The request should behave like this:

ELB(80) --> Nginx Service(32000) --> Nginx Container(8000) --Redirect--> ELB(443) --> Nginx Service(31453) -- Nginx Container (80)

Answer 2

https://github.com/ranjith-ka/Docker/blob/master/minikube/nginx/values.yaml

Try this, I configure ELB to fwd the traffic to http port, this works fine for me. Anyhow i want to use terraform to bring up ELB, not nginx helm charts