Resolve Nginx file upload limits and 504 gateway timeout in Kubernetes

wzdushu@gmail.com (wanzi) — Mon, 08 Nov 2021 17:22:42 +0800

Recently, two recurring issues have arisen in our business operations using the Kubernetes cluster. Here’s a record of the solutions:

Frontend page file upload limited to 1M
POST requests from frontend to backend timing out with a 504 error

Solution for the first issue:
By default, Nginx limits upload size to 1M. To resolve this, add the following configuration in the http, server, or location blocks of the Nginx config:

`1`	`client_max_body_size 800M;`

Solution for the second issue:
By default, Nginx has a 1-minute timeout for reading responses from the backend. To extend this, add the following in the http, server, or location blocks:

1
2
3

proxy_connect_timeout 120;
proxy_read_timeout  900;
proxy_send_timeout 900;

Final Nginx configuration:

server {
    listen 80;
    server_name web_server;
    location / {
        root /usr/share/nginx/html/;
        index index.htm index.html;
        try_files $uri $uri/ /index.html;
    }
    location /business-operation {
        proxy_pass http://business-operat.common:5000;
        
        proxy_ignore_client_abort on; 
        keepalive_timeout 900s;
        client_body_timeout 900s;
        proxy_connect_timeout 120s;
        proxy_read_timeout  900s;
        proxy_send_timeout 900s;
        client_max_body_size 800m;
        proxy_pass_header Authorization;
        proxy_pass_header WWW-Authenticate;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Port $server_port;
    }
}

After applying the above Nginx configuration, ensure these annotations are included in the Ingress definition:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: "800M"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: '120'
    nginx.ingress.kubernetes.io/proxy-read-timeout: '900'
    nginx.ingress.kubernetes.io/proxy-send-timeout: '900'
  name: enterprise-mark
  namespace: frontend
spec:
  rules:
    - host: enterprise-marketing.xxxx.com
      http:
        paths:
          - backend:
              serviceName: enterprise-mark
              servicePort: 80
            path: /
status:
  loadBalancer:
    ingress:
      - ip: 39.107.xxx.xxx

Nginx on Wanzi's Blog – Notes on life, tech, and DevOps.

Resolve Nginx file upload limits and 504 gateway timeout in Kubernetes