1

Since December 24 my server experiences increased resource consumption. Additionally the Nginx server seems to be unstable which causes several errors.

The CPU usage used to average at around 5%. But a few days ago it increased and is now at a constant 10% to 30%. The same behavior holds true for IPv4 traffic. Plus I frequently get errors such as Error 525: SSL handshake failed or Error 500: Internal server error when trying to access my Nextcloud or website. The error messages and the increased traffic arose at the same time. To exclude the possibility of recently done configuration causing the problems, I restored to a Dec 20 backup. So it must have been external influence causing the trouble.


I scanned for viruses using ClamAV on the entire system, but no infected files were found:

----------- SCAN SUMMARY -----------
Known viruses: 8844122
Engine version: 0.103.0
Scanned directories: 28082
Scanned files: 167224
Infected files: 0
Data scanned: 15009.11 MB
Data read: 23880.07 MB (ratio 0.63:1)
Time: 3684.616 sec (61 m 24 s)
Start Date: 2021:01:02 23:54:21
End Date:   2021:01:03 00:55:45

Also I checked for suspicious activity with Netstat:

$ netstat -nt | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -r

   2648 104.218.232.38
   2589 104.218.232.37
    143 5.182.209.124
    143 185.189.14.123
    132 5.182.209.47
    131 54.198.115.81
    121 23.8.7.207
    113 23.224.103.238
     95 185.255.134.153
     64 142.93.135.65
     37 31.206.5.1
     37 134.209.92.79
     37 103.29.71.18
     35 81.70.202.141
     34 194.87.95.95
     28 106.52.158.118
     26 23.32.85.243
     26 116.17.102.163
     25 94.103.87.21
     25 118.193.41.157
     25 111.229.125.162
     24 120.53.118.158
     23 173.249.18.223
     22 81.70.210.159
     22 43.227.180.230
     22 193.109.79.134
     21 139.162.72.45
     21 116.17.102.198
     21 115.238.196.100
     20 159.226.21.39
     20 113.100.209.209
     19 45.84.196.129
     19 173.249.44.200
     19 172.104.118.85
     19 161.97.135.26
     19 113.100.209.120
     18 172.105.240.46
     18 172.105.191.4
     18 172.104.85.88
     17 172.105.35.35
     17 116.17.102.82
     17 116.17.102.251
     17 116.17.102.225
     17 113.100.209.153
     16 61.160.223.228
     16 59.38.222.34
     16 45.118.135.77
     16 207.180.206.180
     16 173.212.225.16
     16 161.97.135.28
     16 139.162.116.216
     16 116.17.102.190
     16 116.17.102.128
     16 113.100.209.91
     15 198.58.96.176
     15 164.68.101.83
     15 116.17.102.71
     15 116.17.102.141
     15 113.100.209.159
     14 176.58.109.91
     14 172.104.127.52
     14 172.104.117.113
     14 139.162.3.85
     14 116.17.102.87
     14 116.17.102.77
     14 113.100.209.9
     14 113.100.209.215
     13 59.38.223.85
     13 42.192.15.120
     13 212.102.60.158
     13 178.151.141.116
     13 116.17.102.117
     13 113.100.209.234
     13 113.100.209.135
     12 195.154.241.248
     12 176.119.156.84
     12 161.97.135.33
     12 161.97.135.32
     12 161.97.135.30
     12 139.162.104.140
     12 113.100.209.207
     11 91.193.173.1
     11 45.118.133.9
     11 207.180.203.143
     11 188.195.109.42
     11 173.212.226.149
     11 172.105.233.224
     11 172.104.98.78
     11 161.97.76.238
     11 161.97.135.224
     11 144.202.8.244
     11 116.17.102.68
     11 116.17.102.237
     11 113.100.209.134
     11 113.100.209.127
     11 113.100.209.119
     11 113.100.209.104
     10 81.70.103.9
     10 59.38.223.115
     10 59.38.223.107
     10 49.12.66.76
     10 191.19.149.198
     10 144.91.114.81
     10 139.162.77.193
     10 127.0.0.1
     10 116.17.102.241
     10 113.100.209.219
      9 59.38.222.171
      9 212.7.210.103
      9 207.180.237.10
      9 178.79.179.193
      9 172.105.237.55
      9 161.97.135.225
      9 139.162.117.120
      9 116.17.102.96
      9 116.17.102.36
      9 116.17.102.142
      9 113.100.209.71
      9 113.100.209.52
      9 113.100.209.235
      9 113.100.209.213
      9 113.100.209.118
      9 113.100.209.115
      9 108.28.122.6
      9 106.53.136.62
      9 103.29.70.181
      8 59.38.223.82
      8 59.38.222.43
      8 27.221.79.31
      8 182.254.223.162
      8 172.105.196.229
      8 164.68.111.16
      8 161.97.135.223
      8 161.97.135.221
      8 161.97.135.220
      8 151.106.3.179
      8 116.17.102.32
      8 116.17.102.254
      8 116.17.102.130
      8 116.17.102.112
      8 113.100.209.147
      7 92.241.9.162
      7 60.169.78.63
      7 212.7.210.104
      7 207.180.213.12
      7 207.180.211.45
      7 164.68.106.182
      7 161.97.135.219
      7 116.17.102.137
      7 113.100.209.80
      7 113.100.209.2
      7 113.100.209.179
      7 113.100.209.125
      7 106.55.53.215
      6 61.184.1.10
      6 59.38.223.238
      6 59.38.222.63
      6 42.48.184.9
      6 221.8.141.164
      6 173.249.20.2
      6 172.105.58.130
      6 172.104.68.177
      6 164.68.108.221
      6 116.17.102.247
      6 116.17.102.223
      6 116.17.102.150
      6 116.17.102.129
      6 113.100.209.69
      6 113.100.209.249
      6 113.100.209.245
      6 113.100.209.169
      5 97.107.137.170
      5 59.38.222.207
      5 47.90.205.159
      5 45.87.2.231
      5 222.180.195.154
      5 180.232.99.133
      5 176.99.159.19
      5 172.105.37.185
      5 172.104.62.99
      5 172.104.173.94
      5 164.68.107.32
      5 154.27.68.105
      5 116.17.102.75
      5 116.17.102.45
      5 116.17.102.172
      5 116.17.102.134
      5 113.100.209.186
      5 113.100.209.181
      5 113.100.209.18
      4 89.108.84.27
      4 82.77.76.92
      4 59.38.222.175
      4 51.103.40.29
      4 204.93.226.69
      4 192.46.233.130
      4 178.63.149.89
      4 173.249.31.254
      4 121.29.46.177
      4 121.29.46.138
      4 118.193.42.237
      4 116.17.102.9
      4 116.17.102.21
      4 113.57.148.194
      4 109.27.192.44
      4 1.193.20.197
      3 81.91.179.207
      3 81.71.42.207
      3 70.37.160.210
      3 59.38.223.98
      3 5.255.183.209
      3 47.90.255.174
      3 47.89.181.151
      3 45.82.68.174
      3 45.12.212.75
      3 36.51.254.229
      3 27.147.202.120
      3 195.2.67.224
      3 185.87.51.122
      3 178.124.185.120
      3 161.97.76.240
      3 139.9.216.230
      3 139.204.122.237
      3 139.204.117.87
      3 136.175.9.57
      3 136.175.9.105
      3 135.148.12.143
      3 116.17.102.20
      3 115.231.218.252
      3 113.100.209.162
      3 113.100.209.140
      3 104.131.180.136
      3 104.128.58.19
      3 103.107.161.129
      2 96.126.118.183
      2 95.217.249.73
      2 94.60.176.83
      2 94.50.240.252
      2 94.198.98.138
      2 94.198.100.8
      2 93.77.19.241
      2 91.236.120.189
      2 81.16.141.51
      2 81.16.141.28
      2 59.38.222.202
      2 51.75.255.151
      2 43.248.186.67
      2 42.192.16.54
      2 39.89.64.117
      2 36.51.254.228
      2 31.135.149.97
      2 3.239.88.227
      2 3.236.246.248
      2 27.159.82.67
      2 27.145.211.135
      2 222.93.16.183
      2 217.182.173.209
      2 203.195.195.235
      2 198.27.100.135
      2 194.67.218.133
      2 188.40.57.143
      2 187.107.10.10
      2 185.81.158.109
      2 183.17.231.237
      2 182.253.176.11
      2 177.47.87.13
      2 173.249.30.9
      2 171.252.189.83
      2 171.107.124.35
      2 163.172.30.116
      2 154.8.246.137
      2 143.244.42.77
      2 143.178.170.214
      2 139.204.117.240
      2 139.155.172.64
      2 122.238.117.25
      2 121.29.46.172
      2 121.29.46.146
      2 118.193.41.84
      2 116.17.102.217
      2 116.17.102.155
      2 115.159.92.188
      2 111.49.79.113
      2 110.249.208.137
      2 104.161.112.234
      2 1.189.60.149
      2 1.183.243.31
      1 servers)
      1 Address
      1 95.216.244.56
      1 95.182.120.9
      1 95.168.183.69
      1 95.141.46.182
      1 95.106.255.97
      1 95.10.232.21
      1 94.249.192.218
      1 94.244.50.10
      1 94.103.90.30
      1 93.204.184.102
      1 92.53.65.210
      1 91.206.15.91
      1 90.225.65.71
      1 88.226.100.225
      1 88.218.16.105
      1 84.64.221.58
      1 82.223.104.78
      1 82.162.58.171
      1 81.69.44.108
      1 8.208.82.133
      1 78.47.32.154
      1 75.109.4.43
      1 74.208.253.135
      1 69.167.7.49
      1 69.164.210.76
      1 66.228.34.13
      1 64.64.250.83
      1 61.145.49.81
      1 59.80.30.164
      1 59.38.222.195
      1 58.58.237.82
      1 51.68.120.72
      1 51.210.43.24
      1 51.178.240.246
      1 51.103.72.158
      1 5.9.215.100
      1 49.232.87.68
      1 47.88.170.127
      1 47.75.190.154
      1 46.91.22.28
      1 46.4.148.26
      1 46.17.43.98
      1 45.91.20.228
      1 45.76.161.122
      1 45.236.149.152
      1 44.242.167.214
      1 42.192.52.67
      1 42.192.138.217
      1 40.120.54.92
      1 39.156.65.236

Then I requested IP information using an IP Geolocation API:

$ curl "http://ip-api.com/line/example_ip_address?fields=country"

And although it should only be accessed by friends, family, teachers and myself, it's pinged from all around the world. It seems to be getting hundreds/thousands of requests from China, Singapore, Bangladesh, Vietnam, Russia, France, United States, Netherlands and more.

I also checked for invalid logins in /var/log/auth.log. There were multiple attempts to login to my server using usernames that don't exist on the system.

# grep "Invalid user" /var/log/auth.log

Jan  1 10:09:54 server sshd[20560]: Invalid user jake from 117.247.183.216 port 59544
Jan  1 10:11:18 server sshd[20637]: Invalid user pydio from 106.12.97.115 port 36824
Jan  1 10:26:14 server sshd[21278]: Invalid user ts3 from 106.124.136.227 port 43942
Jan  1 11:03:58 server sshd[22909]: Invalid user test1 from 37.114.36.172 port 41906
Jan  1 11:04:00 server sshd[22912]: Invalid user paco from 67.205.142.48 port 40838
Jan  1 11:05:50 server sshd[22998]: Invalid user trade from 114.207.139.203 port 32833
Jan  1 11:07:43 server sshd[23084]: Invalid user teamspeak from 61.155.106.101 port 55632
Jan  1 11:11:05 server sshd[23265]: Invalid user maria from 81.68.83.82 port 49822
Jan  1 11:14:55 server sshd[23434]: Invalid user ts3user from 51.68.226.27 port 57540
Jan  1 11:22:02 server sshd[23737]: Invalid user dave from 43.226.69.100 port 45332
Jan  1 11:53:54 server sshd[25138]: Invalid user pi from 188.76.66.65 port 23060
Jan  1 11:53:54 server sshd[25139]: Invalid user pi from 188.76.66.65 port 22840
Jan  1 13:19:49 server sshd[28963]: Invalid user csgoserver from 61.93.240.18 port 1665
Jan  1 13:23:22 server sshd[29130]: Invalid user hxeadm from 178.128.80.85 port 39950
Jan  1 13:25:05 server sshd[29187]: Invalid user mcserver from 195.29.102.42 port 42286
Jan  1 13:28:52 server sshd[29354]: Invalid user felix from 37.252.190.224 port 59594
Jan  1 13:30:52 server sshd[29440]: Invalid user dinesh from 81.183.213.37 port 60185
Jan  1 13:41:13 server sshd[29920]: Invalid user testuser from 161.82.130.186 port 39300
Jan  1 13:41:48 server sshd[29957]: Invalid user ranger from 106.124.136.227 port 34749
Jan  1 13:46:34 server sshd[30171]: Invalid user vbox from 115.159.161.81 port 36826
Jan  1 13:51:11 server sshd[30352]: Invalid user admin2 from 105.73.83.18 port 36252
Jan  1 13:52:32 server sshd[30428]: Invalid user test from 51.210.5.171 port 54958
Jan  1 13:57:08 server sshd[30609]: Invalid user pmd from 185.234.219.5 port 15368
Jan  1 14:09:00 server sshd[31116]: Invalid user ftpadmin from 111.229.181.50 port 35512
Jan  1 14:13:01 server sshd[31338]: Invalid user maximo from 112.196.43.202 port 42158
Jan  1 14:20:54 server sshd[31680]: Invalid user www from 51.38.70.175 port 60434
Jan  1 15:06:16 server sshd[1391]: Invalid user rd from 49.235.11.137 port 36864
Jan  1 15:19:07 server sshd[1996]: Invalid user roberto from 45.155.205.86 port 44624
Jan  1 15:48:27 server sshd[3277]: Invalid user dennis from 123.58.109.42 port 40322
Jan  1 15:50:35 server sshd[3365]: Invalid user deploy from 106.52.22.230 port 48356
Jan  1 15:52:42 server sshd[3454]: Invalid user admin1 from 122.152.215.115 port 37214
Jan  1 16:05:15 server sshd[3976]: Invalid user user from 195.19.102.173 port 45690
Jan  1 16:12:21 server sshd[4322]: Invalid user git from 118.145.8.50 port 56276
Jan  1 16:51:57 server sshd[6066]: Invalid user ubuntu from 157.231.102.250 port 51841
Jan  1 16:54:17 server sshd[6157]: Invalid user hdfs from 51.77.230.49 port 36038
Jan  1 16:54:29 server sshd[6161]: Invalid user rabbit from 165.22.234.248 port 39244
Jan  1 17:47:33 server sshd[9479]: Invalid user pi from 182.84.124.120 port 50662
Jan  1 17:47:33 server sshd[9480]: Invalid user pi from 182.84.124.120 port 50660
Jan  1 18:09:04 server sshd[10427]: Invalid user test1 from 130.61.134.151 port 58688
Jan  1 18:24:56 server sshd[1387]: Invalid user botuser from 179.131.11.234 port 45754
Jan  1 18:53:49 server sshd[3748]: Invalid user jenkins from 157.230.97.148 port 47838
Jan  1 18:55:20 server sshd[3830]: Invalid user dlwsadmin from 157.230.97.148 port 49102
Jan  1 18:56:50 server sshd[3881]: Invalid user ascend from 157.230.97.148 port 50382
Jan  1 18:58:15 server sshd[3958]: Invalid user dlwsadmin from 157.230.97.148 port 51648
Jan  1 18:59:37 server sshd[4009]: Invalid user ascend from 157.230.97.148 port 52920
Jan  1 19:10:21 server sshd[4539]: Invalid user es from 157.230.97.148 port 34834
Jan  1 19:11:43 server sshd[4590]: Invalid user dolphinscheduler from 157.230.97.148 port 36114
Jan  1 19:57:54 server sshd[1466]: Invalid user bserver from 106.55.41.76 port 33176
Jan  1 19:58:11 server sshd[1500]: Invalid user www from 62.171.157.83 port 64476
Jan  1 19:58:41 server sshd[1507]: Invalid user tom from 86.61.70.243 port 51011
Jan  1 20:00:10 server sshd[1589]: Invalid user admin1 from 150.158.175.66 port 41138
Jan  1 20:09:33 server sshd[2039]: Invalid user guest3 from 49.234.24.246 port 39462
Jan  1 20:09:42 server sshd[2035]: Invalid user upload from 13.82.0.138 port 34294
Jan  1 20:43:07 server sshd[3522]: Invalid user pi from 212.68.244.157 port 45541
Jan  1 20:43:07 server sshd[3521]: Invalid user pi from 212.68.244.157 port 45542
Jan  1 20:54:24 server sshd[3993]: Invalid user support from 185.156.74.65 port 8975
Jan  1 20:54:24 server sshd[3995]: Invalid user support from 185.156.74.65 port 9161
Jan  1 21:04:18 server sshd[4437]: Invalid user ansible from 167.99.210.58 port 51446
Jan  1 21:04:26 server sshd[4441]: Invalid user ansible from 167.99.210.58 port 37472
Jan  1 21:04:59 server sshd[4484]: Invalid user butter from 167.99.210.58 port 37914
Jan  1 21:05:17 server sshd[4496]: Invalid user dev from 167.99.210.58 port 39260
Jan  1 21:05:26 server sshd[4498]: Invalid user user from 167.99.210.58 port 53592
Jan  1 21:12:36 server sshd[4857]: Invalid user sdtdserver from 36.250.229.84 port 50448
Jan  1 21:14:35 server sshd[4943]: Invalid user uftp from 107.175.153.27 port 36842
Jan  1 21:15:39 server sshd[4997]: Invalid user testa from 45.64.184.140 port 51020
Jan  1 21:16:47 server sshd[5042]: Invalid user teamspeak from 113.250.0.149 port 44582
Jan  1 21:21:01 server sshd[5247]: Invalid user jenkins from 167.172.195.99 port 36110
Jan  1 21:39:47 server sshd[6068]: Invalid user devel from 118.24.123.34 port 36368
Jan  1 21:49:22 server sshd[6489]: Invalid user debian from 129.226.225.117 port 33020
Jan  1 21:54:08 server sshd[6670]: Invalid user weblogic from 3.138.200.187 port 40742
Jan  1 21:54:17 server sshd[6705]: Invalid user spravce from 45.155.205.87 port 49303
Jan  1 21:56:04 server sshd[6765]: Invalid user smbuser from 167.172.185.34 port 37432
Jan  1 21:56:36 server sshd[6802]: Invalid user hadoop from 130.61.100.68 port 52070
Jan  1 21:57:38 server sshd[6846]: Invalid user devel from 212.64.71.254 port 55110
Jan  1 21:59:49 server sshd[6935]: Invalid user debian from 174.88.178.92 port 46002
Jan  1 22:07:14 server sshd[7269]: Invalid user ubuntu from 45.148.10.54 port 2536
Jan  1 22:17:13 server sshd[8069]: Invalid user samba from 45.155.205.87 port 15070

It should not be possible for anyone but me to login though, because I've hardened SSH access a long time ago, by limiting action on port 22 using ufw, by installing Fail2ban and by only allowing access using a private authentication key and a password, both of which only I have. I also don't see successful logins by anyone else but me when running the last command. Furthermore I put my server behind Cloudflare to protect it against DDoS attacks, which didn't help to solve the issues.

I also checked the Nginx error log at /var/log/nginx/error.log and it lists the alert 768 worker_connections are not enough over and over, because I've only configured one worker process with 768 worker connections. That would actually be enough for my use case, if the server wasn't attacked/probed by bots. Should I try to increase the number of worker connections anyway?

Thanks in advance!


Update

I just reviewed the Nginx access log at /var/log/nginx/access.log. This is a small sample of it's contents:

5.45.74.22 - - [04/Jan/2021:00:01:27 +0100] "POST http://5.188.211.72/check.php HTTP/1.1" 200 1161 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)"
172.104.98.78 - - [04/Jan/2021:00:01:27 +0100] "GET https://wesley.kunlun301.com/?u=http:// HTTP/1.1" 200 292 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; AdsBot-Google-Mobile; +http://www.google.com/mobile/adsbot.html)"
103.29.71.18 - - [04/Jan/2021:00:01:27 +0100] "GET https://wesley.kunlun301.com/?u=http:// HTTP/1.1" 500 588 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3599.0 Safari/537.36"
172.104.68.177 - - [04/Jan/2021:00:01:27 +0100] "GET http://console.bestacdn.com:1122/?u=http:// HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Linux; Android 5.0; SM-G920A) AppleWebKit (KHTML, like Gecko) Chrome Mobile Safari (compatible; AdsBot-Google-Mobile; +http://www.google.com/mobile/adsbot.html)"
45.118.135.77 - - [04/Jan/2021:00:01:27 +0100] "GET http://wesley.kunlun301.com/?u=http:// HTTP/1.1" 200 292 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/18.18247"
176.58.109.91 - - [04/Jan/2021:00:01:27 +0100] "GET http://console.bestacdn.com:1122/?u=http:// HTTP/1.1" 499 0 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
172.105.35.35 - - [04/Jan/2021:00:01:27 +0100] "GET http://wesley.kunlun301.com/?u=http:// HTTP/1.1" 499 0 "-" "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
62.113.115.240 - - [04/Jan/2021:00:01:27 +0100] "CONNECT steamcommunity.com:443 HTTP/1.1" 400 166 "-" "-"
121.57.146.76 - - [04/Jan/2021:00:01:27 +0100] "CONNECT production-game-api.sekai.colorfulpalette.org:443 HTTP/1.1" 400 166 "-" "-"
139.162.116.216 - - [04/Jan/2021:00:01:27 +0100] "GET http://wesley.kunlun301.com/?u=http:// HTTP/1.1" 499 0 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
172.104.173.94 - - [04/Jan/2021:00:01:27 +0100] "CONNECT m.facebook.com:443 HTTP/1.1" 400 166 "-" "-"
172.104.127.52 - - [04/Jan/2021:00:01:27 +0100] "GET https://wesley.kunlun301.com/?u=http:// HTTP/1.1" 200 292 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Safari/537.36"
61.136.101.153 - - [04/Jan/2021:00:01:27 +0100] "CONNECT www.alipay.com:443 HTTP/1.0" 400 166 "-" "-"
193.109.79.134 - - [04/Jan/2021:00:01:27 +0100] "GET http://api.steampowered.com/IPlayerService/GetSteamLevel/v1/?key=682AA980899BA2C3A331538849BBC8D4&steamid=76561198013106964 HTTP/1.1" 200 52 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0"

Are these requests to be expected? They seem to cause errors in /var/log/nginx/error.log.

2
  • 2
    You should take a look at the nginx logs, and see what kind of requests you're getting and what errors they're generating. It's likely that some botnet just noticed your server, added it to their target list, and are now throwing various probes (just like the ssh logs indicate) at you. But without examining the logs that's just a guess. Commented Jan 3, 2021 at 5:55
  • @GordonDavisson I checked /var/log/nginx/error.log and it lists the alert 768 worker_connections are not enough, meaning that bots are actually probing my server, right? Should I try to increase the number of worker connections or the number of worker processes? Commented Jan 3, 2021 at 16:04

1 Answer 1

0

First of all thanks to @Giacomo1968 and @GordonDavisson for pointing me in the right direction. After making sure my server wasn't infected with malware and it's SSH access was hardened, I configured Nginx to deal with requests by bots that resulted in a DDoS. The configuration file is usually located at /etx/nginx/nginx.conf. A good resource is this guide to DDoS mitigation using Nginx.


Increasing the number of worker connections

I increased the maximum number of simultaneous connections (worker connections) that can be opened by a worker process (e.g. 2048).

worker_connections 2048;

Limiting the Rate of Requests

I limited the rate at which Nginx accepts incoming requests to a value typical for real users (e.g. 2 seconds).

limit_req_zone $binary_remote_addr zone=one:10m rate=30r/m;

server {
    # ...
    location / {
        limit_req zone=one;
        # ...
    }
}

Limiting the Number of Connections

I limited the number of connections that can be opened by a single client IP address, again to a value appropriate for real users (e.g. 10).

limit_conn_zone $binary_remote_addr zone=two:10m;

server {
    # ...
    location / {
        limit_conn two 10;
        # ...
    }
}

Closing Slow Connections

I configured Nginx to close connections that are writing data too infrequently, which can represent an attempt to keep connections open as long as possible (thus reducing the server’s ability to accept new connections). Slowloris is an example of this type of attack.

server {
    client_body_timeout 5s;
    client_header_timeout 5s;
    # ...
}

Now my Nginx server still uses a bit more resources than before the attacks started, but at least it isn't overloaded anymore. I hope this is helpful for other people facing similar attacks.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .