Question

今天有很多来自机器人的请求。如何在Nginx或Fail2ban中阻止它们？

# tail -f -n 100 /var/log/nginx/access.log
176.28.122.158 - - [16/Sep/2017:16:00:16 +0300] "GET /actor/%D0%A2%D0%B8%D0%BC%D0%BE%D1%82%D0%B8%20%D0%A0%D0%B5%D0%B4%D1%84%D0%BE%D1%80%D0%B4 HTTP/1.1" 200 18298 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:16 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:16 +0300] "GET /actor/%D0%99%D0%BE%D1%85%D0%B0%D0%BD%20%D0%A5%D0%B5%D0%BB%D0%B4%D0%B5%D0%BD%D0%B1%D0%B5%D1%80%D0%B3 HTTP/1.1" 200 18390 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:16 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:16 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:16 +0300] "GET /movie/id569071-zhena-smotritelya-zooparka-the-zookeeper-s-wife HTTP/1.1" 200 33660 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:16 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:17 +0300] "GET /movie/id885658-dzhon-uik-2-john-wick-chapter-two HTTP/1.1" 200 32346 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:17 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:17 +0300] "GET /movie/id885658-dzhon-uik-2-john-wick-chapter-two HTTP/1.1" 200 32346 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:17 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:17 +0300] "GET /year/2017 HTTP/1.1" 200 72389 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:17 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:17 +0300] "GET /country/%D0%A1%D0%A8%D0%90 HTTP/1.1" 200 71408 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:17 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:17 +0300] "GET /genre/%D1%82%D1%80%D0%B8%D0%BB%D0%BB%D0%B5%D1%80 HTTP/1.1" 200 73832 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:17 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:17 +0300] "GET /genre/%D0%B1%D0%BE%D0%B5%D0%B2%D0%B8%D0%BA HTTP/1.1" 200 72251 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:18 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:18 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:18 +0300] "GET /genre/%D0%BA%D1%80%D0%B8%D0%BC%D0%B8%D0%BD%D0%B0%D0%BB HTTP/1.1" 200 62785 "-" "Java/1.6.0_24" "-"
176.28.122.158 - - [16/Sep/2017:16:00:18 +0300] "GET /director/%D0%A7%D0%B0%D0%B4%20%D0%A1%D1%82%D0%B0%D1%85%D0%B5%D0%BB%D1%81%D0%BA%D0%B8 HTTP/1.1" 200 17674 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:18 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:18 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:18 +0300] "GET /actor/%D0%9A%D0%B8%D0%B0%D0%BD%D1%83%20%D0%A0%D0%B8%D0%B2%D0%B7 HTTP/1.1" 200 17408 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:18 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:18 +0300] "GET /actor/%D0%A0%D1%83%D0%B1%D0%B8%20%D0%A0%D0%BE%D1%83%D0%B7 HTTP/1.1" 200 17362 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:18 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:18 +0300] "GET /actor/%D0%98%D1%8D%D0%BD%20%D0%9C%D0%B0%D0%BA%D0%A8%D0%B5%D0%B9%D0%BD HTTP/1.1" 200 17454 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:19 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:19 +0300] "GET /actor/%D0%9A%D0%BE%D0%BC%D0%BC%D0%BE%D0%BD HTTP/1.1" 200 17247 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:19 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:19 +0300] "GET /actor/%D0%A0%D0%B8%D0%BA%D0%BA%D0%B0%D1%80%D0%B4%D0%BE%20%D0%A1%D0%BA%D0%B0%D0%BC%D0%B0%D1%80%D1%87%D0%BE HTTP/1.1" 200 17730 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:19 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:19 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:19 +0300] "GET /movie/id885658-dzhon-uik-2-john-wick-chapter-two HTTP/1.1" 200 32346 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:19 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:19 +0300] "GET /type/%D1%84%D0%B8%D0%BB%D1%8C%D0%BC%D1%8B/2 HTTP/1.1" 200 71649 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:19 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:20 +0300] "GET /type/%D1%84%D0%B8%D0%BB%D1%8C%D0%BC%D1%8B/3 HTTP/1.1" 200 51007 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:20 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:20 +0300] "GET /type/%D1%84%D0%B8%D0%BB%D1%8C%D0%BC%D1%8B/4 HTTP/1.1" 200 18296 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:20 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:20 +0300] "GET /type/%D1%84%D0%B8%D0%BB%D1%8C%D0%BC%D1%8B/5 HTTP/1.1" 200 18296 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:20 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:20 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:20 +0300] "GET /movie/id589290-begushii-po-lezviyu-2049-blade-runner-2049 HTTP/1.1" 200 33391 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:20 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:20 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:21 +0300] "GET /movie/id623250-chernaya-pantera-black-panther HTTP/1.1" 200 32793 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:21 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:21 +0300] "GET /content/8-y-sezon-kultovogo-seriala-igra-prestolov-vyydet-ne-ranshe-2019-goda HTTP/1.1" 200 36418 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:21 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:21 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
176.28.122.158 - - [16/Sep/2017:16:00:21 +0300] "GET /content/vankuver-nodovolen-semkami-filma-dedpul-2 HTTP/1.1" 200 35782 "-" "Java/1.6.0_24" "-"
199.168.139.211 - - [16/Sep/2017:16:00:21 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:21 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:21 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:22 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:22 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:22 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:22 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"
199.168.139.211 - - [16/Sep/2017:16:00:22 +0300] "\x03\x00\x00)$\xE0\x00\x00\x00\x00\x00Cookie: mstshash=NCRACK_USER" 400 173 "-" "-" "-"

2012年有人提到这个机器人。

2012年1月11日 - https://twitter.com/mubix/status/157115321155723264

2012年10月3日 - https://twitter.com/mubix/status/253705438581903360

Answer 1

不应该阻止什么是坏方法，而应该允许什么是可接受的好方法

add_header Allow "GET, POST, HEAD" always; 
if ( $request_method !~ ^(GET|POST|HEAD)$ ) { 
   return 405; 
}

如果您的应用使用PUT，PATCH，DELETE，OPTIONS

等方法，则必须向此添加更多方法

或者您可以使用下面提到的方法阻止请求

https://serverfault.com/questions/772833/fail2ban-regex-to-block-x00-requests

如何阻止请求＆＃34; Cookie：mstshash = NCRACK_USER＆＃34;？

1 个答案: