主页 / unix / 问题 / 777550
Accepted
Chris
Asked: 2024-06-01 23:55:59 +0800 CST

机器人爬行获取 301/重定向而不是 404，因此它可以躲避fail2ban。怎么会变成 301 而不是 404？

  • 772

我有fail2ban 设置，它对于大多数扫描都非常有效。它会触发 nginx 错误日志中的任何 4xx。但是，请注意以下机器人扫描。不知何故，这个机器人触发我的服务器返回 301 而不是 404，就像所有其他机器人一样。怎么会这样呢？因为它是 301 而不是 4xx，所以它直接通过了我的fail2ban 并且从未被禁止。我想检测并防止这种情况。

关于如何做到这一点以及如何防止它有什么建议吗？

178.20.44.82 - - [30/May/2024:21:28:48 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "GET /.DS_Store HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.1.15"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "GET /.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:49 +0000] "POST /.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:106.0) Gecko/20100101 Firefox/106.0"
178.20.44.82 - - [30/May/2024:21:28:50 +0000] "GET /.env.prod HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:50 +0000] "POST /.env.prod HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:50 +0000] "GET /.env.production HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:51 +0000] "POST /.env.production HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:51 +0000] "GET /redmine/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:51 +0000] "POST /redmine/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "GET /__tests__/test-become/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "POST /__tests__/test-become/.env HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:106.0) Gecko/20100101 Firefox/106.0"
178.20.44.82 - - [30/May/2024:21:28:52 +0000] "POST / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:53 +0000] "GET /debug/default/view?panel=config HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:53 +0000] "GET /debug/default/view.html HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:53 +0000] "GET /debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /frontend/web/debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /web/debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /sapi/debug/default/view HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:54 +0000] "GET /_profiler/phpinfo HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:55 +0000] "GET /app_dev.php/_profiler/phpinfo HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:55 +0000] "GET /phpinfo.php HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:55 +0000] "GET /owncloud/apps/graphapi/vendor/microsoft/microsoft-graph/tests/GetPhpInfo.php HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
178.20.44.82 - - [30/May/2024:21:28:56 +0000] "GET /info.php HTTP/1.1" 301 178 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:105.0) Gecko/20100101 Firefox/105.0"
178.20.44.82 - - [30/May/2024:21:28:56 +0000] "GET / HTTP/1.1" 301 178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"

我唯一的 301 重定向是 certbot 设置的：

server { if ($host = www.mydomainname.com) 
    { return 301 https://$host$request_uri; } # managed by Certbot 
if ($host = mydomainname.com) 
    { return 301 https://$host$request_uri; } # managed by Certbot
nginx

1 个回答

  1. Best Answer

    该机器人会收到 301 重定向，因为它尝试访问 HTTP 页面，因此您发布的 nginx 配置片段会将其重定向到 HTTPS。

    您可以尝试将 nginx 配置为进行 URL 重写而不是重定向，将 Certbot 生成的代码片段更改为以下代码：

    server {
   listen 80;
   server_name www.mydomainname.com mydomainname.com;
    
   location / {
      rewrite ^ https://$host$request_uri? permanent;
   }
}
    • 1

相关问题