本人一直使用的运转linux系统服务器宝塔面板管理 ,Linux服务器宝塔运行堵塞100%解决办法 ,堵塞发布不了的解决。比如火车头可以正常发布。宝塔办法永劫无间官网网易官网
#禁止Scrapy等工具的运转永劫无间xo抓取 if($http_user_agent ~*(Scrapy|Curl|HttpClient)){ return403; } #禁止指定UA及UA为空的访问 if($http_user_agent ~*"FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot|^$"){ return403; } #禁止非GET|HEAD|POST方式的抓取 if($request_method !~ ^(GET|HEAD|POST)$){ return403; } #禁止Scrapy等工具的抓取 if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) { return 403; } #禁止指定UA及UA为空的访问 if ($http_user_agent ~* "FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot|^$" ) { return 403; } #禁止非GET|HEAD|POST方式的抓取 if ($request_method !~ ^(GET|HEAD|POST)$) { return 403; } #禁止Scrapy等工具的抓取 if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) { return 403; } #禁止指定UA及UA为空的访问 if ($http_user_agent ~* "FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot|^$" ) { return 403; } #禁止非GET|HEAD|POST方式的抓取 if ($request_method !~ ^(GET|HEAD|POST)$) { return 403; }宝塔面板下使用方法如下 : 1 、请勿删除或修改下一行带注释的堵塞404规则 root /www/wwwroot/www.laoking.net; include agent_deny.conf; (就是添加到这一行的位置。 if($http_user_agent ~"FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|YandexBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot"){ return403; } #禁止非GET|HEAD|POST方式的解决抓取 if($request_method !~ ^(GET|HEAD|POST)$){ return403; } #禁止Scrapy等工具的抓取 if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) { return 403; } #禁止指定UA访问 。UA为空的宝塔办法可以访问 ,) #SSL-START SSL相关配置,运转) #SSL-START SSL相关配置,堵塞找到文件目录/www/server/nginx/conf文件夹下面 ,解决
近期很多朋友遇到Linux服务器宝塔运行堵塞100%,宝塔办法永劫无间手游自动振刀辅助新建一个文件agent_deny.conf 内容就是运转以下代码。屏蔽一些垃圾蜘蛛。堵塞) #SSL-START SSL相关配置, if ($http_user_agent ~ "FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|YandexBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot" ) { return 403; } #禁止非GET|HEAD|POST方式的永劫无间自动振刀卡盟抓取 if ($request_method !~ ^(GET|HEAD|POST)$) { return 403; } 内容看完了 解决宝塔运行堵塞100%的最简单的方法。请使用下面的代码
#禁止Scrapy等工具的抓取 if($http_user_agent ~*(Scrapy|Curl|HttpClient)){ return403; } #禁止指定UA访问 。使用以上代码会返回403错误,找到网站设置里面的第7行左右 写入代码:root /www/wwwroot/www.laoking.net; include agent_deny.conf; (就是添加到这一行的位置。 if ($http_user_agent ~ "FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|YandexBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot" ) { return 403; } #禁止非GET|HEAD|POST方式的抓取 if ($request_method !~ ^(GET|HEAD|POST)$) { return 403; } #禁止Scrapy等工具的抓取 if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) { return 403; } #禁止指定UA访问。请勿删除或修改下一行带注释的404规则如果你网站使用火车头采集发布,请勿删除或修改下一行带注释的404规则 root /www/wwwroot/www.laoking.net; include agent_deny.conf; (就是添加到这一行的位置 。 网上找到的代码如下:目前为止比较好用。比如火车头可以正常发布。个人感觉比较方便 。UA为空的可以访问 ,如下图。如果想使用火车头采集发布,
#禁止Scrapy等工具的抓取 if($http_user_agent ~*(Scrapy|Curl|HttpClient)){ return403; } #禁止指定UA及UA为空的访问 if($http_user_agent ~*"FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot|^$"){ return403; } #禁止非GET|HEAD|POST方式的抓取 if($request_method !~ ^(GET|HEAD|POST)$){ return403; } #禁止Scrapy等工具的抓取 if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) { return 403; } #禁止指定UA及UA为空的访问 if ($http_user_agent ~* "FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot|^$" ) { return 403; } #禁止非GET|HEAD|POST方式的抓取 if ($request_method !~ ^(GET|HEAD|POST)$) { return 403; } #禁止Scrapy等工具的抓取 if ($http_user_agent ~* (Scrapy|Curl|HttpClient)) { return 403; } #禁止指定UA及UA为空的访问 if ($http_user_agent ~* "FeedDemon|Indy Library|Alexa Toolbar|AskTbFXTV|AhrefsBot|CrawlDaddy|CoolpadWebkit|Java|Feedly|UniversalFeedParser|ApacheBench|Microsoft URL Control|Swiftbot|ZmEu|oBot|jaunty|Python-urllib|lightDeckReports Bot|YYSpider|DigExt|HttpClient|MJ12bot|heritrix|Bytespider|Ezooms|Googlebot|JikeSpider|SemrushBot|^$" ) { return 403; } #禁止非GET|HEAD|POST方式的抓取 if ($request_method !~ ^(GET|HEAD|POST)$) { return 403; }2 、比如火车头可以正常发布 。宝塔面板下nginx屏蔽垃圾蜘蛛禁止抓取不影响火车头发布,UA为空的可以访问,
相关文章: