V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
xiaop
V2EX  ›  站长

Sogou的蜘蛛怎么这么疯狂?堪比DDoS了

  •  
  •   xiaop · 2013-12-24 23:11:30 +08:00 · 5540 次点击
    这是一个创建于 4019 天前的主题,其中的信息可能已经有所发展或是发生改变。
    66.249.64.1 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /redirect.php?goto=findpost&pid=567887 HTTP/1.0" 302 20 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    218.30.103.97 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /topic/144666.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    220.181.89.164 - - [24/Dec/2013:23:10:56 +0800] bbs.quxizang.net "GET /topic/6636/last/ HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.87 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /member/154806.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    61.135.189.124 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/167728.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.157 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/140326/p2.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.148 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/29477.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    66.249.64.2 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/178887.html HTTP/1.0" 200 7465 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    218.30.103.132 - - [24/Dec/2013:23:10:57 +0800] bbs.quxizang.net "GET /topic/22570.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    66.249.64.4 - - [24/Dec/2013:23:10:57 +0800] m.quxizang.net "GET /bbs/t7730.html HTTP/1.0" 200 2151 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    218.30.103.154 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/119864.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    61.135.189.70 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/112342.html HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    218.30.103.152 - - [24/Dec/2013:23:10:58 +0800] bbs.quxizang.net "GET /topic/29939/ HTTP/1.0" 403 4571 "-" "Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07)"
    7 条回复    1970-01-01 08:00:00 +08:00
    xiaop
        1
    xiaop  
    OP
       2013-12-24 23:15:36 +08:00
    其中一台小服务器已经挂掉,赶紧屏蔽了搜狗蜘蛛,日志是屏蔽后的。
    zhouzb889
        2
    zhouzb889  
       2013-12-24 23:22:24 +08:00
    曾经被这鬼东西消耗完流量,果断拒绝。
    它的Agent包括:Sogou web spider,Sogou News Spider,New-Sogou-Spider

    在DNS设置中解析到一个不存在的IP貌似也是可行的。

    另外,Sosospider也很疯狂!
    binux
        3
    binux  
       2013-12-24 23:28:05 +08:00
    又到年底覆盖率评估的时候了
    juicy
        4
    juicy  
       2013-12-24 23:38:56 +08:00
    。。。。我最近还刻意去添加了搜狗的SEO。。。。。悲剧。。。。
    katekiss
        5
    katekiss  
       2013-12-24 23:42:48 +08:00
    哎,不行就是不行
    solo7net
        6
    solo7net  
       2013-12-25 09:34:15 +08:00
    我的带宽被百度跑满过
    jjplay
        7
    jjplay  
       2013-12-25 10:41:13 +08:00
    藏ICP备很少见的啊,LZ西藏汉子啊
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   5650 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 36ms · UTC 03:09 · PVG 11:09 · LAX 19:09 · JFK 22:09
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.