金关村村长 发表于 2011-7-26 18:36:05

twitter上面的链接google不来爬啊

184.72.46.206 - - HEAD /technology-09-93892.html HTTP/1.1 200 0 - Mozilla/5.0 (X11; U; Mac OSX; it; rv:1.9.0.7) Gecko/2009030422 Firefox/3.0.7 -
166.90.142.9 - - GET /technology-09-93892.html HTTP/1.1 200 15476 - Voyager/1.0 -
50.18.121.47 - - HEAD /technology-09-93892.html HTTP/1.1 200 0 - Mozilla/5.0 (X11; U; Mac OSX; it; rv:1.9.0.7) Gecko/2009030422 Firefox/3.0.7 -
199.59.149.166 - - GET /technology-09-93892.html HTTP/1.1 200 6590 - Twitterbot/1.0 -
204.236.161.173 - - GET /technology-09-93892.html HTTP/1.1 200 15476 - JS-Kit URL Resolver, http://js-kit.com/ -
204.236.254.109 - - HEAD /technology-09-93892.html HTTP/1.1 200 0 - PostRank/2.0 (postrank.com) -
199.59.149.10 - - HEAD /technology-09-93892.html HTTP/1.1 200 0 - Twitterbot/0.1 -

发了大约一百条，来爬的爬虫有twitterbot还有不知名的或者可能采集者的bot

google的没来
004.gif

004.gif

004.gif

004.gif



twitter刺激收录？怎么我的不行，娘亲的

hepac 发表于 2011-7-26 20:39:35

"

你已被识别为SPAMMER

qianli 发表于 2011-7-27 05:03:54

你sb了吧
页: [1]
查看完整版本: twitter上面的链接google不来爬啊