How to Block 360Spider (the web robot)

rage image by http://www.flickr.com/photos/istolethetv/



360Spider was once again TOO demanding

I draw the line at webrobots that frequently morph their user agent names. Spider360 really got out of control today.
Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.0.11) Gecko/20070312 Firefox/1.5.0.11; 360Spider

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; MDDR; .NET4.0C; .NET4.0E; .NET CLR 1.1.4322; Tablet PC 2.0); 360Spider

Blocked with StreamCatcher

I blocked the bot using StreamCatcher ISAPI Filter for IIS.
Two configuration changes were required.
[SCWebSiteConfig_wwwMySite.cfg]
-----------------------------------------------------
StreamCatcher WebSite Configuration File for wwwMySite
-----------------------------------------------------

[wwwMySite.Settings]
(snip)
UserAgentBlocking=Enabled

 
[wwwMySite.UserAgentsToBlock]
1={..}360Spider{..}
Then IISRESET, test with Firefox and User Agent Switcher to make sure it worked.... now all requests from any user agent containing 360Spider will receive NoResponse nada nothing http connection times out.

How to Block for ALL WebSites

Work at the Master configuration level.

SCConfig_Master.cfg

[GlobalSettings]
UserAgentBlocking=Enabled

[UserAgentsToBlockForAllWebSites]
1={..}360Spider{..}

Have a better day.

Comments

Popular Posts