Websites are plagued with bad bots and often come grinding to a halt without the aid of a bot blocking tool. Here, I’ll review the latest kid on the block, Spyder Spanker.
First off, Spyder Spanker is a WordPress plugin, so if you don’t have a WordPress powered site then you’re out of luck. If you do however, then read on, it’s pretty impressive. Here’s a video that shows its merits:
Seen enough? Get Spyder Spanker Pro now at the best price or continue reading the review.
Spyder Spanker full review
Initially installing the tool is as simple as uploading your provided zip file to your server via the WordPress plugin page. Once uploaded and activated, Spyder Spanker installs itself as an admin menu item. There, you can add your licence details and you are straight in to the interface, a very neatly styled area where each component is separately presented to you.
Allow trusted bots
Setting up the plugin is very easy because all of the major trusted bots are whitelisted. By trusted, I mean the ones you actually WANT to come to your site such as GoogleBot, BingBot etc. Without these, the search engines wouldn’t know your site content and you would never get listed in the search engine results pages. This is something that is a welcome addition to the software, other packages leave it up to you to select your own trusted bots.
Disallow bad bots
You don’t even need to add any bad bots either because these are also setup when the plugin installs. Bots such as Baidu (Chinese search bot) and Yandex (Russian search bot) are unneccesary on many English language sites as they steal bandwidth to add you to their results pages, regularly returning to your pages to re-crawl them and taking valuable bandwidth.
Allowing individual bots
OK, so we have a good setup straight out of the box, but let’s dig a bit deeper and see what we can modify. Let’s take the scenario where an English language website sells products to China. If this is the case then it would make sense to allow the Baidu bot to index the website. To do this is a simple 2-click operation, tick the Baidu bot and then click on ‘Remove selected’. Very slick and no messing about with CSF firewall rules or .htaccess country-blocking or IP address blocking rules.
The differences between Spyder Spanker and other tools
I wanted to point out that Spyder Spanker is predominantly a bot blocker and doesn’t do a lot of what tools like WP Better Security does such as secure admin areas, make files ineditable etc. What it does excel at is blocking the bots that use your resources on a daily basis and it can throttle back the good bots when they spider your site aggresively too.
WP Better Security comes with a basic list of bad bots for .htaccess (which I use) but they’re a bit more devious these days and use new names. Here is an example of a logfile entry in Spyder Spanker:
Blacklist agent testing... --------------------------------- 0447705 - 2013-05-03 03:12:52 CDT / IP: 208.115.113.90 / HOSTNAME: AGENT: Mozilla/5.0 (compatible; Ezooms/1.0; [email protected]) --------------------------------- BLOCK: blacklist agent: mozilla/5.0 (compatible; ezooms/1.0; [email protected])
This is a bot you won’t generally find on many htaccess blacklists but it’s a ‘rule-breaker’ for sure. SS responded by blocking it and will pass the rule to my other sites and the community network.
Let’s be honest, a bot with a gmail address probably shouldn’t be trusted that much anyway!
Spyder spanker review – the verdict
I’d thoroughly recommend you buy this tool, you’ll recoup any outlay back in a short time with reduced bandwidth fees, time saved and more sales if you run any type of ecommerce or affiliate site. One thing though, go with the Pro upgrade that is presented as a “One Time Offer” when you have paid for the basic version because Spyder Spanker Pro integrates beutifully with Project Honeypot. This means that it can be run ‘hands-off’ and will be updated against the raft of ever-increasing spammers and bad bots out there. Add this to the community update facility and you’ll be protected for years to come across ALL of your domains.