I Trapped a Bot
Friday, November 12, 2004 at 2:46 am | 
Comments off
Seems I managed to trap a bot. It hit one of my files over 14,000 times in the past two days. No worries, though, I just added this line to my .htaccess file:
# DENY THE TROUBLESOME BOT! deny from 64.124.85
As far as I can tell, the hits seem to be coming from a place called Exava. Hmm... maybe they should work on that - their bot isn't going to get very far if it allows itself to hit the same file 14,000 time.
Comments
Well, looks that this search engine will be more up-to-date than Google, if it's crawling that often! :P
I've been hit by the Exava bot twice in the last week - it seems to have a problem realising that it's on the same page (on my site it got stuck on the calendar, going all the way back to 1981 one month at a time!)
I've been visited by that but more near the 200 hit mark, which is neither here nor there IBM tends to hit more than that for my site anyway.
I haven't had the problem of it being stuck though.
Is this the same as "exabot"? I keep getting hit lately by some bot from ng20.exabot.com. When I try http://www.exabot.com, it forwards me to http://beta.exalead.com/search. I suppose it's some new search engine...
Umm, I just noticed that they are based in Mountain View, CA. Isn't that the same city as Google based in? Whats up with that?
Just a note, Mountain View, CA is where Google is based. Headquarters are like 5 miles from my house. As far as exabot goes... this should answer just about all of your questions
"Exabot is the user-agent for Exava's new web crawler. Exava is crawling the web to build a next generation search engine."
hardly true now is it
Comments are automatically closed after 45 days
November 12th, 2004
6:07 AM | #
Umm, Maybe they could work on that a bit, eh? I don't think they want their bots hitting that much ;).