I Trapped a Bot

Thursday, November 11, 2004 at 6:46 pm | Comments off

Let my content go!

Seems I managed to trap a bot. It hit one of my files over 14,000 times in the past two days. No worries, though, I just added this line to my .htaccess file:

# DENY THE TROUBLESOME BOT!

deny from 64.124.85

As far as I can tell, the hits seem to be coming from a place called Exava. Hmm... maybe they should work on that - their bot isn't going to get very far if it allows itself to hit the same file 14,000 time.

Comments

Jordon
November 11th, 2004
10:07 PM | #

Umm, Maybe they could work on that a bit, eh? I don't think they want their bots hitting that much ;).

Daniel
November 12th, 2004
12:14 AM | #

Well, looks that this search engine will be more up-to-date than Google, if it's crawling that often! :P

Matthew Pennell
November 12th, 2004
3:56 AM | #

I've been hit by the Exava bot twice in the last week - it seems to have a problem realising that it's on the same page (on my site it got stuck on the calendar, going all the way back to 1981 one month at a time!)

Robert Wellock
November 12th, 2004
4:18 AM | #

I've been visited by that but more near the 200 hit mark, which is neither here nor there IBM tends to hit more than that for my site anyway.

I haven't had the problem of it being stuck though.

Ray
November 12th, 2004
9:13 AM | #

Is this the same as "exabot"? I keep getting hit lately by some bot from ng20.exabot.com. When I try http://www.exabot.com, it forwards me to http://beta.exalead.com/search. I suppose it's some new search engine...

Jordon
November 12th, 2004
11:35 AM | #

Umm, I just noticed that they are based in Mountain View, CA. Isn't that the same city as Google based in? Whats up with that?

mastab
November 24th, 2004
12:56 PM | #

Just a note, Mountain View, CA is where Google is based. Headquarters are like 5 miles from my house. As far as exabot goes... this should answer just about all of your questions

Rich Turner
December 7th, 2004
12:08 PM | #

"Exabot is the user-agent for Exava's new web crawler. Exava is crawling the web to build a next generation search engine."

hardly true now is it

Comments are automatically closed after 45 days