Some old broken Blosxom URL I had created a spider trap on my blog, infinite URLs. But so far Inktomi is the only one dumb enough to fall into it, querying URLs like
I fixed the bug a month ago and have now modified Blosxom to return 404 on these URLs. But Inktomi continues to hit me thousands of times a day.

Spiders are a really dumb way to index the web. Too bad more clever solutions don't work.

  2003-08-02 19:43 Z