Some old broken Blosxom URL I had created a spider trap on my blog, infinite URLs. But so far Inktomi is the only one dumb enough to fall into it, querying URLs like
/~nelson/weblog/tech/Value%20Added%20<something>.html/I fixed the bug a month ago and have now modified Blosxom to return 404 on these URLs. But Inktomi continues to hit me thousands of times a day.
Spiders are a really dumb way to index the web. Too bad more clever solutions don't work.