About eight months ago I moved my blog to a new domain and set up 301 redirects to point everyone to it. 301 means "moved permanently": all bots should eventually stop hassling the old URL. Does it work? Mostly; here's a list of hits in last week's traffic from various bots.
808 hits, Rojobot
A feed aggregator with several old RSS URLs they apparently have no way of updating.
424 hits, TailRank robot
A service that claims to "find hot stories", in my case by looking at eight-month-old URLs and not following the "hot lead" that the site has moved. Grabs both RSS feeds and some HTML.
390 hits, Feedfetcher-Google
RSS crawler for Google Reader. I'm pretty sure I know the guy responsible for this bot; tsk tsk, B.
254 hits, Yahoo! Slurp and 160 Googlebot
Basic crawlers trying to grab some HTML. Arguably legitimate: they're verifying the 301 redirect is still live.
136 hits, BecomeBot
Some shopping site bot. Guys, I got nothing to sell.
108 hits, AppleSyndication
There's one person who uses the Safari RSS reader! Too bad about the bug.
None of this matters much; the 301 is cheap to serve and other than the Rojo and Safari examples I think no one would care that I stopped serving it. At least the big consumers like BlogLines have all switched over.
techbad
  2006-08-13 16:45 Z