Canon has a yearly amazing rebate program; if you buy three qualifying items you get a 10-30% rebate on all of them. Rebates cover many things (including camera bodies), but you have to buy things by January 15. Here's a list of all the prime lenses that are covered by the program. Prices listed are the cheapest at Amazon, rebate is the maximum 3x rebate.

Length	Aperture	Weight	Retail	Rebated
20	f/2.8	405g	$420	$375
28	f/1.8	310g	$400	$355
50	f/1.4	290g	$315	$270
85	f/1.8	425g	$340	$295
100	f/2.0	460g	$390	$345
135	f/2.0L	750g	$890	$785
200	f/2.8L	765g	$660	$585
300	f/4L	1190g	$1150	$1015
400	f/5.6L	1250g	$1100	$980

All lenses have ring USM autofocus except the 50 which is micro USM. The 300mm is also image stabilized. More data on the Canon lens chart PDF.

I'm learning how to effectively give to charities. This year I did a fair amount of research to find efficient organizations on GuideStar and Charity Navigator. I also asked Mefi for some ideas for specific charities. Then I made donations via American Express: it's convenient, has good research, and has a reasonably low transaction fee.

I hope it's not tacky to talk about this, but in trying to figure out what and where to give I had a hard time. Most of my friends don't talk about this kind of thing, so I thought maybe this blog post would help the conversation. Here's the charities I came up with:

Services for those in need

Homeward Bound of Marin: homeless services
San Francisco Food Bank
St. Anthony Foundation: homeless services
ModestNeeds.org: microcharity
My Two Front Teeth: Christmas toys
San Francisco AIDS Foundation
Save the Children: international aid
American Red Cross: Hurricane Katrina

Services I use

Wikimedia Foundation
KQED: public radio
Reed College: my alma mater

Social causes

Human Rights Watch
ACLU Foundation
The Point Foundation: gay student scholarships
Electronic Frontier Foundation

Google has a great employee gift matching program, which is part of my impetus to put this all together. Mostly it feels good to do. I wonder whom I've forgotten.

Christmas, the magical time of year when jolly ol' Santa Bezos and his elves in brown sleighs come from the Amazon and deliver presents to all the good children.

The Dell 2405FPW is good hardware. Dell continues to make excellent LCDs at reasonable prices. The 2405 is much like the 2001, only 24" wide and 1920x1200. It has a good "scale but preserve aspect ratio" option for 4x3 video sources.

My only complaint is that it's really bright with a gamma of about 2.0 Photos look a bit washed out. I still haven't figured out which of three different calibration techniques I should use to get this under control, but I'm about to do some serious photo editing so I need to figure it out.

If you're thinking about buying a new LCD you may want to hold off a couple of weeks. Dell's about to introduce the 2407FPW. It's major feature is the evil HDCP, but the new model may make the 2405FPW cheaper. They're also going to inrtoduce the 3007WFP, a 30" LCD at 2560x1600. That sounds really cool but honestly the 24" is already too big. The viewing angles mean that the screen doesn't appear to be constant brightness.

One practical nuisance with widescreen LCDs; many full screen games don't use the extra width. At worse you just play with black bars on each side, for more info see widescreen gaming forum. I've compounded the problem by hooking my other LCD up as a second monitor, 1200x1600. Everything I've run still works, but some programs get a bit confused.

Webmasters get anxious when they move a website for fear they'll lose visibility in search engines. I'm glad to say that in my move the three major search engines all did a great job following my blog to the new URL.

The key thing that made this move work was getting the 301 redirects right. When you move a web page the old URL should still work, but instead of serving content it should return a 301 Moved Permanently pointing at the new URL. I did this with Apache mod_rewrite, specifying the flag [R=301]. It took a bit of fiddling to redirect both hostname and path in a single response.

With the redirect in place everyone quickly started indexing the new URL. The first search engine to find the new host was Yahoo, which had 47 pages for my new hostname in less than 15 hours. Google had the new site in about 24 hours. MSN got it in about 48 hours. Now, 2 weeks later, all three engines have a fairly good collection of my pages at the new URL. Google and MSN now favour the new hostname but Yahoo still has the majority of pages from the old name.

I'm also curious about when the search engines start forgetting the old host. Google decided my canonical URL was the new host in about 36 hours. MSN switched in about 3 days, although it took awhile for the new URL to float to the top of the search results. Yahoo took about 12 days to switch. Of course with my redirect in place, people find my blog either way.

So what's the bottom line? 301 redirects worked for me and all the major search engines did a good job of following mine. Search engines are complicated; it's possible something I haven't thought to check didn't work. But folks can find my blog at the new URL, so I'm happy. And it's pretty amazing how fast this update happens. My blog is only one of billions of web pages out there; search engines are able to track changes like mine across the entire web in just a few days.

Showtime's got great shows this year; first Weeds and now Sleeper Cell. On the surface it's a spinning-plates anti-terrorist drama in the vein of 24. Which makes for fun TV. But Sleeper Cell has more depth than that, particularly in the subtlety of the characterizations.

The script borrows from recent history to build a motley crew of terrorists intent on carrying jihad to America. Blake Shields is great as the unfortunate John Walker Lindh type. Alex Nesic is fun as the French Islamic radical who enjoys hookers and strippers. Both good guys and bad are drawn with complexities and imperfections that make the story compelling.

But the real standout is Michael Ealy. Mild spoiler, highlight to read: He does a great job as the conflicted undercover agent. By making our hero himself be a Muslim the show gives a much more nuanced story than the usual "kill the Muslim terrorists" shtick. He's also a fantastic actor, and quite handsome, so it all makes for good entertainment.

The show just finished its 10 hour miniseries run, but Showtime promises a rebroadcast January 10. And it looks like a DVD release is planned, too.

When I was nine I wanted an Atari 2600 more than anything. I pestered my mom all fall for an Atari at Christmas, but I knew this was a bit expensive for our family. My mom just kept saying no, we can't afford that, I'll get you something nice and you can play with your friends' Ataris.

But my mom found a way and bought the Atari mid-December and hid it in the house. I know this because my sister hinted just a little bit and I figured it out and prowled around the scary garage closet until I found it. I was so excited that I couldn't wait until Christmas, so I told my Mom I'd found it. In my nine year old logic that meant she'd give it to me early.

My Mom was devastated that her surprise was ruined. She took Christmas seriously, doubly so with such an expensive gift. I remember her crying. I remember her getting mad and telling me she was going to take the Atari back to the store since I'd ruined the gift. And she did. I went sneaking around the house and couldn't find it. Christmas Eve all my presents were under the tree, but no box big enough for an Atari.

I was sad, but I was well enough behaved not to sulk at Christmastime. Santa had come and there was candy in the stockings, and mom made a special breakfast, and all was fine. Then I opened my smallest present, and it was an Atari 2600 joystick. My second smallest present was an Atari cartridge. And so on. My mom had broken the Atari up into lots of little boxes and wrapped each one individually. I was the happiest kid ever.

I just went to myspace for the first time. I've still never been to facebook. I don't know anyone who has pages on either site. I normally pride myself on being hip to Internet culture, but I think the lowest on the age curve I go is LiveJournal. I wonder what the natural generational groupings on the Internet are? I naïvely thought there wouldn't be any, but of course the net is all about community and culture and, therefore, generation.

The Xbox 360 is a dry run for "Trusted Computing". The new console has a slew of anti-modder technology to make it impossible to run unauthorized software. Who authorizes Xbox 360 software? Microsoft.

Intel, Microsoft, IBM, etc are colluding to bring the same sort of technology to PCs to prevent you from using unauthorized software. Who authorizes software? Not you.

It's a sure bet "Trusted Computing" is going to bring us a new level of digital rights management hell. Sony won't have to do something stupid to stop you from legally copying music; the operating system and processor will do it for them. Your monitor will collaborate. Dell is about to ship LCDs with HDCP so that your screen won't display unauthorized video. Who authorizes video? Not you.

If all this byzantine technology just made it harder for people to steal music and movies I wouldn't mind. But the collateral damage is too great. The new wave of hardware-assisted DRM means that your PC is going to be a jail full of complex interior locks that prevent you from doing anything that some pointy-headed copyright holder doesn't think you should do. I should have the right to authorize the software that runs on my computers. I won't.

The old Xbox tried to stop modders too, but they got it wrong. It remains to be seen how hard it will be to mod the Xbox 360 or the new PC systems. I fear once the restrictions are in the hardware it will be very hard to fix.

I finally settled on a domain name for myself. I've set up a bunch of redirects to move my blog to its new official home, http://www.somebits.com/weblog/. The old ~nelson was all nice and Web 0.9, but it's time to move on. Please update your links so Googlebot has no trouble finding me! The linkblog moved, too.

If you notice any problems, please mail me.

This seems like a good time to give a tip of the hat to Drrt, Dugsong, and the other good folks of monkey.org whose name I've been using for so long. They're a neat geek collective coming originally from University of Michigan and number among them some of the best Unix security hackers out there. I'll continue to do various things over there, but it's nice to have a name of my own too.

Between being in Switzerland for three months and all the new games out for the Christmas season I have a giant backlog of interesting computer games to play. Should last me for six months! If you're thinking of a gift for the gamer geek in your life, maybe this list will help you.

Shadow of the Colossus (PS/2)

Poetic and innovative platform game for the PS/2. I've played a bit and it's really beautiful.

Civilization IV (PC)

Empire building and wargames, many hours time sink.

Burnout: Revenge (Xbox)

Arcade racer; not clear it add much to last year's standout Takedown, but it's fun.

Battlefield 2: Special Forces (PC)

BF2 is the only online FPS worth playing. This expansion pack is getting mixed reviews, but it's sort of unavoidable.

The Warriors (Xbox)

Cheesy 70s gang movie made into a Rockstar ultraviolence game. On a quick play it's pretty damn silly, but amusing.

Puzzle Pirates (PC, Mac)

I actually got into this while still in Zürich. It's a very innovative game, a cross between an MMOG and a bunch of Tetris-like puzzle games. Pleasant diversion.

Sniper Elite (PC, Xbox, PS/2)

Sniper simulation. The reviews are pretty poor, but I'm a sucker for this kind of gameplay.

Rag Doll Kung Fu (PC)

Indie game excitement of the year, distributed online via Steam. I haven't seen a review yet, not sure if it's worth my time.

What's missing from this list is a good story based RPG. I haven't seen one this Christmas season. It's a shame Oblivion got delayed.

I wish today's MMOGs lived up to the promise of player created content that was in the earliest online games. LambdaMOO set the pace. Pavel Curtis had a fanatical devotion to player-generated content. The whole world was player created. Players could build rooms and program new behaviours in MOO. Players even came up with their own government.

The first commercial MMOGs continued to let players shape the world. Ultima Online in particular had a significant amount of effort devoted to a player created economy and players building their own houses. But contemporary games like World of Warcraft and City of Heroes are essentially stateless worlds. Players can't make permanent changes. Players can't create a building, or modify a sign, or even leave a hat in the middle of the desert. And there's nothing at all like being able to write code to run in-game.

The one shining example of player created content in the commercial world now is Second Life. While not quite a mainstream MMOG, it's interesting for its devoted player-creators, impressive authoring tools, and successful player created games. Even the problems are interesting. Let's hope the business works out.

Player created content is hard. The gameplay and quality issues significant, it's hard to build server technology that scales when you don't know what the content will be, and it's hard to build good creation tools. But player created content is the interesting thing possible for an MMOG, both for the creators and the consumers.

The newest entry in the Civilization franchise is out. The new UI is great, nice use of 3d and lots of info packed on the main screen. Gameplay is improved but familiar, with one major addition: religion.

Religion is a fun twist on the culture system. Cities occasionally found new religions. Religions spread to other cities along trade routes. You can hurry the process along by sending out missionaries. Cities get bonuses for being part of your state religion; you also collect tithes. And religion affects your diplomacy with other civs.

I can't think of another mainstream game that's significantly touched the topic of religion. There's no Sims Church, is there? The developers were quite cautious, including this charming disclaimer in the manual:

We know that people have extremely strong opinions about religions ... We at Firaxis have no desire to offend anyone. However, given the importance that religions have had in human development, we didn't want to just leave them out of the game altogether; instead we have tried to handle them in as respectful, fair and even-handed manner as possible. (All religions in the game have the same effects, the only difference being their technological requirements.) ...

We offer no value judgements on religion; we mean no disrespect to anyone's beliefs. We're game designers, not theologians.

It is true that all religions are equal in-game, in the sense that each one gives you the same bonuses. But the game is by no means free of value judgements. Having religion is better than having no religion. Religion always aids science and income. Multiple religions get along fine; you'll never have a Belfast or Jerusalem. You're rewarded for stuffing as many religions as you can into each happy pantheistic city. That's gotta piss off the true believers.

The religion implementation is fun, a nice complement to fighting army men and growing culture blobs. And the implementation is as realistic and tasteful as the rest of Civilization's concepts. I just find it striking how gingerly they're moving. Maybe in a few years religion in games will mature and Rockstar will offend everyone with "Crusades: Fist of Righteousness".

ethereal has a cool little-known feature: the -z option to generate statistics from a pcap file. For instance, here's how to get a count of how much traffic you saw every five minutes:

$ tethereal -q -r foo.pcap -z io,stat,300

===========================================
IO Statistics
Interval: 300.000 secs
Column #0: 
                |   Column #0    
Time            |frames|  bytes  
000.000-300.000    3832    344880 
300.000-600.000    3744    336960 
600.000-900.000    2316    208440 
===========================================

The query language is pretty bizarre and protocol specific but there are some useful analysis tools. There's also the proto,colinfo option which lets you select out arbitrary protocol columns to print in the text dump from tethereal.

As much as I like ethereal it has a really annoying flaw; it only works on files less than 2 gigs big. I've been dumping some NTP traffic for 2 months and have 18G file to process, and tethereal complains I'm not patient enough to wade through the ethereal code to fix it, so instead I wrote a quick and dirty Python script to split a pcap file into 1 gig pieces: splitpcap.py. Maybe someone will find this useful.

I had a sort of weird moment today on the train coming back from Luzern to Zürich. I was reading the International Herald Tribune (the American paper) and there was an article about how the European Union was investigating whether any of its member nations were hosting the recently disclosed secret American prisons.

I was thinking "good for the EU!" when I realized, wait, I'm American. And my country is holding people in secret jails and torturing them. And there's no response, no outrage, no investigations and resignations and arrests. The only one doing anything is the EU, making sure they aren't participating in the gross violation of human rights. What else can they do? What else can I do?

What has happened to America? Despite my criticisms of US politics, I'm proud and happy to be American. Except for the torture part. And the imperialistic wars. It has to stop.

This week's issue of The Escapist is a 36 page special on girl gamers. Read the PDF. The article on page 27, OMG Girlz Don't Exist on teh Intarweb!!!!1, is great:

When I look at myself, I see a girl on the internet and a girl with an internet life. I see a girl who loves to play games and kill the dirty Alliance faction in WoW. I see a girl who can bunny hop with the best of them and keep her kills higher than her deaths in Counter Strike. I can talk the talk and walk the walk. But I am not a girl on the internet, because as I've been told before, I do not exist.

There's been a lot of writing on women and gaming that amounts to nothing, including some in The Escapist. But this article and a couple of others present a direct view into what it's like to be a hardcore girl gamer without a bunch of obnoxious overanalysis getting in the way of your understanding. Worth a read.

The EFF made some big news. DocuColor printers hide the date, time, and serial number of the printer in every image they print. It's not really a surprise that this watermarking exists, but it is a surprise that it's so transparent. Any idiot with a blue lightbulb can read it; no cryptography at all.

The EFF does good work and could use your support.

For the last 24 hours I've been listening over and over again to this new new Thelonious Monk / John Coltrane release from the Five Spot era. It's absolutely brilliant, overwhelming in its beauty. If you care at all about jazz rush out and buy a copy of this now.

The existence of this recording is a miracle, something like finding a previously covered up section of ceiling in the Sistine Chapel. It comes from the legendary 1957 collaboration between Monk and Coltrane, at the height of their development, of which there are almost no recordings. Until someone stumbled into a tape in the Library of Congress archives this year. And it was good.

But forget all that, just listen to this recording. It's amazing.

Today's IHT has an astonishing article about the North Korea Mass Games.

20,000 schoolchildren filling the side of one of the world's largest stadiums flip giant cards with such synchronicity that they form a gigantic screen flashing picture-quality images and communist slogans. ...

It gave me shudders. I watched again and again, but could see not a single child making a mistake ... the children moved as if they were controlled by the enter key in your computer.

Mass gymnastics foster particularly healthy and strong physiques, a high degree of organization, discipline and collectivism in schoolchildren ... The schoolchildren, conscious that a single slip in their action may spoil their mass gymnastic performance, make every effort to subordinate all their thoughts and actions to the collective.

When it comes to making people move like robots, nobody can beat North Korea.

Someone must have video.

Overlooked Firefox keyboard shortcuts. Ctrl-PageUp and Ctrl-PageDown cycle between tabs in a window. So do Ctrl-Tab and Ctrl-Shift-Tab.

For a brief time my blog was serving all 767 posts in a single page and in a single RSS feed. So some folks were treated to four year old blog posts as if they were new. Sorry about that. Mark all as read and carry on.

I turned off clicktracking on my blog too. The data was interesting, but I had stopped looking at it.

Thunderbird with imap is good software. For about 13 years I've been using the same email client: vm in emacs. While reasonably quick and infinintely adaptable, it's really not suited to modern email with attachments, HTML formatting, and multiple servers.

I just switched to Thunderbird and Dovecot imap and it's like a whole new world, simpler and neater. I now have much less reason to fire up a vt100. All I do in emacs now is coding and blog posting. I'm still using procmail and spamassassin for mail processing, but wondering if I can dispense with some of that, too.

Imap is really great, the server-side model is definitely good. But it doesn't go quite far enough: imap should provide mail delivery and address book management too. I don't want to have to configure ldap and smtp also.

What I'm really missing is the ability to easily search my 300 megs of email archives going back twelve years. Despite having written two mail search systems, I still go back to grepmail on the raw files. Gmail is cool and all but I don't like browser hosted email.

One of the most common data analysis things I do in Unix is something like Given an input file with a million bottles of wine in it, this shows me how many bottles of each type I have. It works for other things besides wine. In fact, it works for a lot of things, and I've been doing this for 15 years.

But the first sort is really inefficient, just something you have to do to make uniq work. So for big inputs I use a little Python script, countuniq.py. It does the same thing but more efficiently. Remarkably useful tool.

The early registration deadline for EuroOSCon is almost past. If you're thinking about going now's the time to sign up. I'm in the neighbourhood anyway so I'm definitely going: the list of speakers looks pretty impressive. The open source community is way more global than the US-dominated software venture world. Time to meet the talent from somewhere other than home.

apt is good software. I just upgraded my server from stable (Sarge) to testing (Etch). The trick is my server is 5000 miles away from me. And I will never have physical access to it, so if something goes wrong, I'm screwed.

But Debian did me proud. Run apt-get dist-upgrade, watch 150 packages go by, watch the new kernel install, reboot, say a little prayer, and I'm back. Some scariness from the kernel install script:

I repeat: you have to reboot in order for the modules file to be created correctly. Until you reboot, it may be impossible to load some modules. Reboot as soon as this install is finished (Do not reboot right now, since you may not be able to boot back up until installation is over, but boot immediately after). I can not stress that too much. You need to reboot soon.

I've done major system upgrades like this three times now with Debian, and everytime it's worked. That's pretty impressive.

I love NTP, the magic Internet protocol that keeps computer clocks synchronized. One of the nice things Apple does is run an NTP service for all their computers. That's how your Mac always knows the right time. Here are some notes on what they do.

My American MacOS 10.3.9 box has the following /private/etc/ntp.conf:

That means my Mac asks time.apple.com for the time somewhere between once an hour and once every day and a half. That's less often than usual for NTP, presumably an accuracy / scalability tradeoff.

time.apple.com currently resolves to 4 IP addresses, 17.254.0.26, .27, .28, and .31. All of them seem to be in San Jose, California. .26 is the most accurate clock (stratum 2), the others are just behind at stratum 3.

Using ntpdc you can find out where .27 gets its time from. They're well synchronized, talking to several stratum 1 clocks. They poll each of those every 128 seconds, aggressive but maybe appropriate for such a well-used clock. They sync around the world, and also run services in Europe and Asia for their clients.

I love my new Canon 350D. It's the first SLR I've owned, and the pictures are automatically 10x better than my little Canon Elph S400. It's the only time in my life where spending money made me a better artist.

But inexpensive digital SLRs don't shoot a full 35mm frame. Instead they have smaller sensors. The 350D has a "1.6x crop factor", which basically means your pictures look OK but your lenses are more telephoto than you'd think. A 35mm lens acts like a 56mm lens. If you want a wide angle 28mm shot you have to buy a 17mm lens. Really wide angle lenses for these small sensor cameras are very dear.

Canon has tried to turn this liability into an advantage by selling "EF-S" lenses, special lenses that only work on small sensor cameras. They go deeper into the camera body, and in theory allow for cheaper / lighter lenses. All I know is they don't work on all cameras, just these small sensor ones.

And while right now most digital SLRs are small sensor, there's reason to think that Canon may soon switch to all full size sensors. The newly announced full sensor Canon 5D is a bellweather here. At $3300 it's a lot more than the $800 for a 350D, but a lot cheaper than the $7600 that the current cheapest full sensor camera costs. Folks seem to be excited by the 5D full frame. Wide angles that work like wide angles, better viewfinder, what's not to like?

All of this is a long-winded way of suggesting that EF-S lenses may not be such a good investment, that in four years you may not be able to buy a good camera that uses them. Which is sort of an awkward position for Canon. They've made some serious EF-S lenses, will they abandon their customers?

I've been waiting a lot for my computers.

I have a bunch of video files on my Mac. When I select one in the Finder to open it, I first have to wait for ~3 seconds with a spinning cursor while Finder generates a thumbnail preview. I don't want a preview, I just want to watch the video.

On my Windows box in Zürich I have an SMB mount to a fileshare in California. I expect it to be slow, it's a long way. But it's slow even when I don't use it. I'll sometimes accidentally drag a file across the remote folder icon on my desktop. At that moment Windows decides it has to query the server for a bunch of stuff, forcing me to wait.

Applications should not force me to synchronously wait for some action I don't even want. Do the work in the background or substitute a low fidelity fast operation.

Then again doing that wrong can be a hazard too, as Picasa and iPhoto show. When browsing photos full screen, both apps first show you a low-res version, then 1–2 seconds later swap it with a high-res version. I assume this is a choice to make the app seem more responsive (less waiting!), but it's really just obnoxious because of the visual distraction.

I whipped up a little Javascript library to format relative timestamps. Instead of labelling blog entries something nerdy and confusing like "2005-09-07 08:32 Z", ago lets you say "3 hours ago". You can see it here on my blog. The idea was shamelessly stolen from NetNewsWire, Flickr, and a dozen other apps. I implemented it here on my blog to help me deal with time zones; I'm in Zürich, my server is in Texas but running in UTC, and most of my friends are in California.

To use it, put some javascript in your document like

It will be replaced with a friendly English string in the client's browser. The number is a Unix seconds since epoch timestamp. The clever thing here is that by doing this in Javascript, the relative timestamps look correct even if the page is pulled from a cache. (It will be broken if the client's clock is wrong, but that's their problem.)

The code is freely available, public domain. You'll also find JsUnit tests, a demo, and a Blosxom plugin to make it easy to add to my blog.

It's hard to tell what's going on when I'm not in the US, but from what I've read and seen New Orleans sounds like a horrorshow. Not just the flooding and the evacuation, but the looting and the deaths and the thousands of people living in a closed sports arena with no toilets or fresh air.

I don't normally think of this kind of disaster and consequent chaos happening in the US. Because you know, we're civilized and technological and all that. A whole city is going to be gone, for two months, and the people left behind are going nuts. It makes me shudder to think what would happen if a serious earthquake damaged the bridges out of San Francisco.

The Canon i80 printer drivers are bad software. I'm living on a Mac laptop in Zürich, which is mostly OK. But boy, Canon doesn't make it easy to use their printers. On a Windows machine I just plug it into the USB port and it works. On the Mac I have to go to the web site, manually find the drivers, download the DMG, mount the DMG, guess which file to open, open it, type my password, wait 2 minutes for drivers to install, wait 3 more minutes for the Mac to mysteriously "optimize system performance", then reboot. Only then do I get to print my photos.

I suspect the fault is more Canon's than the Mac's. Except for the optimize and reboot part; that's the Mac's fault. Isn't this Unix? Doesn't it have loadable device drivers? If I have to be aware of lpd, I'm gonna scream.

Pat Robertson on the 700 club, speaking about Venezuelan President Hugo Chavez.

I finally figured out what it is that bugs me about the Segway. It's impossible to ride one without a look on your face that says "hey, look at me! I'm on a Segway!".

Well, that and the whole "wheelchairs for healthy people" thing.

I've been reading The Ultimate History of Video Games. It's full of lots of details I didn't know, a thorough accounting of the development of the game business with lots of first person accounts. But what's most fun is the pointers to weird-ass games I'd never heard of before. Like Chiller. It's a fairly straight-forward Exidy light gun game, only completely fucked up. See, the game is about shooting corpses and zombies. See the man in the Rocky Horror underwear on the left? Shoot his skin to flay him alive! See the guillotine? Shoot it to behead the woman! See the press on the right? Shoot it six times to crush the man's skull! If you get all the way to the last level there's a girl zombie whose clothes you can shoot off. Boobies! Then you shoot her to pieces. My favourite is the winch you shoot repeatedly to lower the guy into the river of blood where a crocodile eats him, one joint at a time.

There was some controversy when the game came out, and apparently it did not sell well in the US. I think it was intended to be funny, but really it's just disturbing. And not a very good game. Still, there it is.

It took me three tries to pass Yahoo's captcha test. I'm a human alright, but I can't tell what the hell that says.

I'm finding Fire Emblem really engrossing. It's a great mix of puzzling strategy and cheesy Japanese RPG. I love the simple nobility of the charaters' stories combined with the clean anime-style animations. The story is incredibly sprawling, involving three main protagonists and 20+ supporting characters. I will destroy the Black Fang!

One of my favourite memories of my trip around Europe in 2001 was discovering the art of Space Invader. I was wandering around Aix-en-Provence and there was this funky mural of a Space Invader character. How odd, I thought, then moved on. Later I saw more space invaders in Genève and by the time I saw them in Paris I knew I'd discovered something truly weird. It's been a few years now and the artist has become more explicit about his work, including a fancy web map. And a lot of people have caught on to it. There's still a special thrill for me when I run into one of his mosaics somewhere, and it's fun to get photos. I love the way his work turns urban spaces into a game. Photographs of the murals seem to capture interesting context.

I've moved my blog from 63.194.75.26 to 72.36.170.170. Also switched to Apache 2. If you notice any trouble, please let me know.

The new host is layeredtech.com. I shopped around for an inexpensive dedicated Debian host. I like layeredtech because they don't sell you management, just a host.

One of the more interesting things we did when visiting New Zealand was go to Stewart Island. south of the South Island. All through our trip we'd tell locals "We're going to Stewart Island" and they'd say "really? I've never been there". It's a tiny place, about 400 permanent residents.

Stewart Island has retained most of its unique endogenous life. Ulva Island was outstanding, a small nature refuge with lush forests and odd birds. There are hiking trails all over the island, too, seemed like a great place for a ten day trek.

We particularly enjoyed our tour with Sam and Billy the Bus. Sam's what they call "a character", an old timer who enjoys taking the tourists around in his van and spinning yarns about the people on the island. Definitely worth the experience.

I love remote places like this. There's something very comforting about the quiet, the odd collection of people, the knowledge that there's not much to do. I enjoy being in places like this, and I enjoy knowing I'll be leaving again in a couple of days.

PS: if you're thinking of visiting, do yourself a favour and take the plane. Even the locals say the ferry crossing is awfully rough.

I've been emailing on the Internet since 1990. I use the one true quoting style when responding to emails, dating back to at least the earliest days of Usenet. It allows people to carry on a conversation in email. Reading this snippet it's easy to see the conversation. Previous relevant text is succinctly quoted with >, you can chain the > to indicate history. This style is so ingrained in the Internet culture I'm amazed to find myself explaining it.

But it's disappearing. I frequently get emails from people now who don't understand it. They gamely try to follow my style by writing a full reply on top of my email, or inventing their own mechanisms like replying in ALL CAPS or inserting random other markup. I'm not surprised when non-tech people have trouble with email quoting. Frankly I'm glad my lawyer is on email at all, so I won't quibble. But lately I've even seen a fair number of Google employees who don't get this, either. We're supposed to be the most savvy of the net savvy!

Part of the issue is cultural dilution: you can't increase the Net population 10,000x and expect the culture to stay the same. I also think it's technological. Outlook, Gmail, etc all encourage bottom quoting by starting you out in an email reply with the original message below where you're typing. Wrong choice. And modern email technology often does away with the notion of line breaks entirely, making the line-oriented > quoting mechanism fail.

We need to find something that works for everyone.

I'm looking forward to something fairly exciting: I'm going to be living in Zürich for three months, starting the end of August. I'm going to work in Google's Swiss office.

I'm really looking forward to it, both as a career opportunity and as a way to try living in Europe for a time. I've travelled a fair amount, even lived in Budapest for a month, but this feels like a significant thing. Should be fun.

I've been listening to Swiss German tapes and it's clear my high school Hochdeutsch didn't prepare me for living in Switzerland. I'm sure I can get by with just English, but it seems like much more fun to order a beer in the local language.

I am a computer
I have been programmed to dance
I am here to digitize you

I love BitTorrent. Not because it lets me steal the w4rz0rz, but because it's a really great new transport for large downloads. I've been impressed with Bram's stewardship of the protocol and am glad to see he's pursuing commercial options.

But open source protocols in hackerland are fragile, there's a danger they fragment. And that seems to be happening with BitTorrent. I'm seeing torrents now with three kinds of tracking: the standard TCP tracker, UDP tracking, and trackerless. Which client works for all?

I'd been wondering why my torrent downloads recently weren't working. I think the problem is that Debian testing installs bittorrent 3.4.2-4, which doesn't seem to work very well. The 4.0 .deb from bittorrent.com works better.

IGN has just published their 2005 list of top 100 games. It's a great list, well written, with lots of interesting editorial choices. It's also hard to understand the whole list because of the article format. So I created a version of their list in CSV form as well as an HTML table. The table has the tablesort magic, click column headers to resort. Interesting to group by year, or publisher, or platform.

IGN is a great publication, you should definitely go visit their article. Their list includes box art and interesting editorial comments on each game. I'm only providing this list as a simple index into their articles.

I tend to forget Nintendo is a Japanese company. With all the popular franchises like Kirby, Zelda, Metroid, Ninja Gaiden, and Mario having been ported to the US, we miss the hidden jewels like Fire Emblem that have never made their way here. But now Nintendo has released US versions of two Fire Emblem games for Game Boy Advance. And they're good. Fire Emblem hasn't changed a lot since the early days. The original 1990 NES game is a mix of RPG and turn based strategy. You read some story, then enter into a tactical battle where you move your little guys on the map to attack their guys, then read some more story. The fighting is entirely algorithmic; your choice is in how you manoeuvre. What's appealing is that you have enough units to really feel like you're managing a battle, while still having few enough that they have individual identity. It's surprisingly complex, and the entire franchise looks good. It's too bad Nintendo took so long to bring these games to the US. But between the popularity of Final Fantasy-style RPGs and Advance Wars-style strategy games it's now clear there's a market here. Normally I find these kinds of games boring, too much reading and not enough playing. But on a handheld it has a nice rhythm.

One of my favourite writers about games, Greg Costikyan, has just published slides from his latest talk (PDF version). The slides are worth a look, Greg does a great job arguing that the current increasing cost of game development is going to kill creativity and we need an alternative distribution channel to save the industry.

But I wanted to comment on some of the side-issues Greg's talk raises. Slide 9 makes an arresting argument:

Greg argues the long term game markets is for all ages, not just young people. What an opportunity, both creatively and financially!

Greg's post also reminds me of my frustration that games are too long. A movie is $10 for 2 hours of fun. A game is $50 for (in theory) 30+ hours of fun. But most people I know get bored of games in less than 20 hours, the rest is wasted. I'd rather games were shorter, cheaper, and more varied. But the economics of the industry make that tough. The $50 price point is fixed by development costs, and they don't get much cheaper for a shorter game. And if the consumer pays $50 they expect 30+ hours of gameplay. So we're stuck with an industry that makes long, expensive games.

Sitting at home, eating a chili dog, drinking a couple of beers, and watching Repo Man. Life really doesn't get much better.

Photoshop's JPEG code is bad software. I'm a longtime Gimp user. But lately I've been playing around with RAW images and colour profiles, and Gimp does badly with those. So I'm giving Photoshop Elements a free trial.

I mostly like it, but its JPEG saving is insane. Not only does Photoshop not use the same 0-100 quality scale that all the other tools I know use, they have two different ways to save JPEG, with different scales. Neither work right.

I have a picture, a portrait from my Digital Rebel XT. It's 3456x2304 and about 9 megs as a PNG. If I convert it to jpeg using cjpeg -q 75, the baseline for JPEG creation, it's a 580k file. OK, all well and good.

If I try to use Photoshop's "Save for Web", the quality slider lets me choose 0–100. I set it to 75 and the resulting image is 1.4 megs! That's about like a cjpeg 92. OK, so they have a different scale. Fine. But no matter what I do I can't get a 580k file out of the tool. A quality of 50 yields 554k, a quality of 51 yields 677k. Why the giant jump? Shifts to 49 or 52 don't yield nearly as much a difference. To make matters worse, "Save for Web" leaves out all your EXIF data. And the tool gives you dire warnings about 8 megapixels being too big an image. Um, right.

The normal "Save as.." in Photoshop is a bit better. The JPEG option has a quality slider, too, but this one goes from 0–12. Because you know, 12 is better than 10. Generating a 580k file is still impossible. Given there's only 12 quality steps that's not too surprising. But there's still a giant jump: quality 8 is 607k, quality 7 is 400k. What's going on?

I guess I shouldn't be this fiddly about JPEG settings, but I'm truly mystified as to what Photoshop is doing. cjpeg gives a nice smooth quality/size tradeoff with lots of fine grained adjustments right in the middle of the sweet spot. Photoshop seems to make it impossible to control the same tradeoff. No wonder so many online images are poorly compressed.

Scientific American used to be a great magazine, but a few years back they made an editorial decision to become less nerdy, more sensational, less interesting. Technology Review has somewhat filled the gap for me, although the new editorial policy is way more business oriented than I would prefer.

The June 2005 issue had a few interesting bits in it. Larry Lessig has a good free culture essay. And Howard Anderson, of Battery Ventures, has a somewhat shocking essay on why he no longer wants to be a VC.

And while it's in the July issue, I can't help but quote this zinger:

Want to type arbitrary Unicode characters on a Windows box? You have several options. This one worked for me:

1. In HKEY_Current_User/Control Panel/Input Method, create a key "EnableHexNumpad" and set the value to REG_SZ, "1". Do it manually, or use this registry file.
2. Reboot (yes, you have to)
3. Hold down alt, type numpad +, type a 4 hex digit code (263a, 0041, whatever), release alt.

Full disclosure: when I rebooted Norton AntiVirus failed to start. It's hard to believe this had anything to do with enabling Unicode input. But if the idea of breaking your Windows box scares you, maybe you want to stay away. Auto-Protect didn't start with error (4002,517), reinstalling shared files fixed it. Isn't Windows fun?

I've been a happy user of the Ethnologue database for years. It lists all languages of the world in a simple database form. Just yesterday I did a search for Malagasy ethnologue to learn that Malagasy is a language of Madagascar. It's a lot of fun to browse, skipping from country to language to country as you follow immigration patterns.

The New York Times today has an interesting article about Ethnologue, on the occasion of the publication of the new version of the database. I had no idea the database was developed by Christian missionaries! In retrospect that makes perfect sense, just like LDS maintaining the best geneaology data. Sometimes religious fervour has practical benefits. I also didn't know Ethnologue is the source for the three letter language codes we use in internationalized software, soon to be ISO 639-3.

Don't miss the NYT's visualization of language distribution. I love the NYT graphics; this one is by David Constantine.

I preordered the new Harry Potter book months ago. I was surprised to get this email from Amazon today: A retroactive discount? That's a clever way to build business. A quick search reveals Amazon is now undercutting the other major stores on price.

In related news, you can steal the new book electronically if you like reading OCRed pages. A torrent of an audiobook version (lovingly read by a robot, one presumes) has 2000+ downloaders.

Hikaru No Go is a great Japanese manga. It's the story of an elementary school boy who discovers an ancient Go board inhabited by the spirit of an expert Go player from Japan's feudal past.

The comic deftly tells stories that combine the pressures of being a young kid, the nuances of the game of Go, and anime-style epic battles. It makes playing Go seem incredibly cool. The art is awesome: see some samples here.

The English editions (published by Shonen Jump) are well done. My eight year old nephew just devoured one in a day. Amazon makes it difficult to find them. Here's a list of the English editions: Hikaru No Go Vol 1, Vol 2, Vol 3, Vol 4.

WarioWare for the Gameboy was a breakthrough game combining insane graphic design, simple gameplay, and frenetic pacing to make a fun game. The follow-on WarioWare: Twisted is just as great.

The novelty in Twisted is a motion sensor in the cart. To make things happen in the game you spin your Gameboy around. It's true that this basically turns your Gameboy into an Atari 2600 paddle, but it really works. The rumble pack's tactile feedback adds a lot, little clicks and thumps that let you feel your turning. My favourite minigame so far is a balance game where you tilt the Gameboy to keep the umbrella balanced on the end of your finger.

The game starts with dire warnings to spin only your Gameboy, not yourself, lest you get sick. The warning is so detailed and over the top the first thing I wanted to do was stand up and try spinning myself around.

The best part of my trip to France last month was poking around the Southwest. Carcassonne, the Lot, Perigord, the Dordogne. Beautiful, restful, lots of interesting little towns. Below is some of the nice things we found. See also my notes on restaurants from the trip and my photos.

More inside ...

rsnapshot is good software. It's a simple Unix backup program that gives you historical snapshots, sort of a low rent NetApp in free software. I now have views of my files from four hours ago, and one day ago, and one week ago, and... Having multiple versions protects you from the "I deleted some important files two days ago" problem. Behind the scenes it's doing rsync and hard links to keep the size reasonably efficient. What I like best about rsnapshot is that it's very professionally produced and was easy to set up on my Debian box.

I don't understand how anyone backs their machines up anymore. Disk is fifty cents a gigabyte. I have something like 200 gigs of data in my house and I didn't even try. Removable media sure isn't going to work to back it up: tape's too expensive and slow, CD and DVD are too small. So all I'm left to do is backup to other hard drives.

That doesn't help you if your house burns down. I'd love to backup over a network (rsnapshot supports this), but I don't have the bandwidth to even back up the 5 gigs that really matter. For now I'm also making copies to a USB drive that I store outside my house.

I serve this blog from my home DSL (an increasingly dumb idea). My DSL went to hell at 11am today. Pinging from my house 63.194.75.26 to the DSL gateway on the other end of the line 63.194.75.25 is taking well over one second. No packet loss. It's not my router, either.

I've seen this slow link failure mode before, but never for this long and with no packet loss. Usually it's gone away on its own, a couple of times I've called for support and they "cleared the line" and things got better again. I wonder, does their router infrastructure go flaky or get overloaded? Do they monitor it?

I hate not being able to get real help. Their phone support is so far removed from an actual expert that I can't say things like "I monitor the ping time on the link and it's gone from 1ms to 1000ms". At least their support script did lead to them actually detecting the problem.

Argh, now my link is entirely dead. Man, sonic.net is looking better and better.

Google Sightseeing has a Gibraltar post today that's cool. Gibraltar is so small that the only road in crosses the airport's runway. The new satellite imagery is so fine grained that you can see cars waiting to cross the runway. Scroll far enough east, and you can see the plane they're waiting on. Who knew there'd be so much drama in satellite images?

    

I like Joe Kraus' blog entry on hiring engineers, particularly the "do you have a blog" and "do you contribute to open source" questions. I love interviewing engineers with blogs or open source work because I can do a bit of research and have something meaningful and in depth to talk about.

His other question, "what's your home page", made me smile. Yes, I had a start page back in 1994, but I've long since stopped using it. I use MyWay these days, but my real start page is DQSD, the fastest way to do Google searches on Windows.

Joe quotes my friend Marc saying "Jedi Knights make their own lightsabers and great engineers make their own homepages." But remember how in the original Star Wars movies Yoda didn't have a light saber? That's because he was so kick ass, he didn't need one. about:blank is the start page for Yoda engineers.

I've been totally hooked on Battlefield 2, the new squad FPS. It's got great teamplay, aided in part by built-in voice over IP so you can talk to your teammates.

But adjusting the microphone so the gain is correct is a total pain. In addition to the in-game settings I've found at least three different audio control panels in Windows that affect it. The key is to ignore all of Creative's crap software, go to Settings / Control Panel / Sounds and Audio Devices, click the Voice tab, go to the Voice recording panel, click Volume, then boost the Microphone slider. If you click Advanced you can also set a +20dB boost which helps too. Basically turn up the gain in every setting in Windows, then use the in-game control (with feedback) to lower it back to a reasonable setting.

I bet this isn't so hard on MacOS.

The packet sniffer ethereal is essential for reverse engineering applications and diagnosing network problems. But it's primarily an interactive tool with a GUI. I can't figure out a way to use the Ethereal packet analysis code as part of a program I write.

One hack is to store packet captures to disk, then use tethereal on the command line to dump what you're interested in to ASCII. Here's a quick and dirty way to print out all the URLs that were fetched:

Still, there's gotta be a better way to process pcap data with the high level analysis Ethereal does. ImPacket is an interesting alternative, but it seems orphaned.

I had lunch at Aqua recently, lovely until the dessert. I ordered coffee mouse with "Havana cream". I expected something rum flavoured, but no, it was cigar flavoured. I'm open minded about that, but the flavour was unpleasant: peppery with a weird feeling in the throat. And I felt ill for the next few hours, presumably from the nicotine. Avoid.

Ken's the family chef. He's doing the low carb thing these days. One of the substitutes he makes is cauliflower rice; you put cauliflower in the food processor until it's rice sized, then saute it quickly in butter. It's delicious! You gotta love a diet where "saute in butter" is one of your main techniques.

Formal French table service includes a sauce spoon, a nearly-flat spoon with a notch in one side. (I grew up calling this a fish knife.) I thought I had it all figured out: you scoop up the sauce with it, put the spoon in your mouth, and enjoy the yumminess. So I did that on my last trip, and to my embarassment found my next two dishes were brought to me with actual spoons, not sauce spoons. Had I done something wrong? I think I was right.

Here are some notes about some of the more memorable places we dined in France (in addition to the steak frites place). I'm not a food critic, so this may be boring, but maybe it'll be helpful for people looking for places to go in Paris or in the Southwest.

More inside ...

I enjoy good wine and after several years learning about wine I think I know something about it. But I've gotten lazy and stopped taking notes about the wine I drink. It just seemed too much like work.

When we were in France we had a lot of great wine, mostly from the Southwest: Bergerac, Cahors, Languedoc... But since I don't know the wines there I can't remember what we had. The only wine I thought to write down was a Languedoc, a 1995 Domaine de La Grange des P�res that was fantastic. But specifically how was it fantastic? I can't tell you now.

Maybe my friend's manageyourcellar.com is the solution. It's a web application for tracking your wine cellar and tasting notes. The views of your own cellar are nicely customed: "ready to drink", statistics, recommendations. A nicely detailed database of wines holds it all together. The site has a wine blog too. Now to get some discipline about keeping notes!

Ken and I are food tourists and we eat well in France. But sometimes we get tired of foie gras, delicate vegetables, and French menus (just what are St. Jacques' nuts? Mussels? Scallops? Clams?) When we get overwhelmed and want comfort food in Paris, Ken and I head to one of several steak-and-fries places. Couldn't be simpler. You sit down and the waitress immediately hands you a little salad (walnuts, mustard dressing). No menu, just two questions. How do you want the steak cooked? A point. Do you want wine? Yes. (But get the wine list: the house red is awful.) The waitress swiftly comes back with a plate of delicious steak bathed in a yummy mustard and sage sauce, along with some perfect pommes frites. You eat your steak and fries, they bring you a second helping (kept warm over candles), and then it's on to dessert. Skip dessert.

The whole process is the opposite of the usual French temple-of-food experience. But it's very satisfying and, judging by all the Parisians we see there, quite popular with the locals. The restaurants are a franchise. We've been to Le Relais de l'Entrec�te in the 6th (20bis rue St. Benoit, conveniently a block from St. Germain de Pr�s) and Le Relais de Venise in the 17th (271 bd. P�reire). The joint in the 6th even has the same 1945 Cognac Rouyer poster I have at home!

The PC version of Grand Theft Auto: San Andreas is out, and boy is it nice. It's just as great as the PS/2 version, only with beautiful graphics (and, alas, crappy controls).

But the best thing about the PC release is that it's open to modding. A bunch of mods are already out, including the Hot Coffee mod which restores a sexually explicit mini-game that the developers apparently took out at the last minute. See some screenshots to get the idea.

As the Guardian points out the sex is not particularly sexy. You gotta love a progress bar that says "Excitement". But it is consistent with the cartoonishness of GTA, and I think this is the first time a mainstream game has had explicit sex in it. It may be that the most shocking part isn't that the game depicts sex, it's that it depicts sex between a black man and a white woman.

Hey there, I'm back from a nice little vacation to France. Paris, the Lot valley, Perigord. I actually got sick of eating foie gras and truffles: life's pretty good. But before I write up the fun stuff, here are some reasons to hate Air France:

1. The cabin in their 747-400 from SFO to CDG is older than I am.
2. Paris has two airports, who woulda thought? And if you miss your domestic flight because you're an idiot and went to CDG instead of ORY, Air France makes you buy full fare tickets to replace the flight you missed.
3. They don't have proper gates at CDG, so you get to spend 30 minutes riding a bus all over the airfield to get to your plane.
4. They lose one bus of passengers, it takes 30 minutes to find them.
5. Only one of your two suitcases is on your flight.

Bah! But France was great, lots more to come. One quick suggestion: the Chateau de Mercues is fantastic if you want a relaxing country stay in the Lot, near Cahors.

Speaking of finishing things, I just finished playing Jade Empire, Bioware's latest epic RPG. It's very good, albeit not great.

The RPG gameplay is the same as Knights of the Old Republic and other Bioware games: multiple choice dialogue, a party of characters you interact with, and choices between good and evil. The big gameplay innovation is the real-time combat system, an entertaining action combat thing like Prince of Persia or Ninja Gaiden. It mostly works, although it is awfully simple.

The best thing about the game is that it's set in a faux Ancient China. The æsthetic is well done, very beautiful, and refreshingly different than the usual RPG fare. I particularly liked the way Chinese philosophy linked into the story, very different from the usual chivalry-tinged stuff.

The other thing that worked well was the NPC characterization. In previous games I never really cared much about the rest of my party, I think because I didn't find their backstories interesting. But in Jade Empire I found them quite compelling, particularly Sky, the man driven to avenge the daughter captured by slavers. Despite the conversation trees being very simple (and nothing approaching KotOR2's influence system), the NPC interaction was good. The romance plots are well worked too, including various alternative options.

I've had enough of Bioware-style RPGs for awhile.

There's a great feeling of accomplishment at the end of May, at the end of the mainstream TV season. All these shows finished! If by "accomplishment" you mean "done sitting on my butt passively starting at mindless entertainment". But I'm done with The Amazing Race, and Survivor, and Desperate Housewives, and Lost, and 24. That was five hours a week, heavy responsibility.

Java is the language for building large web applications. One reason Java is better than C is that it's garbage collected, making programming easier. But garbage collection isn't free, it can have significant overhead.

Web applications have a simple memory allocation pattern. An HTTP request comes in, some work is done in a thread over a second or two, and then you're done. Unless you're using session objects heavily (and you shouldn't), at the end of a request there's really nothing more in server memory than there was at the beginning.

This simplicity suggests a better memory management strategy for web applications. Allocate all objects in a zone specific to the HTTP request. Don't bother reclaiming memory during a request; when the request is done, just drop the entire zone at once. Poof. This won't work if you need to reclaim some of the working set during a request, but I think that's unnecessary for many apps.

I wonder if anyone's done this? It'd be easy enough to code in a C web application framework. Impossible in Java unless you can modify the JVM. Python sort of implicitly does this, since it collects most objects as soon as they fall out of scope.

Bleep, the online music store for Warp Records, is awesome. DRM free MP3 for cheap and convenient.

I just went there to get the new Autechre release Untilted and was surprised to see Bleep is now selling FLAC as well as MP3. FLAC, if you don't know, is a lossless compressed format. Giant files, but "perfect quality" for the audio nerds. $10 for the album on MP3, $14 for FLAC. Honestly, high bit rate MP3 is fine for me. But it's nice to have the option.

BoingBoing notes that Bleep will also be selling XviD for a new video collaboration between Chris Cunningham and Aphex Twin. I'd probably never buy this video if I had to get a physical object, but I'll definitely buy a download.

Microsoft ClearType is good software. It's a font rendering option in Windows XP that makes things look better on LCDs. Between laptops and good desktop LCDs a lot of folks have LCDs. So go ahead, turn on ClearType! And get the tuner while you're at it. The technology is surprisingly straightforward. LCDs are made up of lots of tiny red, green, and blue pixels. ClearType uses that structure by doing subpixel rendering of fonts, so instead of drawing a white pixel where you'd expect it draws a blue pixel next to a yellow one, which still looks white. You can see it above in the colour fringing; at normal resolution on the LCD it looks right. It reminds me of the old Apple ][ colour hack where they fit 6 colours per pixel in 1.14 bits.

Microsoft's implementation is quite robust. I was particularly impressed when I rotated my LCD 90 degrees and it still did ClearType, only in the Y axis. Still, Windows typography has a way to go. Apple's font rendering is still head and shoulders above anyone else's. I'd love to read an article that explains exactly why text looks so beautiful on a 12" Powerbook.

It took me 72 minutes to drive 33 miles home today. That's 28 miles per hour. 31 miles are on the freeway.

Fuck.

FlickrClient.py is good software. It's a Python library for the 68 methods in the Flickr API in 48 lines of code. Such parsimony! Here's the nut of it: If you're not familiar with Python, what's going on here is a bit of metaprogramming genius. The __getattr__ makes it so that FlickrClient.foo() automatically means "invoke the Flickr API call foo", for any API call. The method() code builds a URL out of the Python method name foo and the parameters the method was invoked with, then does a GET to Flickr. Because of the consistent naming of the Flickr REST API this all works like you'd expect.

Parsing the response data is done by passing the Flickr XML to xmltramp, a quick and dirty Python XML binding that does what you want. The _parseResponse() method (not shown) does error handling in four lines.

I love how simple and Pythonic this is. Data is loosely typed. API calls are built dynamically. Everything is so abstract that the library code doesn't even have to be updated when new API methods or datatypes are added to Flickr. Clean, simple, perfect for scripting. It's about the exact opposite of SOAP and WSDL where you typically use thousands of lines of generated code to call the service. The WSDL approach gives you strong typing, which can be helpful, but can also be a dual edged sword.

I've never really minded the New York Times login. I only do it once a browser and the content is good enough it's OK. But yesterday I went to login and was told "we've changed our system and your login nelson@santafe is no longer valid". To their credit, they did have an easy path for changing my login, but why the change at all? Who decided email addresses weren't valid logins?

My biggest problem with all the sites I log into is remembering my username. It has to be unique, but different sites have different restrictions. Am I Nelson (not unique), or NelsonM (ugly), or NelsonMinar (too long for some), or nelson@monkey.org (contains punctuation), or nelson+foo@monkey.org (plus is often not accepted)? By contrast I can usually use the same low security password on all these sites, because passwords have fewer restrictions and don't have to be unique.

I realize I'm repeating myself, but this whole thing about having thousands of accounts on different websites is really stupid. I should use bugmenot more.

I'm kind of late to the party, I guess, but I'm still not understanding Yahoo 360. Every once in awhile I find a link to a blog post there. Like in this Make:Blog article, which links to what sounds like an interesting post. So I click on the link, and what do I see? "To access Yahoo! 360° Sign into Yahoo!". I'm supposed to log into Yahoo to read a freakin' blog article? Oh well, nevermind.

Am I missing something?

I finally finished Knights of the Old Republic 2. While I liked how ambitious the gameplay and writing was, I ended up being deeply disappointed by the missing content, particularly at the end.

Apparently there's some fantastically epic story about Malachor V, the Sith training at Trayus Academy, and the triumvirate of Darth Traya, Darth Sion, and Darth Nihilius. And some cool side plots involving HK-50 droids, G0T0, Bao-Dur, and Atton or my Disciple falling in love with me. But the only way I know this was planned out is because people scraped some of the missing bits off the game image. When you actually play the game, what you get is a horribly rushed sequence of barely coherent events capped off with some deliciously epic writing that barely makes sense.

And the most ambitious part of the game design was neutered. There's a complex system where you interact with the characters who join you, try to gain influence over them. You could tell the game was being designed so the end depended heavily on whom you influenced through the game, a brilliant idea. Alas, an idea that is totally absent in the actual ending.

I feel cheated, like reading a novel that's missing 80 of the last 100 pages. If they had the same number of hours of content, just more coherent, it would have been great. They were just too ambitious on too tight a schedule. I wish more reviewers would punish game publishers for rushing things out the door before they're ready.

Seeing Goldman is in New Zealand prompted me to review my notes from my trip there. I really loved that trip.

One thing that fascinates me is the role of the Maori culture in modern New Zealand. NZ is one of the very few places where native peoples were able to survive the invading Europeans and preserve their culture. For tourists, the easiest place to see that culture is in Rotorua, in the heart of the volcanos of the North Island. The town itself is kind of frumpy, and some of the tourist kitsch is a bit much, but overall it's a good place.

The highlight of Rotorua was Te Puia, a one-stop spot for your New Zealand tourist checklist. In one place you get Maori arts and crafts (including a beautiful marae), Maori feasting and dance, a geothermal valley, even kiwi birds. The center of the place is a serious school for traditional carving, so it's got some depth.

We stayed at the Millenium hotel. It was OK, not great, undergoing a badly needed renovation. Dinner was at Herbs Restaurant, in the middle of the little restaurant row near the end of Tutanekai street. I wouldn't go back. I regret now we didn't do go to a traditional hangi feast. I'm told it's quite nice.

Psychonauts is a wacky new platform game by Majesco games, more specifically Tim Schafer, the guy behind Grim Fandango. It's a psychedelic platform game. The conceit is you go inside people's minds where you navigate all their psychological weirdness. Mostly an excuse for some very wacky level design, as you can see from the screenshots. In the vein of Rayman or Beyond Good and Evil, only trippier. The reviews are great and the PC demo is definitely worth trying out. Oddly the game is value priced at $30 on PC, but $50 on consoles.

My mother was a travel agent. When I was eleven, I proudly announced that I was going to get married and go on a honeymoon cruise around the world. She looked at me deadpan and said "I'm sure I could find something more interesting for you to do."

I just got back from a ten day cruise to Mexico. We had a good time, very relaxing, but boy was it boring. We didn't really go anywhere I wanted to see: San Francisco to Monterey, Catalina, Mazatlan, Puerto Vallarta, and Cabo San Lucas. The Mexican beach resort scene just isn't me.

The demographics of the cruise ship scene aren't me either. I fear that for most of our cotravelers, baked alaska and prime rib will be the most elegant dining experience of their lives. I'm really glad that we booked with a gay travel group. We were part of a group we had something in common with and we met some nice people.

Being on a cruise ship is certainly very comfortable and it was great to drop out for ten days and do nothing. I'd take a cruise again, just somewhere more interesting like the Mediterranean.

As much as I love Grand Theft Auto: San Andreas, I stopped playing as soon as I unlocked Vegas. The environments were just too beautiful to tolerate the crappy PS2 graphics. Happily, the PC version is due out June 6 and is looking good. The screenshots are hot.

The one thing missing in GTA: SA is a Latino soundtrack. You're cruising the barrio in Los Santos and all you can find on the radio is is gangsta rap, crappy house, and power metal. Where's my mariachi? Where's my slow soulful Mexican ballads? Corazon, corazon.

The PC version lets you create custom radio channels with your own music. There's even a new feature where they'll loop in the game's hysterical chatter inbetween your songs. All I need now is the perfect Latino mix tape.

Remember when a tank of gas cost $20? Between gas costing me $2.75/gallon now and a new 21 gallon gas tank, a fillup for me is over $55. Je suis Californien.

But the two gas stations I go to in the Bay Area have $48 limits on credit card charges. They won't let me buy a full tank of gas. I asked the guy at Valero about this, he said the limit was set back when gas was $1.30 a gallon. Clever.

One of the beautiful things about the Flickr API is it enables interesting things without the Flickr team having to do the work.

Flickr has no "most popular photos" page, but there's a whole "most favorites" user group. For instance, there's a recent list of the top 60 photos. The #1 photo on Flickr is a technically amazing motion shot, with 394 favourites votes since Nov 2004. It's picked up 115 favorites in five weeks. The problem with favorites lists is they're self-reinforcing. More people see the photo, mark it favorite too, and before you know it you have a boring ol' power law. "New favorites" would be more interesting.

GustavoG has done some serious social network analysis. His FOAF plot shows two major clusters that are only sparsely connected; another view shows distinct clusters that seem consistent with the history of Flickr. He's also drawn the social center of Flickr, a scatterplot of who has lots of mutual contacts, and an interesting graph that shows the imbalance of incoming vs. outgoing links. That last graph shows lots of aspirational linking to Caterina and Stewart, the Flickr founders, and points out popular people I should know about like lightpainter.

There's a wealth of data in Flickr. The API enables all sorts of volunteer analysis.

Hey, something I did made the Google Blog! It's a picture of where people were drawing maps for a day. I love doing little visualizations like this, and Google Maps is particularly cool.

I'm just getting into Knights of the Old Republic 2, and boy is it great. A Star Wars RPG, sequel to the great KotOR of last year. Playing it is like being part of a really great, if long, movie.

You don't sit back and read the story, you are part of it, you influence it. Do you want to save the fledgling government of Dantooine, or do you want to overthrow it? Do you want to follow the light side and restore the Jedi order, or relish in the evil powers of the dark side and smash the Jedi forever? Given that the game really has a fixed, linear narrative, this feeling of choice is quite an acomplishment. It's a shame this narrative experience is missing from online games.

The big innovation in KotOR 2 is not only do you control your own destiny, but you can influence the fates of your companions, too. I find myself playing the dialog with my party members very carefully, trying to make them my friends while turning them to the dark side. The writing is good enough to make this quite compelling.

The best RPGs for the last few years have all been written by Bioware and Obsidian / Black Isle. Alas, even they fall prey to game industry demands. It's widely believed that content was cut from KotOR 2 to meet their very aggressive release schedule.

Firefox's Tools / Page Info dialog is good software. It has tabs to show you lots of data about the page, particularly what it links to and what media it has embedded in it. Very handy!

My friend Peter's new book just came out: Practical Common Lisp. I haven't seen the hard copy yet, but the online version looks great. Lisp is an important language to understand, from the core abstraction to the elegant things it can do. It's great to see a modern book describing using Lisp for practical things like web applications and managing MP3.

I love the (?P<foo>) named regexp groups in Python. They make the code so much more readable! But are they slower? Not much. The named groups version is about 6% slower. Consistent, but not very significant.

I live in the unhip part of San Francisco, the vast residential southwest that's full of middle aged people, single family houses, abundant parking, and practically no restaurants. I love it for its sanity and comfort.

Nothing captures the frumpiness of my neighbourhood better than the local newspapers published four times a year, full of columns from nervous women advising having the water department test your pipes for lead and crotchety men rambling about how they hate being retired. This restaurant review from the West of Twin Peaks Observer (no website, of course) is a great example:

One of our newest advertisers is the recently opened Taylor's Taqueria and Breakfast in the Diamond Heights Shopping Center. It has taken over the spot formerly occupied by Burger King, but it still serves great and varied burgers. I had one of their American-style breakfasts that the new restaurant features, and I have to tell you, it was terrific. Food is prepared exactly the way you order it, portions are large enough to satisfy any appetite and the prices are moderate. In addition to the breakfast menu (which is availalbe all day) Taylor's, named after owner Thong Le's young daughter, also serves traditional Mexican fare such as Tacos, Burritos, Nachos, and Quesadillas, and they even have gourmet teriyaki plates, along with salads for those who are seeking lighter dishes.

Burgers as good as Burger King, large portions for little money, and a bafflingly syncretic menu of American, Mexican, and Japanese food cooked for you by a Vietnamese family. What's not to love?

My camera doesn't have raw output, but it does have three JPG quality settings. I just tested empirically to see how they correspond to the JPEG Q parameter:

Setting	Size	jpg Q
Superfine	1824k	98
Fine	1148k	94
Normal	580k	81

Given that quality of 75 is considered adequate in most JPEG software, there's some comfortable headroom here. Fine's 94 quality seems plenty for the photos to me and saves almost half the space. Then again with 1 gig of Compactflash for $83, there's not much need to economize storage. Bandwidth is still expensive, I'll have to remember to recompress the images before uploading to Flickr.

The Battlestar Galactica remake finished on the SciFi channel, and I have to say it was great. I loved the original show as a little kid, but it's unwatchably cheesy now. The new show somehow is also cheesy, yet somehow uses that cheesiness without being obnoxiously hipster ironic. Instead it's gritty, sexy, clever, and fun.

My favourite part of the show is the grimy sexiness of it all, overlayed with the anachronism. Everytime Starbuck lights up a cigar on the hangar deck next to the fuel depots I smile. It's absurd and yet so TV-realistic. But then that's all tempered with the religious stuff, which seems both hokey and moving at the same time.

The best improvement over the original show is Gaius Baltar, who manages to be evil, batshit crazy, and sympathetic all at once. He's a center of the sexy, too.

I like Norton AntiVirus. The UI is OK, the protection is strong, and it doesn't get in my way. But its worm blocking doesn't play nice with FileZilla, the FTP client. When I'm uploading thousands of little files via FTP the worm blocker goes nuts.

I think the problem is that FTP still uses the awful non-PASV mode, where the server opens a socket back to my FTP client. Dumb, but that's the way FTP works. The worm blocker sees the incoming connections and assumes we're under attack. Fortunately it's easy to disable on a per-program basis with "Program Control" in "Internet Worm Protection". I see I've also disabled the checking for Half Life 2 and my X server. Guess this worm blocker doesn't like anyone opening connections to it.

The blocker is slow too. After I disabled it my FTP upload of lots of little files was 10x faster.

While I've admired Flickr for awhile, I wasn't much of a user. Until now. Now I'm totally addicted, checking it eight times a day. What's so great about Flickr?

The most compelling part of Flickr is the social network. Flickr is not just photo hosting, it's a whole community system. And it's not just a social network site, it's a photo site. I share my photos with friends, they share theirs with me. I join groups. I look at strangers' photos and comment on them. I see what photos my friends like, and the photos of my friends' friends. There's always something new and a social context for what we're sharing.

Another key innovation of Flickr is the tasteful use of metadata. Flickr tags are awesome. And thanks to the Creative Commons licensing Flickr is full of photos ready for reuse. Very handy.

The technical execution of Flickr is excellent. The UI is clean and fun, if a bit like a maze of twisty passages. There are RSS feeds everywhere. They have a great API; see Clay's notes for more. There's blog integration with badges and automated blog posting. And something I overlooked at first: updates happen instantly. You upload a new photo and it is immediately visible on every tag, group, and contacts page where it should be. It's not easy to make real time updates work at this scale.

Here's the addictive part. Flickr is a game, an MMOG * . There's the explicitly playful aspect, groups like squared circle and clock works. There's also an exploratory aspect to being on Flickr, where you go from photo to tag to photo to person to group to tag to photo, picking up images along the way and sharing them with your friends. And there's lots of interaction to keep you coming back, your recent activity and the group discussion boards. It's all quite compelling.

Want to know how many people are reading your blog from the hosted feed readers BlogLines, Yahoo, LiveJournal, NewsIsFree, and NewsGator Online? They're all kind enough to put some stats in their User-Agent string when they fetch your blog. I wrote some code to scrape the data out of Apache logs. Here, take it. I had plans to do fancy graphs like I did awhile ago for BlogLines, but the data isn't very interesting. BlogLines blows everyone away with 290 readers. Pretty much no one reads my blog via Yahoo, LiveJournal, or NewsIsFree. NewsGator Online has about 30 readers.

My new car yielded a nice surprise; Bluetooth phone support. Basically the car is one giant hands-free headset. I just step into the car with the phone in my pocket and I can make calls from the dashboard. It even synced up my phonebook. I feel like 1985 James Bond.

When I was at the Media Lab I used to mock Bluetooth. 1996, 1997, 1998, 1999, they kept saying "cheap low power wireless is just on the horizon" and it never happened. But it finally happened. Bluetooth is to wireless what USB is to wired connections; simple, autoconfiguring device interaction. Now if I could just play music to my car stereo via a Bluetooth MP3 player and if I could get a Bluetooth camera to sync photos to my PC, I'd be set. Oh yeah, and I need wireless power.

Now that the news is finally out, congratulations to both Flickr and Bloglines on being acquired! They are the best web startups I've seen in the last year. Great products, well executed, plugged into the blogosphere publicity engine without pandering to it. They both showed that a small company with the right people can build out a major service in a year. And if you believe the rumours, they built a lot of value quickly. I think it's interesting they chose to be acquired instead of going the venture capital route.

Two of the best talks at ETech were Stewart on Flickr and Mark on Bloglines. Both of them gave direct, honest descriptions of how they built their companies and what social and technical challenges they faced. Great information if you're of a mind to start your own business, along with Marc's VC funding for geeks.

On the heels of ETech, Ken and I spent a few days in Santa Monica. Ken's a pilot so it's easy to pop up and down the west coast. Particularly around LA where the municipal airports are so convenient.

We stayed at the Fairmont Miramar, a nice big hotel. Too expensive for what it was, but you can't beat the location right next to the beach and the Third Street Promenade. Santa Monica in general is pretty nice. A bit scungy on the edges like all beach towns, but lots of mellow good feel and a compact enough center you don't really need a car.

We had a fantastic dinner at Michael's, one of Santa Monica's oldest top tier restaurants. Great service, good wine list, very nicely made food, and a comfortable outdoor patio. It was only about a third full on a Friday night, a bad sign, but the food was terrific. We also had a good dinner at Rocca, an achingly authentic upscale Italian place. I had beautiful tagliatelle with pheasant.

The ostensible reason to go to Santa Monica was to visit the Getty. It's a pretty amazing museum. Not so much for the collection on display, which I found sort of pedestrian except for the impressionists. But the architecture is fantastic. Brand new, luminous yellow limestone, views from every angle and a huge complex. Also very friendly. Well worth a visit. If you drive, do yourself a favour and stay off 405; take Sunset Blvd instead.

See also some Santa Monica photos.

The phrase the long tail is now officially overexposed.

I'm a sucker. I show up for my flight an hour and a half early, as ordered, even if the flight is only an hour and I know that SJC security takes three minutes. But I'm on my way to ETech, and I've got a beer and wireless, so life is good.

The wireless is $7 at San Jose airport. Someone should really fund this for free, you know? Still, cheaper than the "special juror rate" of $9 that the San Francisco courthouse tries to press on its citizen-juror-volunteers. And it's money well spent.

I love how airport bathrooms are designed so you never have to touch a surface.

The fooferall around AIM's recent change in terms of service reminds me how the cypherpunks movement has failed. Email is still not encrypted.

It's been at least twelve years since the gauntlet was thrown down. Untrusted networks and servers gives individuals the need to protect their privacy. Cryptography gives us the means. And yet despite the efforts of projects like PGP/GPG email is still unencrypted. Ashcroft knows who all is reading your email.

The problem is the cryptonerds have always focussed too hard on getting things exactly correct at the expense of usability. A big part of the problem is key exchange. PGP deliberately exposes the details of key exchange. This protects you against man in the middle attacks, but at the cost of no one ever using PGP. Contrast to Trillian or Skype which gloss over key exchange niceties. You may be vulnerable to a man in the middle attack, but on the other hand at least you're using some crypto.

I think the thing that will force an email crypto solution is spam. Authenticated senders are strong spam protection. Once you have the infrastructure for authentication the encryption is easy. But that infrastructure is going to be too centralized for my tastes.

Car buying is a crazy process. The salesmen are unpleasant, the negotiation process is at best shady, you can't get the car you want, and a lot of money is involved. Why isn't buying cars more like buying groceries, clothes, or consumer electronics?

This is the third car I've bought and this time I used a broker: Hammer Auto. I was going to buy direct from the dealer but the salesman didn't seem particularly interested in helping me find what I want. So I called the broker, paid them a fee of a few hundred bucks, and let them find me a car and negotiate a price.

I ended up with the car I wanted at a better price than I would have gotten myself, even got a better deal on the trade-in than I hoped for. But the best thing was the convenience. No strong-arm negotiation tactics, no bait and switch, no "let me talk to my manager" crap. There was a bit of weirdness with the broker, but in the end I think it was all an honest mistake. Basically the broker's working for you, not the car company. It's a good thing.

The APC Back-UPS ES 500 is good hardware. It's a $60 uninterruptible power supply, quick battery backup for when the power dips or dies. Think of it like a better surge protector, good when you're in a neighbourhood with flaky power.

The battery is small. I've plugged in way more than I probably should, so I only get two minutes of battery life. But that's about one minute and fifty-five seconds more than I really need. It's nice to have the "smart" UPS: plug the USB cable into your Windows or Linux box and you get a log of outages, automatic shutdown on power failure, etc For Linux, you need apcupsd to do the work.

APC makes a huge variety of UPS products from these cheapo power strips to megawatt scale datacenter power units. The ES line is nice for home use because of the power strip form factor. If I were buying another one, I'd pay the extra $20 for the beefier battery in the ES 725.

1972–1989	grew up in Houston, TX and couldn't wait to leave.
1993	one of the founders of Queer Nation Portland.
1993–1995	wrote a popular emacs mode for HTML.
1990–1994	an active participant in soc.motss.
1989–1994	hair to my waist, went around barefoot, and scrounged leftover food from other Reedies.
1987–1996	was vegetarian, but got better.
1994–1996	worked at the Santa Fe Institute.
1996	drove 11,000 miles around the US.
1996–1999	graduate student at the MIT Media Lab and bailed on my PhD after three and a half years.
2000–2001	founded a failed startup.
2001	spent a month driving around France, Italy, and Switzerland.
2001–now	work at Google.

I just quit playing World of Warcraft. I really enjoyed the game the two months I played it, it's a nicely put together MMOG. But I get bored of games once I learn how they work, and I have other things to do with the time, so it was time to quit.

I went out in style. I gave away all my herbs to the first nice person who offered to help me and all my money to my guild. Then I went home, to Thunder Bluff, launched a bunch of fireworks, and jumped off the highest cliff with nothing but my guild tabbard to clothe me.

Two things interested me in WoW. One was the auction system. I like virtual economics, so I cooked up a bunch of code to dump auction prices from the game, put prices in a database and analyzed them. Lots of interesting inefficiencies in the marketplace, but I don't have the time to conduct a really proper economic analysis.

The other interesting thing is player vs. player combat. WoW was designed from the start to really feature PvP. Alas, the game as launched today doesn't really do it right. If they get the battlegrounds and honor system right it could be a truly new thing.

The trailer for the film of A Scanner Darkly is online. Looks good on Yahoo, or try this direct download. Everyone's talking about the rotoscope visual effect, which is definitely cool and appropriate to the story. But what's most exciting is that a new Philip K. Dick story is being made into a film. And unlike other films there's no way to paper over the schizophrenic horror of this story, so we should finally get the unmedicated creepiness that makes PK Dick novels so great. A Scanner Darkly is the most disturbing and depressing of all his novels, particularly when you ignore the scifi trappings. If they stick to the story, it's gonna be a hell of a film.

I worry about the cast. Keanu Reeves, Robert Downey Jr., Woody Harrelson, Winona Ryder. If they just add Christian Slater they'd have the Asstastic Five. But it could work.

I didn't think it was possible, but the Michelin Guide Rouge is coming to the US! Reviewing 500 restaurants in New York, starting 2006. Finally some serious restaurant ratings, the perfect antidote for the tiresome populism of Zagat's. Sounds like they might dumb down the Guide a bit for the US market. I can't imagine photos adding anything and I sure hope they don't try to disguise being French. Still, good news.

Photocopiers don't let you photocopy money. Modern scanning and printing computer tools don't work with money either. How do they know it's money you're scanning? It's the EURion constellation, a magic pattern of five dots that says "do not copy me". Details in Markus Kuhn's analysis and these images.

It's like a talisman, a symbol with mystical powers that is feared but not understood. The fear is built into copiers and scanners. Adobe went along too so you can't work with currency images in Photoshop, although the feature it detects is not EURion.

I bought a cheap toner cartridge for an old printer. Look at the lovely packaging! Nothing says "honor those killed on 9/11" quite like an unauthorized HP Laserjet 4P cartridge named American Spirit. I wonder if the toner contains actual ashes from Ground Zero?

This is even tackier than Liberty Goldfish.

Ellen Ullman has a great opinion piece in today's NYT, The Boss in the Machine: We need to do more work designing calm user interfaces. Desktop appliations should take a hint from computer games, which in general have learned to replace the clutter of 15 toolbars and status displays with a more immersive, simple interface.

Open Office is good software. At least the 2.0 previews I've been using.

I used to openly mock StarOffice: ugly, clunky, Java where you don't need it, incompatible, slow. Then again I used to make fun of Firefox, too, and look how wrong I was about that. I've been using OpenOffice 2 exclusively on my new computer, particularly the spreadsheet. And it's quite good! Basically, well, it works exactly like Excel. No better, but just as well and it's free. If anything, it seems even faster than MS Office.

I've only run into one problem importing from Excel. Some of my own spreadsheets have some funky formula that work in MS Office and not OpenOffice. On import they get replaced with #NAME with no hint what the broken formula was, so I can't fix it. But that seems unique to my crappy spreadsheets, documents from others have worked fine.

It'd be nice if I could say OpenOffice were better than Excel, but it's not. At least for the simplistic things I do. Well, there's one improvement: the UI doesn't jump around confusingly when I type a space in a formula, trying to let me pick cells by the keyboard. It just lets me type. Nice.

After bragging about hacking my new router I learned that World of Warcraft doesn't work with the WRT54GS v1.1. It only fails in WoW, and only on v1.1 routers. I got as far as analyzing a packet trace and finding some corrupted data packets, then gave up and exchanged the wrt54gs for a wrt54g. Almost the same thing but without the WoW problem. Coming on the heels of being offline entirely (my Netgear RT314 flaked out), I'm sick of routers.

These little routers are amazing: $80 for RAM, CPU, wireless radio, ethernet, power supply, case, and software. But if it breaks or flakes out even a bit, no one's going to help you. It's particularly bad with a problem like "I think the router is corrupting packets from this one application, but only after an hour". Your only hope is to exchange it for something different and hope for the best.

I just bought a new router, a Linksys WRT54GS, solely because the thing is so hackable. Linksys based their firmware off Linux and politely followed the GPL, making the firmware source available. A bunch of smart hackers then made custom firmware that lets you log into the router, tweak wireless power settings, install network tools, etc. You basically get a little solid state Linux box with lots of network ports for $80. Great deal, and smart of Linksys.

The tweaked firmware that got things going is sveasoft. But those guys are assholes, trying to sell the GPL code for money. So I installed the brand new HyperWRT 2.0. It's a very lean tweak of the core firmware that gives you more control over the radio, more routing and QoS options, and a shell with a basic BusyBox setup. You can install batbox on top of HyperWRT for more features.

I love tinkering. I'm the kind of guy who spends half a day tweaking his video card settings for max framerate, then resets it to factory default because overclocking is stupid. I probably won't do much with my custom firmware except boost the radio and maybe install some monitoring on the router. But it's important to me to be able to tweak things.

I buy absolutely everything I can from Amazon. Books, music, gifts, software, hardware, cooking supplies, everything. It's not just because Jeff Bezos is a nice guy. Amazon's service is truly fantastic. Good prices, reliable, and I don't have to drive somewhere in San Francisco to buy something. Lazy consumer.

The inflection point for me was when Amazon started offering free shipping for orders over $25. I stopped worrying about the friction of buying things, just ordered what I needed and waited. And thanks to Amazon Prime, I don't even have to wait that long. Now that I'm in the club I get everything I want sent to me in two days.

It's a brilliant move on Amazon's part. I was surprised at first you can share your $79 membership with family members, then realized that's the viral part. Now my whole household will buy everything on Amazon. I wonder how many more sales they made when they started the free shipping option? Prime can only increase their business.

Sorry about the blog outage; my DSL at home has been out for 36 hours. It's amazing how awful this feels. A whole Sunday with no Internet? How do I get my email? How do I check in at work? Do we have to shudder use dialup?

I remarked how silly it felt that we were already so dependent on Internet access that its absence seemed a crisis. Ken's retort: "yeah, they used to feel that way about electric lights, too".

A big raspberry to SBCGlobal, who took my initial query about 40% packet loss and turned it into four hours of level 1 outsourced tech support hell asking me time after time "which operating system do you have"? (Uh: Windows XP, Linux, MacOS, and a Netgear router). On Sunday they killed my link entirely, and when I finally got a knowledgable tech I learned why; they changed my gateway from 63.194.75.30 to 63.194.75.25. Why? How? Beats me. I'm now looking into alternatives: sonic.net and speakeasy.net.

1up has written an amazing piece, the essential 50 video games. Excellent catalog of the significant milestones in video game history. The individual game articles are well written, good combination of history and critique.

Video games are our culture's newest medium, as significant as the development of film. We're in the 1930 stage right now; a lot of the craft has been defined, but we're still trying to figure out the narrative form. But movies are easy to see, even eighty years later. By contrast video games require complex and idiosyncratic machines. I think only half of the games on this list are playable today unless you have the original hardware lying around.

Here, go play the original spacewar on a Java applet PDP-1 emulator. Brian actually typed in the code listing from a printout.

OK, OK, I finally switched to Firefox at home. I've liked MSIE ever since 4.0, but I like Firefox for one simple reason: tabs. Ctrl-click is my friend. The fact that Ken and I spent nine hours exorcising CoolWebSearch that came in through an MSIE security hole also colours my thinking.

Firefox has a brilliant extension architecture. Here are the extensions I find essential.

Clone Window

Makes it so that Ctrl-N and Ctrl-T make a copy of the current page rather than sending you to a blank page. This is absolutely essential.

Tweak Network

Lets you turn on HTTP pipelining.

TabBrowser Preferences

Lots of options for configuring tabs. Almost none of them are useful, but a couple are handy.

Download Manager Tweak

Improves Firefox's weird download manager UI.

Slogger

Records every web page you ever visited. Annoying for the popup errors when it can't save a page, but useful. Also a nice workaround for the lack of Firefox cache support in Google desktop search.

googlebar

Not just for starting searches, but so that you have the quick find option on your keywords. Shame it can't live next to other toolbar buttons.

BugMeNot

Bypass stupid site registrations.

Many of these tweaks should be in the default install, or else rolled into a "large install".

I've really been enjoying the new Binary Zoo game Mono. It's frenzied Robotron-style gameplay with some beautiful æsthetic and gameplay twists. The graphics are amazingly hypnagogic, lots of tracers and bright colours. Simple, free, and quite fun. The shooter genre is pretty much dead in the commercial world (poor Rez), it's just not the kind of game that lends itself to $20M budgets and fifty hours of gameplay. But the homebrew shooter scene is making some awesome games. See also the BulletML games at ABA games, or ZUN's games.

I've been doing a lot of MySQL hacking in Python. And like all Python projects I do, I start by stuffing data into anonymous lists and remembering "oh yeah, foo[3] is the name of the wine, and foo[1] is the year". This doesn't scale well, and fortunately MySQLdb has a better way. The code snippet above is using dictionaries everywhere; both forming the query and handling the response. This lets me name parameters so that if I add a new condition to the where clause or a new field to the select, the rest of my code doesn't break.

I'm taking advantage of two MySQLdb features that go beyond the standard Python DB API. The first is simple; the magic query construction of execute() handles dictionary style substitution just like you'd expect.

The second is more subtle. MySQLdb supports different cursor classes that extend the basic "tuple of tuples" datatype you usually get from fetchall(). I'm using DictCursor, which builds a dictionary from the names in the description field of the cursor. There are also server-side cursors for efficiency with large result sets. It's all implemented via mixins for flexibility.

I'm particularly looking forward to Andy's 2.0 plan to have a "row object that can behave like a sequence or a mapping (dictionary)", giving you the best of both worlds. Combine that with iterators and you could really have something.

PS: if you search for MySQLdb docs, you quickly land at the obsolete module docs. I used these docs for two years! The MySQLdb project has moved to SourceForge and the MySQLdb docs are nicely hosted there.

I had a nice dining surprise last night. After spending twenty minutes driving around the Inner Richmond looking for a place to park, we gave up and drove back towards home, me grumbling about how SF is choking on itself. On the way back we stopped by Chou Chou, a newish restaurant near the Forest Hills Muni stop.

And miraculously they had a parking place, and a table. And best of all, it's a a great little neighbourhood bistro. Friendly, busy, nicely decorated, good staff, good wine list, great menu. I had escargot (tender, and with a flavour more interesting than just butter and garlic) and a seafood stew (fantastic, quite spicy: almost like an etoufee). Ken had foie gras mousse and a lamb daube which looked rich and delicious. Happy all around.

My part of town is outside what most people even think of as San Francisco. I like it that way, quiet and relaxed neighbourhood. But sadly lacking in good restaurants. Chou Chou is a nice addition. They were packed, too, so I hope they are doing well.

I usually pronounce "URL" like a word, rhymes with "hurl". Most of my tech geek friends do too, probably because it's the shortest pronunciation. That means I say "an URL". But if you pronounce "URL" "You Are Ell", then it becomes "a URL". Which is more common?

"A URL" is more popular. Google has 8,320,000 results for "a URL" vs. 658,000 for "an URL", the new MSN search is 5,086,962 vs. 1,107,446. Yahoo search seems to treat the two queries the same.

These are rough estimates, but it gives you an idea. Too bad search engines are case sensitive: curious the difference between "URL" and "url".

I don't talk much about work on my blog, but I wanted to brag a bit. My project at Google launched! I've been working on an API to AdWords that allows advertisers and third parties to write programs to manage AdWords campaigns. Lots of interesting work, I plan to summarize some of what I've learned at the ETech conference in March. See the AdWords API Blog for occasional updates.

AOL announced they will no longer be carrying Usenet (MeFi thread). What's saddest about this announcement is how few people will care. Usenet is dead.

"Imminent death of Usenet" was a big joke in the heydey of Usenet, in the early 90s. The store and forward network obviously couldn't scale and it had no purpose you could justify on a budget, and yet we all knew that it was hugely vital to the growth of the Internet culture. So it survived.

Then, in the mid 90s, it became irrelevant. We got an explosion of communication technologies: web sites, email lists, message boards, instant messaging. And the culture became diluted as the Internet stopped being the elitist haven for a few nerds.

And so Usenet died, but no one really noticed. There are still a few useful pockets, and a couple of vestigal communities. But Usenet's main purpose these days is as a giant distribution network for pr0n, music, and warez. The community is gone, subsumed into a larger Internet.

The referer spam on my blog has gotten awful. Twenty sites a day show up as supposedly having linked to me. Somehow I don't think prozac.best-buy-site.poker.xxx.tk is really legit, though.

One of the spammers is adminshop.com. They advertise a product named "Reffy", which for $50 (PayPal, natch) offers "mass referrer marketing". Even better, it "comes with a pre-generated list of 3047 active blog websites". Including, presumably, mine. I guess this is my punishment for having an automated referer backlink two years ago, before nofollow.

Dear Reffy author. Your use of my site is not authorized. And because you're advertising access to my site as part of your package, I expect a license fee. I'm a reasonable guy: $5 per sale will do it. Just email me for payment instructions!

Ken does all the cooking. I feel guilty about that, so from time to time I try to make something myself. And having seen Marc turn out an excellent home-made pizza, I figured I'd try making one.

Following my new life of buying everything through Amazon, I set myself up with a pizza peel, a baking stone, and a copy of the pizza nerd book American Pie. And read, and it seemed simple enough.

And so it mostly was. Two things tripped me up. I have no intuition for bread dough, so it was a surprise to me that it's not like clay. You can't just fold it up and start over; the gluten structure won't let you do that. Scratch one doughball. The other problem is that Bad Things happen when the dough sticks to the peel. Note to self: more flour, and don't press down on the raw pizza, ever.

End result was fairly good, especially the second one. Not great, but fairly good. More practice needed.

I played Half Life 2 for a bit, then got tired of it and forgot about it for other things. John Sullivan's letter in the Feb 2005 PC Gamer sums it up nicely for me: Half Life 2 is a good rail shooter with superb art direction. But in the wake of games like Deus Ex or Far Cry it's hard for me to enjoy a game with no real player choice.

I'd forgotten how nice old game music can be. Great crunchy synth sounds and surprisingly lyrical melodies. Check out Silent Light, from Chrono Trigger: ogg or original RSN.

This music is still available thanks to the preservation efforts of the emulator scene. Folks have ripped the music code from the cartridges and published it as files: rsn or spc for SNES, nsf for NES. So you get the original music, not a recording. You'll need a player / sequencer. I recommend NotSoFatso for NES and SNESAmp for SNES, both WinAmp plugins.

Archives of music are readily online. Zophar has a good store of NES music (beware: popups), and SNESMusic is great for SNES. There's a lot of music out there. Some useful guides are Skytopia's list and cly5m's list. I like Earthbound, Chrono Trigger, Super Metroid, and the original Zelda.

When is ten actually eight? When it starts with zero. Base 8 notation is a bad left-over feature from the old days of 12 bit words when programmers only had eight fingers. Yet it persists in our modern computing environments, where almost everything has a magic rule that if a number starts with 0, it must be in octal.

The problem is people sometimes left-pad numbers with zeroes so they line up nicely. For instance, the wonderful ISO date format is written like 2004-09-15. If you're not careful and you try to convert that '09' into an integer, you may get an error because 9 is not a valid octal digit. It's worse with IP addresses: what does '063.194.075.026' mean? Well, that depends on who is reading it.

Octal is the cause of all sorts of contemporary bugs for modern ten fingered programmers. Even Javascript has this holdover from the old days, causing confusing errors when users type numbers into fancy forms.

The only place I've ever explicitly wanted octal was setting file modes with chmod. Because Unix permissions come in groups of 3, base 8 is convenient shorthand that's now deep in my brain. Ironically, chmod doesn't require a leading 0. chmod 600 secret.txt is implicitly assumed to be 0600, or 384.

I love The Amazing Race. Like Survivor, it's a well edited show. And even more than Survivor, it enables us to indulge our base desire to watch people in all their awfulness.

This season has been blessed with a truly hideous couple, Jonathan and Victoria. They're billed as "entrepreneurs", always a bad sign. Jonathan is actively abusive, belittling and physically hurting his wife. And Victoria is whiny and useless, although she gives the abuse right back. It's hideous and hysterical and entertaining all at once.

Of course these portrayals are fiction. The situation is artificial and the reality is heavily filtered through the editing process. Only it turns out that in real life, they're just as awful! Yes, Jonathan and Victoria have a web site. And while they were nasty and mean on the show, in real life they're SoCal nightmares.

Jonathan runs a day spa that "creates the physical manifestation of an epiphany". But really he's a film maker. Victoria is a Playboy model (NSFW). But really she's an artist, making paintings of Beanie Babies. From the bios we learn that Jonathan's favourite colours are "wheat and hunter green" and the question he's asked most is "are you really 42 or 24?". Victoria's ideal date is "sushi, sex, and starbucks" and thinks "the secret to success is always choosing the same thing". Remember, they chose to write these things about themselves. No editors.

This is the joy of reality TV: you can't make this stuff up. On the TV show they're awful and abusive, caricatures of aggressive entrepreneurship. And on their web site they're awful and self indulgent, caricatures of SoCal sybarisis. I give them credit, they've constructed an amazing image.

Looking back on my trip to New Zealand, Queenstown was my favourite place to tourist. That's easy to say, since Queenstown is the tourist center of NZ. But it's a lovely spot in the middle of the Southern Alps and the tourist infrastructure is welcome. Lots to do. Queenstown is the center for "adventure tourism". Bungee jumping was invented there, paragliding, etc. Not for me, but I did enjoy the jet boats, rafts that draw only three inches of water and go 50 miles an hour. Kind of like a roller coaster on an alluvial river. And Ken and I enjoyed the luge on the hill above town. Nice view and goofy fun.

We also took a great helicopter tour to Milford Sound on the west coast: beats the 10 hour bus ride, and the views of the glaciers from the air are amazing. And we enjoyed a leisurely boat cruise on the Earnslaw, a steamboat that was improbably built, disassembled, carried by train, then reassembled to be trapped forever in the lake. And there are a lot of wineries to visit south of Queenstown in Central Otago, an up and coming pinot noir area. Akarua was my favourite.

Queenstown also has the best restaurant concentration we found in New Zealand. We had a fantastic tasting menu at Wai, a great seafood place. We were somewhat disappointed by The Bathhouse; eccentric service, OK but not great food. But the best dinner we had was at The Bunker, a tiny little hidden place with amazing preparation. I ate a lot of lamb in NZ, this was by far the most succulent and with the best flavour.

We found most of these things thanks to the help of Lavinia, our host at the Evergreen Lodge. A wonderful little four room hotel about two miles out of town, up on the hill with a great view of the lake. Great hospitality, the kind of place where you enjoy talking to the host and the other guests. It was good to be out of Queenstown itself, much quieter, but still a quick cab ride into town for dinner. Well recommended.

Many thanks to Uche for his thoughts and code responding to my frustration working with XML in Python. If you're reading this because you want to write good XML code in Python, read his stuff! He knows much better than I. And he gives clear guidance: use his Amara if you want something Pythonic that can deal with XML.

But reading Uche's posts confirms my main point. There are too many XML choices in Python. And the obvious ones aren't right. Apparently PyXML isn't what I'm supposed to be using (despite it being the default when I type import xml on my Debian box), and if you use it the way the docs say to you're wrong. Urgh!

And while I like what Uche says about Amara, is this the easy way to say "parse an XML document"?

He explains why all this is necessary for this example (Amara by default doesn't support XPath attributes), but it's just this kind of complexity that frustrates me. Python's strength is that there's a clear, obvious way to do simple things. But not with XML.

I'd thought Queer Eye for the Straight Guy had passed entirely into irrelevance, but the third season premier for Ray S. (ep. 153) was excellent and heartbreaking. The episode is about a 37 year old soldier about to be shipped out to Iraq. The Fab 5 swoop in to lift him and his woman out of their bleak apartment lives, help them quickly get married, and set up Ray's wife for the year and a half he's about to be gone.

Queer Eye is usually a silly show, with howlingly bad fashions (from both sides), Rip Taylor dialogue, trivial consumer indulgences, and some sappy sentimentality that doesn't work. But here we have a guy who's about to go to Iraq, some truly generous and thoughtful gifts, and an emotional honesty that's rare on TV. Maybe the show has some life in it yet.

It seems appropriate that the show was not overtly political. There's one small "gays in the military" joke, but no comments about what's going on in Iraq, just lots of sympathy for the situation the soldier is in. But you can't watch the human scale of this, this poor guy being taken away to fight a disastrous war started under false pretenses, and not get angry. At least, I couldn't. But the Queer Eye team was smart and stayed away from that. Let their viewers draw their own conclusions.

I was having trouble ripping some badly manufactured CDs on my new CD drive, a Sony DW D22A. I think 48x is just too fast, particularly since the CPU can't possibly keep up and so things buffer in weird ways. But slowing the drive down to 12x seems to have fixed things nicely. The rip/encode cycle is just as fast, but it's quieter and more reliable.

I hate working with XML. It's easy to extract data from simple text files or CSV files, but XML is all nested, and has entities, and lots of pointy brackets. Regexp just doesn't cut it, you really need an XML parser. And for some reason Python is not so great at XML.

Python has too many XML choices. There's the stock Python install, which barely does anything. Then there's what you probably should use, PyXML, which has an ugly hack to confusingly install on top of the default Python libraries. But if you follow the advice of Python's most visible XML expert, Uche Ogbuji, you may think there's something wrong with PyXML and install 4Suite instead, which is the same as PyXML only different. Or should you use Amara instead? Then there's ElementTree which is brilliantly fast and simple to use, but limited, or xmltramp, which is even more hacky. On the other extreme there's libxml2, which is fast and powerful but has an awful API.

Mind you, this is all for the basic stuff, like parsing XML. There's lots more Python XML options too. But what's missing is a clear single simple library to use. PyXML seems the most standard, but it seems very slow and it tries to be more DOM-like than Python-like. I hate DOM.

All of this is a long-winded preamble to my attempt to do something simple with XPath in Python.

More inside ...

I listen to an audiobook a month. The online audiobook market is being sewn up by audible.com, which has aggregated lots of books and periodicals and presented them in a uniform web site. You can buy recordings outright for regular audiobook prices. Or for $15 a month you get one book and one subscription a month.

Sounds pretty reasonable, except.. you don't get MP3, you get some screwed up DRM encumbered format that only plays on certain devices. You're allowed to burn a CD, but you can't turn the recording into an MP3.

Why would I want to buy some crappy format whose main purpose is to restrict my usage? After 10 minutes on the web site I still can't figure out whether your book-a-month is owned by you to listen to in perpetuity or if you only get to listen to it for the month you rented it. Who wants the uncertainty? I'm going to keep buying CDs and ripping them to MP3 myself.

As rendered by a T616 cameraphone.

Enron style accounting comes to the White House. The chicanery is all about "meeting" a Bush campaign promise to halve the deficit. How do you do that? You start with a fake target, then you exclude enormous costs that are a direct result of your policies. I really don't understand how this kind of lying happens in plain sight with no one caring. I guess the only accountability is the voters, and the electorate ain't too good at the numbers.

I had a great time at the circus yesterday, the Pickle Circus, a new-style circus that goes back 30 years in SF. Like the famous Cirque du Soleil, the Pickle Circus focusses on artistic presentation and human performance, eschewing the creepy animal acts and tacky carny crap that's the usual American circus experience.

My favourite act was Sandra Feusi and Sam Payne's vertical tango, a Chinese pole act that was astonishingly graceful and sexy. Chris Lashua's German Wheel act was also excellent, and I liked the juggling and the contortionist. This review describes the show well. Alas, you probably missed it: Sunday the 2nd is the last performance.

The Dell 2001FP is good hardware. Nice, simple 20" LCD, 1600x1200 @60Hz. Great picture quality, very bright, mine doesn't even have any dead pixels. And good features, it rotates to portrait mode, has a 4 port USB hub, takes composite and S-Video inputs as well as DVI and VGA and can even do picture-in-picture display. $800 retail, although you can usually get a better deal out of Dell. Dell has always sold good displays. Somehow they seem to do more than just rebrand someone else's displays.

I had held off getting an LCD because I was afraid running a game at lower resolution and scaling it up would be ugly. But it's not bad, at least with the Radeon's video card scaling. Slightly blurry, but the sharper picture and colour from the DVI more than makes up for it. And the pixels are fast enough (16ms) to genuinely draw 60 frames a second; more in this AnandTech preview for how significant that is.

For New Year's last night I stayed at the Harbor Court Hotel, a medium sized hotel on the Embarcadero just south of the Ferry Building in the YMCA Building. Nice place. Not cheap, and the room was pretty small, but the location is fantastic and the place was good. The top floor view of the Bay Bridge was great.

This hotel is part of the Kimpton Group, a nice line of modest sized upscale hotels. I can also vouch for the Hotel Vintage Plaza in Portland and the Pacific Palisades in Vancouver.