Vipul’s Razor

Collaborative spam database. On Debian just do apt-get install razor and you’re razoring. More work required if you want to report spam.

Pyzor

Another collaborative spam database. apt-get install pyzor.

sa-learn

Bayesian spam filtering implementation for spamassassin. Requires 1000+ training messages. sa-learn --ham --mbox archive.mbox

10,000 people grabbing a 2 gig file is a lot of video on demand. Still, crappy quality. The one sample I saw had decent sound and the picture didn’t shake but the colour was all washed out. The cinematography in ROTK is so beautiful $9 is a bargain.

Folks have found various ways to install custom software on it. There are two basic routes: a full Linux install or adding homebrew programs to the native OS (a stripped down Win2k).

Of the homebrew programs XBox media center has gotten the most attention. It’s a multimedia suite that plays movies, music, photos, etc from the local hard drive, from DVD, or streamed from a server.

XPort is also impressive. One individual has ported about 20 different emulators from Windows to the XBox. Gameboy, Nintendo, Playstation, Atari 2600, Intellivision, Apple ][, etc etc, all running neatly. Quite an achievment.

Many folks have speculated that Microsoft’s plan with XBox is to slowly move Windows into the living room. The existing XBox hardware is already sufficient to do this, it’s just the software isn’t readily available yet. The hacker scene is about a year ahead.

I’m still trying to fix my account after November’s customer service meltdown. For the last six weeks customer care has “been experiencing heavy call volume” with “wait times longer than 20 minutes”. The worst thing is the damned ads they play at you while you’re on hold.

This is false advertising. The fastest 802.11g will go is 20Mbps, not 54Mbps. And in a mixed 802.11b/g network the fastest two 802.11g devices can go is 14Mbps. I verified this with a WET54g sitting right next to a WAP54g. Throughput on an FTP? 12.8Mbps. And this is best-case, quiet network with devices right next to each other. In a real deployment I get 30% packet loss.

Between speed, security concerns, and general flakiness wireless is really not a reasonable option for regularly copying 2 gig files around. Good thing Amazon has a generous return policy. At least I get my $150 back.

Dec 20 was the 20th anniversary of Rumsfeld and Hussein’s handshake. More: National Security Archive.

The other place that the RIAA is setting the cypherpunk vision in motion is their own DRM technologies. Watermarks, locked media, Palladium: it’s like a cypherpunks wet dream. Only it’s a nightmare: the cryptokeys are in the hands of just a few people.

The detentions of hundreds of people at Guantánamo is pretty bad. The Bush Administration has been arguing that since the detainees are in Cuba, they have no rights under US or, presumably, any law. Nice! Fortunately the Ninth Circuit said that was ridiculous since the US runs the camp in Guantánamo.

While Guantánamo is bad, the case of Jose Padilla is horrible. Here we have a US citizen, arrested in America, and the government has been claiming he has no rights. Secret detention by the US government: no lawyer, no charges, nothing. It’s absolutely outrageous, and finally a court said so. Even if the man is guilty of all he’s charged with, that’s no excuse.

Americans have died defending our freedoms for over two hundred years. Bush’s Justice Department seems happy to trample all over that, the courts are finally responding.

Watch Paris and Nicole show up an hour late to milk the cows and spill the milk everywhere. That’s OK, they’ll just ‘work’ at the fast food place tomorrow! Watch the well-meaning Sonic manager try to train the rich girls for a wage slave job. Watch the rich girls ridicule the job behind the manager’s back. That’s OK, they’ll just ‘work’ somewhere else tomorrow, and after the show is over they’ll go back to being rich.

After ‘the girls’ are back in their vapid lives the farmer is still going to be working his ass off trying to make a living with dairy cows and the fast food manager is still going to be working as hard as she can at $7 an hour to make ends meet. And that’s going to be the rest of their lives. Rather than sympathizing with the hard and honourable realities of being lower middle class in rural America, the show turns it all into a cheap disrespectful joke.

My friend Marc points out that in this infographic the flu is Republican.

Of Halliburton’s $2.64/gal, $1.17 is the price they pay in Kuwait, $1.21 is the cost of Halliburton transport, and $.26 is Halliburton’s explicit markup. This is just a tiny example of the cost of oil and defense companies owning the White House.

The Houston Chronicle has Halliburton’s story.

I worked hard to help HTTP caching on my blog. It’s complicated, particularly with the pastiche of dynamic content I have. Used to be 40% of my weblog requests were answered with a bandwidth-saving 304. When I added my linkblog it went down to 25%, probably both because the HTML view changes more often and because I removed ETags support.

Most aggregators do fine. Radio Userland is having trouble since I turned off ETags. And NetNewsWire has a surprisingly low number of 304s, although a quick inspection doesn’t show anything obvious.

Here are the top 10 packet sizes my blog has served with percentages. Left column is SYN and ACK packets, right column is data packets.

1500 byte packets are about 75% of my data traffic. That’s good. I’m not positive why a 1400 MTU is second most popular, except that many webpages recommend this setting, particularly for AOL users. It is a conservative setting that’s likely to work anywhere. 1492 is for PPPoE users, 576 is common for dialup and X.25, and 1420 and 1460 are probably various IP-in-IP tunnelling setups.

The ACK packet distribution surprised me. 40 vs. 60 bytes isn’t going to make a big difference, but why is there so much variance? Turns out it’s TCP options.

40 bytes is the minimum size of a TCP packet, no options. These are usually ACKs or RSTs. 52 byte packets are ACKs that include timestamps. 48 byte packets are SYN packets setting up a connection; they are requesting a specific MTU and usually selective acknowledgement. Finally, the 60 byte packets are SYNs with some combination of previous options, often with a window scale option as well.

What can we conclude? 1500 bytes still works as an MTU on the Internet, but not as much as you’d hope. TCP options really are useful, with many TCP stacks taking advantage of them to tune the connection. I’m surprised to see all the timestamps flying around. They’re useful for measuring round trip time, but that’s overkill for short lived HTTP connections.

This analysis is deeply geeky. I really admire how TCP/IP works. It’s one of the most beautiful engineering achievements of the whole Internet. And despite some shortcomings, it’s served us well for 22 years.

Two unanswered trivia questions. TCP headers have to be a multiple of 4 bytes. Why are they always padded with NOPs rather than using TCP’s padding field? And why 52 and 60 byte packets, but no 56?

It’s brand new. The docs are nearly nonexistent and the library isn’t as Pythonic as one would hope. But it works pretty well! Compare also scapy (less libpcap-like).

PS: I ran into a problem installing on Debian

The SF Chronicle is running a five part series on homelessness this week, presumably timed to go with the mayoral election. The first article Shame of the City: Homeless Island is heartbreaking excellent journalism. So are the photos.

Being conservative, optimal sizes for small images are 1100, 2560, 4020, and 5480. 3000 is no better than 4020.

• access.log reporting tool
• Converter to translate blog entries into Blosxom format
• Data scrapers for various websites
• Pinger for weblogs.com and blo.gs
• Email search engine (using MySQL)
• Parsers for various game datafiles
• CGI to record what music I’m listening to
• CGI for capturing data for my linkblog
• Photo management tool using EXIF tags
• MP3 management tool
• Cell phone data management tool
• Emulator ROM management tool
• Various BitTorrent hacks
• Windows systray app for my surround sound system

The most distinctive thing about our turkey compared to a freakish grocery store turkey was that the dark meat was dense and really, really dark. Much more like a game bird. The drumsticks tasted great but were very chewy. That may be our cooking fault.. The breast meat came out juicy and very flavourful, much better than the usual supermarket mediocrity. There was also a surprising number of tendons in the meat, not to mention a lot of fat in the skin. Good gravy! I’ll probably try a heritage turkey again, but it needs some practice. See also Slate on various turkeys.

Anyway, Thanksgiving dinner is more about side dishes and wine than turkey. The corn pudding with chanterelles and cranberry jalapeño relish were great, and the 1978 Santenay was elegant if a bit light to stand up to dinner.

The tool design is great, very simple and clean (other than the color swatch display; more functional than æsthetic). I love the implementation: a single ‘live’ web page running a bunch of complex javascript.

Alas, the demo of DX:IW disappoints. Fans are up in arms that the game is dumbed down for consoles, lacks the subtlety that made the first game great. I found the demo unplayably laggy until Warren Spector posted some fixes. Even then it only seemed OK; nice visuals, boring gameplay, stupid UI. Kind of a shame: instead of building excitement the demo seems to have made me worried about whether this game is up to snuff.

Thanks to Weaknees I was able to repair it in about half an hour, replacing the drive with a newer (4x bigger) one. I could have done it myself with the info in TiVo Hacks, but the extra $40 Weaknees charged me is worth the half day of aggravation saved.

The experience has bummed me out a bit about TiVo:

• Why did my TiVo fail after only three years?
• The setup process takes eight hours!
• I’ve lost all my TiVo state. Given how much data they track about their users, why couldn’t they have stored my season passes server-side?
• The CPU is too slow. My TV should never tell me “please wait”.
• My lifetime service isn’t migratable, so I won’t upgrade.

The next two films strand us, in the words of the script (and Baudrillard), in the desert of the real. Ugly boring world with stupid Zion hippies. And Neo is already brilliant and all powerful, so there’s no discovery. No pleasure.

This is reflected in the special effects technology. The first Matrix film is beautiful because it is, ultimately, a film. The FX are mostly film tricks (slowing, rotating) and the stunts are mostly human stunts, people on wires. The reality-based effects work highlights the gap between the Real World and The Matrix. By contrast the digital effects work of the second two films is just unreal. And boring.

There was no way to make a good sequel.

The biggest change is built-in 802.11b; no more need for an wireless/ethernet bridge. They also put in a digital out and wrapped it in a more conventional case. Looks like nice improvements.

The real challenge here proved to be doing serial I/O. The excellent PySerial helped. But I’d forgotten how hard it is to write code to read data from a device that’s slow, particularly when you don’t know how much you’re reading. I should have tried out Twisted, like Matt Biddulph did, but asynchronous programming seems like overkill for a small project.

I hope to use this to modify my contact list

The site’s in Japanese. Poetic automatic translation:

My dollar calculations are off; I don’t know my real rate. What’s interesting is the incremental costs. Running CPU jobs at full-tilt takes another 30W on my Windows box, or about $3/month. AMD’s power management bug blows. athcool on my Linux box is saving me about 40W. If I could run VCool on my Windows box it’d save another 30W.

Bottom line - configure power saving on your monitor! And turning off your computer really makes a difference.

Because their customer service is fubar, that’s why not. For the past two weeks AT&T has not been able to help GSM customers do things like, say, activate new service. Why? They just deployed a new CRM system and it doesn’t work.

I’ve been waiting for a week to use my new phone. I’m sure glad they have a 30 day refund policy.

Men look great in hats.

Using the same /scrape URL that torrentspy uses, I tracked the BitTorrent activity of a 1.2G copy of the movie over the last week (from the day after movie release).

For what it’s worth, this graph pattern is common for all the popular torrent streams I’ve monitored. The big variance seems to be how long the tracker survives.

The RDF folks have had at least 5 years to prove how great the semantic web is. Where are the successes? There’s a few random database applications like rpmfind. There’s FOAF. And there’s RSS 1.0, where you can do the same simple thing you do with RSS 0.91 or RSS 2.0 only with twice the markup.

Each of the individual applications using RDF I know of could have been done more easily with plain XML. What’s the payoff for using RDF? Where are the fantastic semantic inference applications? I admit being fairly ignorant of RDF, so educate me. Point me to a practical example where the use of high concept RDF stuff has made an application significantly better.

One of my favourite cheeses, Epoisses, is in a grey zone. You won’t find raw milk Epoisses in the US: the cheese is only aged 4-6 weeks. And only 10% of Epoisses is made from lait cru. Is it the best?

One of the finest Epoisses producers, Fromagerie Berthaut, no longer uses raw milk. Berthaut is credited with rescuing Epoisses production from extinction in the 1950s. According to The Art of Eating Berthaut started heating the milk in 1999 after pressure when a nearby unscrupulous cheese maker had a listeria outbreak.

Epoisses can be difficult, strong smelling and tasting. I’ve had a hugely rewarding Epoisses at La Côte Saint-Jacques and I’ve had Epoisses that was mild and dull. I’m hoping the difference is aging because if it’s the milk, you can’t get the good stuff in the US. I have a Berthaut Epoisses (from Say Cheese) relaxing in my cellar now.

I agreed to pay a significant yearly fee for an AmEx card because it is supposed to have good benefits and customer service. My first experience? I call to activate the card and am connected to the worst telemarketer I’ve ever heard, struggling to read her script and sell me lots of crap I don’t want “while waiting for your data to load”.

In the mail with my card is the form I have to mail in to prevent them from sharing my private financial data with “valued partners”. When I try to register on the website I’m greeted with arbitrary restrictions. No “special characters” in my username. My password has to be 8 characters or less. Apparently email addresses can’t contain + in them, but I have to guess at that from the error “you have left the following fields blank or have provided an incorrect format”. After I give them an email address their stupid software will accept I’m told that I have to open “email preferences” to opt-out of the valuable offers they’ll send.

I can’t find the email preferences page ever again. I try hitting the back button to find it but am greeted with errors. The front page is full of offers to up-sell extra services. I finally find the opt-out page, only to be informed it takes them 2-3 weeks to update my preferences.

Stop selling me crap!

White lily.

wget           # command line HTTP client
-q             # don't print out status
-p             # download related files
-k             # rewrite resources to local names
-e robots=off  # ignore robots.txt
http://URL/    # page to download

It’s not perfect. The file may not be named .html, and some sites (Yahoo news) don’t download completely. But it’s pretty good. I’m using this to archive stuff I linkblog.

linkblog.py

Python CGI that creates a bookmarklet and accepts input from it. Standalone tool.

Blosxom

The weblog engine for the linkblog

linknote

Blosxom plugin for my linkblog to read entries from the single file that linkblog.py writes out

linkblog

Blosxom plugin for my main blog that includes the output of the linknote-driven blog (see below)

include flavour

Blosxom flavour that prints out data with no Content-Type; the main blog calls this in linkblog

lastmodified plugin

Hacked up version of the lastmodified plugin

Shouldn’t much matter, right? Wrong. Apache parses these headers from CGI scripts and tries to fix them up. It can’t parse the one without the space, so it silently replaces it with zero and all my pages look like they were last modified 1 Jan 1970. Argh!

Good things it does:

• Displays seed, leech data like TorrentSpy
• Saves .torrent files
• Resumes downloads on startup
• Gives unsorted view of downloaded pieces
• Fast resume algorithm based on piece files
• Simple bandwidth configuration
• Excellent documentation

The GP32 from Korean company GamePark is the most promising. Powerful hardware: 133MHz ARM, 8 mbytes of RAM, 320x240x16 screen, wireless networking, easily programmable. It has a huge community of hackers. Most promising are the emulators: you can play classic Atari, NES, etc games on this thing. The MAME, Game Boy Advance, and SNES emulators have been hyped but they don’t quite run right yet. At a price of around $210, it costs double a Game Boy Advance. And GamePark is having trouble: the European launch was cancelled last week, but the platform should live on. Still, very cool.

The other platform that has folks excited is the TapWave Zodiac. More corporate / licensed than the GamePark, but the hardware is promising. It’s a bit more standard: PalmOS and Bluetooth. Should be shipping any day.

Turns out the games are lousy too, at least according to MetaCritic. Super Monkey Ball, the best-rated of the 16 games, only garners a 63 out of 100. On MetaCritic any game under 75 is generally bad.

I was surprised to learn that after 15 years the netpbm team has added a new format, PAM. It’s a superset of PBM, PGM, and PPM: the header now has keywords next to values and a new “tupletype” keyword is added to specify “BLACKANDWHITE”, “GRAYSCALE”, or “RGB”. The purpose of this seems to be to support alpha channels in netpbm. Tuple types “RGB_ALPHA” and “GRAYSCALE_ALPHA” are on their way.

These programs all act on a catalog of known ROMs with checksums and disposition. DATs are maintained separately: some sources are Rob’s conversions and Logiqx. Unfortunately the Windows programs are awkward and slow, and no one seems to have a simple Linux port. May be a good job for Python.

The reason your Thanksgiving dinner is always dry and tasteless is because the Broadbreasted White turkey everyone raises is freakish and nasty. The New York Times had a good series of articles about this two years ago.

BitTorrent has a flaw; it’s easy to get 99.9% of a file and never complete because no one has the last few bits. The main use of TorrentSpy is to see if any client has a full copy. Unfortunately it can’t detect if the whole file is out there, but no one client has it all. With the average torrent having only 20-30 clients I guess that’s unlikely.

Fine emulators have a diversity of algorithms for scaling images up. Eagle and 2xSal (aka sai2x) are the ones in common use. scale2x and hq3x are new and promising.

What’s in the HTML that causes this in MSIE? This isn’t popups; that’s a different problem with plenty of solutions.

Screen’s at least 14 years old and still useful.

Playing old console games is easy, but playing authentic Apple ][ games is hard. They were copy protected with bizarre data layouts, manual lookups, code discs, etc.

Copy protection failed to protect the companies’ profits, but it makes it harder to preserve Apple ][ history. The popular Apple ][ archives don’t serve the original game; they serve cracked versions. They mostly work, but if you’re into pristine preservation it’s not quite right.

I wonder how folks will play today’s PC games in 20 years. I think the DirectX API will make emulation easier. But the games are still copy protected. There are emulators for today’s PC CD protection, so maybe preservationists will be able to play the original. And there are cracks too, but most are rips that strip out a lot of game content to make the download smaller. Ugh.

Props to the SNES emulation community for making it so easy to see these old games. There are lots of good SNES emulators. SNES9x is a well behaved Windows app; ZSNES is funkier but has better realism for video and sound emulation.

At its best this genre can be a great form of social criticism, highlighting details of our society. But Hominids is just tedious. There’s the chapter on religion, and the chapter on monogamy, and the chapter on crime, ... And woven all through it, a cartoonish story of a recently-raped woman learning to love again. Ugh.

• Should I wear the shoes that are easier to take off?
• Can I put the digital camera in my checked luggage or will security steal it?
• Make sure the laptop is charged in case they inspect it.

I’m not one to make displays like that so it was an accident it came with me to New York. But now where do I put it going home? In checked luggage, where security may find it while I’m not around and decide to punish me for being clever? Or in my hand luggage, where it may cause my bag to be searched and an awkward conversation? Maybe I should just leave it behind.

One if by Land, Two if by Sea: excellent food, very good service. The first course of the tasting menu was my favourite - cabbage, foie gras, salmon roe, arctic char, finished in a cream sauce. Incredibly rich and delicious.

The interesting thing is the dense sound from having 25 people playing and singing together. The deep choruses of “Soldier Girl” and “Light and Day” are really great. Some of it is bad, though, the remixes and anything where the main guy is the only one singing.

I was surprised to realize this was the first upbeat music I’d enjoyed in a long time.

Tron is an important waystation in cyberpunk, borrowing heavily from Metropolis in its milieu and design (particularly the character makeup!) and The Wizard of Oz for the fantasy story. In turn it has inspired films such as the animation in Johnny Mnemonic and the navigational scenes in The Second Renaissance (The Animatrix). The cast is surprisingly good, too: David Warner, Jeff Bridges, Bruce Boxleitner, even Barnard Hughes is fun as that guy.

The new Tron 2.0 PC game is a pleasure. There was talk of a Tron movie sequel but now the rumour is there is no film. Probabaly just as well.

I believe computer games are some of the most interesting commercial art being produced today. Shame to see the commercialism screw it up.

The Internet should not be a battleground.

I’m good at patching my system and the place the virus phones home is offline, so I’m presumably safe. But how would I know for sure? And why did the active virus protection allow this file to be written at all?

Java is useless for embedding in web pages anyway. I probably should turn it off.

The question is how many games the market will sustain. So far there have been a few smash hits: Everquest and Ultima Online of course, but also Dark Age of Caemlot and maybe Star Wars Galaxies. This graph of subscriber stats suggests a bunch of new games are taking off without really cannibalizing existing populations.

The historical record is astonishing as well. The Kalevala, the Finnish national epic, may record the story. Pytheas of Massalia was in the area 350-325BC and wrote of “the grave where the sun fell dead”. The Argonautica of Apollonius of Rhodes (3rd C. BC) may describe the lake:

Google has 153 pages about this spam.

• Fission reactor: 46 years, .12c
• Fusion reactor: 46 years, .12c
• Antimatter reactor: 41 years, .66c
• Laser sail: 12.5 years, 1.0c
• Fusion ramjet: 25 years, 1.0c

But there’s rumours. She really was eaten by that cougar. Even better, next season is rumoured to feature Kim as a lesbian. It’s every straight guy’s fantasy!

It may be a hoax. I tried to find more info, but a search for Elisha Cuthbert Lesbian finds rather different material.

Why do user interface designers think it is ever appropriate to prevent a user from taking an action?

I imagine Atari is trying to head-off pirates stealing the game on KaZaA. Or maybe they’ve found a clever way to get their demo out. Or maybe they’re brave and are actually trying a new distribution model? Greg Costikyan has written some good stuff on online distribution. The movie industry better be paying attention; they only have three years to figure this out themselves.

There’s no way I’m installing the evil scumware that comes with Kazaa and Kazaa Lite is failing me. Bittorrent would be a perfect alternative. I fear it will be about six days before a crack for the time limit is released. There’s a reason demos don’t have all the game assets.

Like all cool software that seems vaguely illicit, Kaillera has some association with Nullsoft. The download is hosted there and the Kaillera author credits Justin Frankel for the net code.

I took me 86 quarters and a couple of hours to get to wave 101. The game doesn’t really change much after wave 60. If only I had two analog dial controllers, it’s a really fun two player game.

This is much easier than the contortions I went through trying to record a RealAudio stream via Total Recorder. Great for recording game soundtracks. And it’s a simple solution for bypassing whatever consumer-hostile DRM crap the rest of your computer tries to foist on you.

This is my first time fooling with Ogg Vorbis. I’m impressed, particularly it’s handling of very low bitrates. I was able to make an Ogg half the size of the smallest listenable MP3. Useful for bandwidth-starved blogs.

The code is quite a hack, using Perl Inline to let me write the actual text processing in Python like Rael’s demo. There is one neat trick: Inline->bind() lets me defer the import of Python until it’s actually needed, meaning there’s no efficiency cost if the Python code isn’t invoked.

The Romanek links may break until the crowd dies down.

Why? Base64 encoding breaks up the pattern so the compressor doesn’t work as well. Base16 preserves byte boundaries. It’s even more efficient to gzip, then base64 encode, then gzip. Summary:

data                      909k
gzip(data)                248k

gzip(base16(data))        296k   base16 is smaller
gzip(base64(data))        385k

gzip(base16(gzip(data)))  280k
gzip(base64(gzip(data)))  250k   base64 is smaller

Unfortunately, GET-based WSDL doesn’t seem to work well. I can’t make Mark’s WSDL work with either Java’s Apache Axis 1.1 or Perl’s SOAP::Lite 0.55. Neither seem to find any methods to invoke. There’s also an issue of getting .NET to do authentication; that may be solvable.

I’m seeing the same problem with Paul’s Google wrapper. WSDL is one of those squishy specs where stuff may be ‘correct’ but it doesn’t work with any of the tools. I suspect the issue here is since no one is using the GET binding, it just doesn’t work in many places. Frustrating situation.

There’s a good overview article from Rob (along with some Slashdot kibbitzing). Also a 1997 patent filing. I couldn’t find Rob’s PhD thesis online, you’ll have to look it up in the MIT library. His master’s has been taken offline but archive.org’s Wayback Machine still has it.

Torcolato is made in the recioto style - the grapes are allowed to dry on the vine, usually picking up some botrytis rot. The result is a sweet wine not unlike a Trockenbeerenauslese. My glass had a lovely balance between sweet and acid, without the heavy cloying botryized taste that Sauternes can have. Very nice.

I know a fair amount about wine; it’s nice to be surprised with something new.

Pineau des Charentes is not wine. The grapes aren’t fermented, instead year-old cognac is added to fresh grape juice before the juice has a chance to ferment. The result is a sweet drink, very pleasant, and honesty if I hadn’t been told I’d have assumed it was a dessert wine. It’s usually had before dinner, but it’s heavy enough I prefer it after. It’s not terribly expensive.

I thought I knew a fair bit about wine, and we gave the nice lady in Sooke a hard time about wine selections. I think she had fun fooling me with something new.

Turns out that while most user browsers support gzip encoding, most spiders don’t. GoogleGuy says this may be because servers don’t reliably serve gzip. I could believe that given the contortions I had to go through.

RSS aggregators are mostly good about supporting gzip. They are good about handling 304 Not Modified, too. Good thing; RSS polling is such a huge source of traffic.

I just tried it out with Scott and was impressed. Installs with no trouble and no scumware. Looks like an instant messenger app. The voice connection to Scott just worked. The sound quality was great even with our crummy desktop microphones. It was only half duplex, but that may just have been one of our sound card setups.

The network details are good too. Despite us both being behind NAT routers we got a direct P2P link going at the click of a button, no configuration required. Traffic was a steady 11 kbytes/sec (just perfect for my 16kbps link). The packet trace of connection setup is a bit odd - it talks to a bunch of different IP addresses. But hey, it works!

Bryan also has a draft RFC for NAT P2P. It’s a great document and includes a technique I’d love to see developed further: opening TCP connections through NATs with a “simultaneous open”:

The only alternate I know of is UPnP Nat. There’s hardware router support and a free Linux implementation.

Not many uses on the net; the earliest I could find is a reference in a June 1997 post on alt.fan.frank-zappa.

Cables to Go also sells a 3-in-1 PC extension cable - keyboard, mouse, and VGA all bundled in one. Great for moving your noisy PC into a closet.

Not easy, but I found a couple of leads on sound libraries. Audiere is nice, but its Python binding doesn’t do much more than “play sample”. There’s a good Python binding for fmod which uses swig to generate the code. It has a lot of functions, but it’s organized like a C API and some of the type mappings (such as float *) aren’t complete.

All this stuff is very low level.

I don’t kid myself that Al Jazeera is perfect, but it’s important to have alternatives to American “fair and balanced” crap that masquerades as objective.

BTW, the best steakhouse in San Francisco is Alfred’s.

The part that makes this remarkable is they can do this 30 times a second at 1280x960 resolution, thanks to the magic of Cg. Cg allows you to write programs that do things like calculate per-pixel shading to create all sorts of crazy effects in real time. Consumer video cards have an astonishing amount of parallel computing capacity.

I went a few years ago and had a great time. It’s a huge event full of prize pigs, hucksters, ferris wheels, fried bread, and lots of ordinary people. Much fun. This year I can’t decide what I’m more excited about: the 600lb performing pig or the corn husking.

I asked a question on Google Answers: How much has the average American’s food consumption grown over time? According to this USDA report calorie consumption has grown about 20% in the last 20 years.

You can pick up a 5 CD box set of Jimmie Rodgers for only $26 on Amazon. Worth the investment. I didn’t grow up with this music, but after the success of the O Brother, Where Art Thou? soundtrack I’m getting interested.

My searches led me to this fine document on audio dithering. If you have sound in a format that’s more accurate than your final output (say you’re doing 32 bit processing and the end result is a 16 bit stream) then you don’t want to just round off the sample - you want to dither in some noise. Why? It allows the lower bits to occasionally be expressed in the output and your ear is capable of picking the signal out of the noise.

The audio demos from the page are impressive. Effects are exaggerated (rounding to 8 bits) so you can hear them. Read the document for explanation.

• Simple roundoff: bad
• Noise: better
• Shaped noise: best

My little pinger.py script works a different way, running as a cron job periodically and checking if it needs to ping. Not robust; only recommended for Python hackers.

Today’s exercise was a little program to tell me when my Creative sound driver has been set behind my back to something other than 6.1 surround. It reads a registry key every 15 seconds and updates a system tray icon if things have changed.

Doing it in Python didn’t prove too hard. The bloat is bad though; the program is 12 megs in RAM and a 2.5 meg standalone distributable. Still, nicer than trying to remember how to program with Visual Studio.

I found nice sample code for system tray icons: systray_py.txt and Flashing Taskbar Icon. To create a system tray icon for a program:

If a program ends in .pyw, then pythonw.exe is used to execute the program and a DOS console window doesn’t pop up.

icoutils helps build Windows .ico files.

Update: Boa Constructor helps you do visual form design. Thanks, John.

py2exe is good for packaging up programs to redistribute to folks who don’t have Python or required site-packages installed.

regmon is useful for figuring out what registry key to watch. Turns out, it’s

Lots of details.

Vera Sans Mono fills the need for a good fixed width sans-serif font; Lucida Console just doesn’t cut it. Vera Sans is a decent proportional font but is too wide; I still prefer Arial. Vera Serif looks hideous, but then all serifed fonts look hideous on screen.

Works on Windows. Download, SlashDot discussion.

Jon Krakauer, of Into Thin Air fame, uncovers fundamentalist Mormonism in his latest book Under the Banner of Heaven. Yes, fundamentalist Mormonism: splinter groups who hold various radical theological positions such as celestial marriage and apocalyptic fervour. The book is ostensibly a history of the murders by Ron and Dan Lafferty, but really it’s an excuse for Krakauer to explore the curious history of the LDS church.

The narrative is unflattering, particularly its characterization of the early days of Joseph Smith and Brigham Young. Parts seem unfair. But Krakauer’s central thesis is fascinating - Mormonism was born out of violence and zealotry and this extremism is alive and well in the fringes of Mormon culture.

The ironic thing is NTP is the most lightweight useful Internet protocol in existence. A server can handle hundreds of thousands of properly functioning clients; when it works it takes one UDP packet every 17 minutes to serve a client. I did an NTP survey back in 1999; a beautiful peer to peer network.

Let’s enumerate the problems:

• I don’t use MSN Messenger, I use Trillian.
• WinXP forces me to have MSN Messenger installed whether I want it or not. Did Microsoft compromise my security when they forced their IM system onto my machine?
• I got 5 different emails to the same address.
• The email came from hotmail.com, not microsoft.com.
• The email is titled “Important”, thereby almost guaranteeing people will ignore it.
• The email is sent as HTML but then the contents are just wrapped in a <pre> tag.
• Why not just use Windows Update?

Note: pretty much all these links are adult content.

Only it doesn’t work so well in HTML. HTML 4.0’s entities don’t define &zwsp;. IE recognizes it but Mozilla doesn’t. So you need to use &#8203; instead. And then IE screws it up when you paste it into ASCII. And while IE6 on WinXP renders it correctly, on Win2K it renders a box. Totally broken.

There’s a good page on line breaking in the Web, detailing all the problems where lines are broken where they shouldn’t be and vice-versa. What kills me is these typsetting algorithms were all solved almost twenty years in TeX, at least for English. Why do the people who do HTML and web browsers hate design so much?

Blosxom supports two kind of permalinks: date based and file based.

Date-based permalinks look like this:

File-based permalinks look like this:

blosxom.cgi, with no flavour files installed, uses date-based links. So do the flavours in the Blosxom flavour sampler. Many of us, including the author of Blosxom, prefer file-based permalinks. The joy of Blosxom is either kind of permalink will work. By editing flavour files you can easily decide which style you serve.

Personally I think the date-styled permalinks are crazy and Blosxom’s defaults should be file-based.

I don’t think any of this would be a huge business, but it’d be enough to fund BitTorrent development. Customers: anyone hosting downloads of more than 10 megs.

And big crazy. Check out this movie of someone flying twenty feet above the ocean with a kite and pulling a guy on a board behind him (for ballast, one presumes).

I’ve thought for awhile about moving by blog to a new domain name but I can’t come up with a good one. Hmm, boingboingboingboing.net is available.

They sound good. Having speakers behind you makes for a big difference, particularly in games. Your eyes always stare straight ahead but you can hear all around you. So why not have speakers behind you in your VR rig?

What I like best is that the product packaging is remarkably good for the price. The speakers look nice. The speakers come with long enough wires to reach the back of my office comfortably. It comes with little stickers to label each speaker. The power transformer is on a cord so it doesn’t cover multiple plugs.

I’m so used to crummy packaging of computer products it’s nice to buy a box of stuff that just works.

My real goal is to make my Mac friends envious. Who wants elegant design when glowing fans are only $7.99?

Game demos are starting to be distributed via BitTorrent. Perfect use: lots of enthusiasts, giant files (200 megs is common). Places to go for legitimate BitTorrent game files: GameTab, 3dgamers. Try out the cool Tron demo!

The story is a bit tough to follow at times, but the artwork and presentation are fantastic. My only frustration is that they use the Web medium so well I can’t imagine these ever being printed.

Also not to miss by the same artist: Apocamon, the book of Revelation reinterpreted as manga.

The <summary> element specfies it’s plain text, no escaped HTML. What happens if I want an HTML entity in there? Am I not allowed? I think <summary> and similar elements should have the same content encoding attribute that <content> does.

My no-content Atom feed is 6530 bytes (1560 bytes compressed), my no-content rss091 feed is 3868 bytes (1284 bytes compressed). Most of the extra space seems to be timestamps.

<entry> elements seem too wordy. <id> is redundant with <link> in Blosxom. I don’t see why I need two timestamps (<issued>, <modified>), identical except for timezone. Recommendation: make <issued> and <id> optional (if <id> is missing, substiute <link>).

<copyright> is optional. Some lawyer should reassure us that its omission does not imply the feed is not copyrighted. I’m happy with implicit copyright and would rather not waste 50 bytes in every single download.

Generating proper timestamps is hard. Timezone is the bad part: -0700 vs -07:00, local vs zulu.

It took me 30 minutes to get something basically working, an hour to customize it to my satisfaction. Too long, but it will get better. Overall the spec is shaping up nicely.

You can fetch the Blosxom bits I used to put this together. I hacked some of the plugins to get the feed to validate.

The more outrageous claim is the fooferall over Robinson’s support of OutRight Concord, a gay youth outreach group. The problem? An affiliated site, Outright Portland, had a link to bisexual.org, which in turn ran a banner ad for threepillows.com, a pr0n site.

Hopefully the good people of the Episcopal church understand what character assassination is.

My blog is my space, my publication; I’m not running a message board. Every time someone posted something rude or dumb I winced. And when someone posted something smart I got frustrated because I didn’t have a good way to discuss it with them. So from now on, if you have something to say please email me.

Many thanks to QuickTopic, who runs a great reliable, free, simple discussion board service.

Spiders are a really dumb way to index the web. Too bad more clever solutions don’t work.

The game designers seem to have freely indulged in the cheese of the original movie, too:

Having WiFi definitely hampered my attention to the conference. I’d say 2/3 of the time I was on the laptop I was doing something other than Gnomedex. But at least I was there. And the other 1/3 was doing conference-related stuff, particularly kibbitzing on the talk on an IRC channel. That was pretty rewarding.

I am a bit alarmed at how many offhand comments I made on IRC or at the party ended up at blogs attributed to “Nelson from Google”. I’m just another person, I like to relax too. Hard to do that when every word may be replayed. (PS: John, I meant it nicely!)

I’m experimenting with being online during a conference, even though I said it made me uncomfortable. And I’m liking it. It is a bit of a distraction, but being able network with the community that’s here is fun. And IRC is a nice icebreaker.

But using chat technology to allow a side channel during a lecture does seem useful - it augments the meeting, it doesn’t replace it. I particularly like the idea I first heard from Clay; project the chat room logs to the room so the chat becomes part of the event.

There are a variety of these programs for the PC - ThumbsPlus and ACDSee are popular. I like CompuPic because it was reasonably priced ($40) and because it worked best. The app is very fast: they seem to have optimized a lot of image manipulation algorithms and take advantage of hardware acceleration.

I need that tool now. Do you know what I’m talking about? I’ve tried searching Google but am overwhelmed with crap and I can’t find it on Memepool or Metafilter.

Tuning the bullet swarm is a fine art. So why not describe bullet swarms with XML? BulletML does exactly that. Try the Java applet (select a file other than template.xml, press start, move mouse in upper left window). Or play a stylish Windows shooter that uses BulletML: rRootage. (There’s a Mac version, too.) rRootage goes further with Bulletsmorph, using genetic programming to come up with new bullet swarms. Crazy.

There’s lots of commentary on rRootage, more than you’d think a simple game needed. I think people miss simple shooter games.

Pretty useful but not exact. For instance it can’t quite match the Google logo font. The closest match Diaconia is handsome but not quite right - the Google logo font has an odd lowercase ‘g’ with the connector on the right, not the left. Serifs are off too. Still, WhatTheFont?! is useful for the “find me fonts that look like this” problem.

MyFonts credits the University of Birmingham for development of WhatTheFont?!. They also point to Identifont as another font finding tool. Identifont looks like an expert system: answer a bunch of questions and it comes up with candidates.

• $11 trillion: US stock market
• $21 trillion: US bond market
• $10 trillion: US gross domestic product
• $07 trillion: US debt

The central problem in font selection is “I need a font that looks like this”. myfonts.com tries to help with that - fonts grouped by category, a “fonts like this” button, and even the ability to browse other user’s albums of selected fonts. Alas none of it quite works. But if you stumble around you can often find what you need anyway.

Why is it that both Bushes preside over rising unemployment rates?

Latcham Winery

One of our favourites, excellent and friendly with reasonable prices. Good Barbera right now, smooth.

Lava Cap Winery

A new find for us. Very powerful wines - the 2000 Zinfandel reserve is terrific.

Cafe Luna

Excellent Placerville restaurant, great local wine list.

Zachary Jacques Restaurant

Frenchman transplanted to California gold country. Odd place with funky decor and excellent food. Gateau St. Honore!

Cary House Hotel

Easy and nice hotel in Placerville. Service is not well trained but it’s a good place to stay.

Sutter Creek, CA

Cute touristy town, excellent antiques store. Good place for lunch and milling about between wineries.

Just yesterday, the Federal Energy Regulatory Commission refused to let California out of the predatory energy contracts it signed after being 0wnz0red by energy companies in 2000 and 2001. Yes, the same FERC who said those companies illegally manipulated the market.

FERC is executive branch. Bush’s friends run the energy companies. They continue to profiteer off their market manipulation. And 101, the major corridor through the Bay Area, is snarled so Bush can sell even more access to his administration.

Scalia’s minority report shows he’s still hateful:

Service was fast and well executed, the price was fair, and now I have a new hot water heater and no cold showers.

Blogging about domestic plumbing problems is awfully dull, but I’m amazed at how much trouble stuff like this is.

I have the same problem hiring landscapers, architects, etc. The sewer guys I hired to fix a problem a few months ago charged me way more than they said they would and did a sloppy job. They didn’t even haul off the old pipe!

This problem seems unique to modern society. The Internet makes it easy to comparison shop and buy goods but finding quality services requires local community. Alas, craigslist doesn’t help, either.

Of my $4000, $1400 is for a new title insurance policy just in case, you know, somehow the title went bad in the last year. Another $700 is to pay the escrow agent who never returned my calls and delayed funding a week because they gave the lender the wrong routing number. For 3 days I pay interest to both lenders - in the 21st century it takes 72 hours to move money around the US. Another $1900 in random fees - appraisal, “document fees”, etc, total junk.

Why don’t banks just let their lenders adjust rates downward, saving everyone the trouble of refinancing and keeping the customer? I tried calling my original lender; they never even answered the phone.

Crazy thing is with this inefficiency I still come out ahead.

RoN sets itself apart from other RTS with a feature borrowed from Alpha Centauri - territory. You can’t just waltz your superarmy in, you have to fight for control of land. It gives the game a lot of strategic depth. Otherwise it borrows very effectively from the RTS genre.

I’m looking for people to play online with - if you’re interested, drop me an email at nelson@monkey.org.

The principle of least surprise is important in scripting languages. Perl fails this principle.

The biggest problem with Courier is that it has serifs. Serifs have no place on screen fonts - 100dpi is not enough to do them right at normal text sizes. That’s why I override fonts in my CSS to be Helvetica and Lucida Console.

• wall
• wall with windows
• graceful plaza

The Internet is grand - lots of shops selling Kubrick. Froogle is helpful for finding things. JList, JTL, Kid Robot, The Outer Reaches, and Giant Robot. Prices are all over the map.

I found my wireframe Tron at a Japanese store, but I don’t read Japanese and the shopping cart interface has defeated me. I see something about ¥14,200 too, ouch.

One thing that was frustrating when I started my company was how little information there was about the VC industry. I had a few friends, colleagues, and angels who gave good advice. But there was a shortage of open conversation. Maybe blogs will open that up; Venture Blog, from some of the folks at August Capital, is remarkably frank.

I bought my first house last year after calculating just how good a deal a home loan is. Now 16 months later I’m already refinancing, saving hundreds of dollars a month just by signing some papers. It’s crazy.

The bad news is if we have a long deflationary period I’m going to feel real dumb holding all this debt. Or if we have a major earthquake, or if the insane San Francisco housing market finally collapses, or.. I’ve never felt so much like an adult as when I signed the loan papers.

Call me vain, but I work hard on presentation. I want people to see my brilliant words here, with my own visual design. RSS is great to let you know when something’s new but it doesn’t let the author have any control over visual layout. You can’t even easily include HTML tags in RSS. And fancy stuff like my carefully crafted mouseover images doesn’t translate well to RSS, particularly if javascript is involved.

That’s why I don’t put the full entry in my RSS feed; I want you to come here in a real browser. I’m not totally happy with this restriction but I like it better than the alternative.

An ad seen today on nytimes.com.

Here are step by step instructions for taking the groovy ultrawave tile and turning it Blogger orange.

1. Open the image
2. Set mode to RGB: [Image/Mode/RGB]
3. Turn the image grey: [Image/Colors/Desaturate]
4. Set the foreground colour (I used 0xff6600)
5. Create a new layer of that foreground colour
6. Set the mode of the new layer to multiply/burn
7. Flatten the image: [Layers/Flatten Image]
8. Adjust colour to suit: [Images/Color/Hue-Saturation]
   I bumped Lightness up +26
9. Downsample colormap: [Image/Mode/Indexed 16]
   Why? Looks the same, saves space (19k to 6k)
10. Save the image

This technique only works for creating single colour images; it’ll ruin fancy tiles with many colours. And show care for your reader’s eyes: this tile is actually pretty awful.

My favourite is still The Second Renaissance; you can watch it online. Of the ones not on the net my favourite is Kid’s Story for its chilling story of an angst boy finding salvation in The Matrix. Peter Chung of Aeon Flux fame also has fun with Matriculated.

The animation has so much detail it looks better to me on computer screen than TV. And shame on the DVD programmer for making such a hostile DVD.

Michael Sanders’ book From Here You Can’t See Paris: Seasons of a French Village and its Restauarant does the American reader a favour by stripping away the romantic picture of quaint French villages to explore what life really is like. And it’s not all vacation: the hard rural work, limitations of relative isolation, and the slow death of French agrarian life. But life in the Lot is good, too. Strong community, excellent local food and wine, and a small breath of life in the destination restaurant that is the center of the book.

I’m thankful to the author for so effectively conveying a lifestyle that is very close to my desire yet very far away.

Huh? My friends who like Perl claim that what’s going on is that Perl is evaluating the right part of the s/STRIP//e, which is empty, as a Perl expression. And apparently in Perl 5.8 this empty expression doesn’t evaluate to, say, an empty string. No, it evaluates to some previous expression’s value.

The lack of a warning may be a bug. My fear is some Perlmonger decided that the empty expression evaluating to something non-empty is a feature.

PS - yes, I know the /e is unnecessary here and removing it fixes my problem.

Geocities is not your only Krull infostop, either. You can find cheats for the game, an online RPG, someone who has made a replica of the Glaive, a loving review, and even a video clip. Amazon will ship Krull on DVD to you in 24 hours for $18, but alas Liam Neeson didn’t choose to participate in the special features.

The British government was caught plagiarising 12 year old student papers to justify the war in February. Downing Street defended the report anyway, but now Blair can’t escape charges the report was hyped.

The story is also falling apart in the US. The Marine commander in Iraq says intelligence about chemical weapons on the battlefield was “simply wrong”. CIA insiders are breaking ranks, shifting blame from analysts to policy makers. Colin Powell himself reportedly thought the weapons argument was “bullshit”. Wolfowitz admits that the weapons issue was used because it was bureaucratically expedient.

And the story of Jessica Lynch, rescued POW, has been overblown, probably deliberately. Her family is forbidden to talk about it. So far Jessica is keeping her head down, but reports that she has amnesia are false.

The pressure is on to find weapons of mass destruction in Iraq. The US has refused access to Iraq for UN arms inspectors, so it probably won’t be long before the US “finds” something on its own.

The political situation in the US is so bad that my friends (pro and anti-Bush) have an unspoken agreement just not to talk about it. Silence = Death.

I did a survey from three sites of dubious legality: torrentfiles.com, animetorrents.com, and torrents.co.uk. In 870 .torrent files I found 32 different trackers. The top 4 trackers accounted for 75% of all the files. Not a lot of diversity. Of course, my sample is biased.

I was also interested in how folks use the BitTorrent metainfo. About 80% of all files use a piece length of 256k, followed in popularity by 512k and 1024k. I also found a bunch of unofficial tags: path.utf-8, creation date, comment, and md5sum.

I’d like to do a more formal survey with a wider sample; this torrent search engine claims 8200+ files.

info                
  length          41470132
  piece length    262144
  name            Halo2_E3.wmv
  pieces          [159 SHA-1 values]
announce        http://news.gametab.com:6969/announce

But it’s easy to criticize; BitTorrent is still awfully cool.

I’m trying to watch Red vs. Blue. So I download the DivX (encoded to 84 kbytes/sec), set it up to watch full screen, then sit back from the monitor. Ugh! Blurry, blocky, smeared. I’m watching the watery reflection of a movie. It looks better if I don’t play full screen, but what’s the point?

I’m beginning to think MPEG-2 set the right limits on encoding. My experiments with DVD ripping left me thinking the 5:1 disk space savings of DivX wasn’t worth the quality loss.

My favourite glitch group is Autechre. Their Tri Repetae++ is brilliant, but it was Chiastic Slide I first noticed. I picked it up cold at a record store because of the beauty of the Designer’s Republic cover art.

The new album is Draft 7.30. So far it’s a bit rhythm-heavy for me, I prefer the lushness of Amber. But I’ve hardly listened it yet.

More Autechre: discography (with cover art), Warp Records site for Draft 7.30 with some RealAudio samples, and the Amazon search leads to albums with lots of samples.

BitTorrent is pregnant hackerware - it works great but is still poorly documented. And the apps aren’t well integrated. Someone could build a killer tech company out of it. Some extra info is available in Brian’s BitTorrent FAQ and the BitTorrent Wiki.

Me, I started downloading by doing

Facial muscle modelling allows them to do dynamic expressions and lip-sync, in-game and in realtime. Physics modelling not only gives each object mass and inertia, but also complex surfaces to tumble against and flexible shapes to bend. It’s harder to judge the AI, but if what they say in the voiceover is true it’s pretty amazing. It’s like cutting edge movie CGI five years ago, but realtime on your PC.

All this technology will make for a beautifully immersive experience. I hope the story will match - not much in the trailer, but the modelling of the locations was so beautiful I’m willing to bet they’ve got a good backstory.

Between Half Life 2, Doom 3, and Deus Ex: Invisible War it’s going to be a good year for first person shooters.

Boy I hate Perl. Why do I have to type ;, $, and @? Ugh. Python rulez!!1@!

What I don’t understand is the diversity of source IP addresses. Are they really users of www.gevasys.de, www.mod.gov.sk, and cair.res.in? Do they 0wnz0r those systems? Can they spoof source IP? Update: Todd, who was hit too, says that the hosts that accessed us were running open HTTP proxies.

I found a pattern to detect the spam, so for now I’m ok. Won’t last though.

Nevermind, that was last season. Wrong annoying character.

SpaceMonger is good software. It tells you where your disk space is going using a two dimensional TreeMap layout. MarketMap is another interesting use of this visualization technique. (Java, caveat browsor).

I think SpaceMonger replaces my favourite disk visualizer, the ten year old xdu (née xdiskusage).

Please tell me if you see any caching weirdness.

The issue is that mod_gzip includes the following header in all responses:

Unfortunately it also triggers a bug in MSIE - the browser won’t cache any document with that header! So with mod_gzip 95% of the world’s browsers won’t cache any pages from the server. Some bandwidth savings.

It’d be nice if mod_gzip was smart enough not to add the Vary: header if it didn’t compress the file, but it’s not. A partial workaround is to turn mod_gzip off for files it won’t be compressing anyway, like images.

Part of the problem with the most recent flaw is that it had been there for months and Microsoft has no way of telling how many accounts were compromised. Nice.

Centralized authority is dangerous.

First I need to get the bits. RealAudio streams are deliberately difficult to copy. Thanks to a friend I found Streambox VCR, outlaw software that is able to download a stream to a local .rm file. Don’t miss Flying Raichu’s story_of_crack.txt where he splices code from an old beta into this version to make it work again.

Now to convert the RealMedia bits to MP3. A search for rm mp3 convert is heavily spammed. The best option seems to be to play the audio through RealPlayer and capture the waveform at the audio driver level. There are zillions of Windows programs that capture audio. I settled on Total Recorder because it was the most popular on KaZaA. (No, I didn’t steal it - $12). It creates a fake sound driver in front of the real one to capture the sound. It also does a cool thing where it tricks Real into playing faster than realtime so the conversion doesn’t take so long.

Total Recorder captures the waveform and can push it through LAME to get MP3. But you can’t easily download a LAME binary, you have to go to France to find it.

The resulting MP3 is too quiet; MP3Gain fixes it.

This is a lot of work to tape a song off the radio. DRM doesn’t stop you if you’re persistent, but it’s enough to stop most people. And just like the way Macrovision prevents you from plugging your DVD in through your VCR, the restrictions on Internet audio and video prevent you from replaying them the way you want.

With Palladium what I did will be impossible. I won’t be able to install Streambox or Total Recorder on my own computer, or if I do I will be locked out of using RealMedia.

I have a big smile on my face now remembering the song “Warm Leatherette”.

I finally figured out how to get Vice City to work with Alcohol 120% - have to rip with DPM turned on to defeat SecuROM 4.8.

Why do I have to have a $50 program just to use the $50 game I bought? At this rate I’m tempted to steal the games off of a P2P network: at least those versions don’t have copy protection.

I believe that Microsoft’s Palladium is going to make all this a moot point as we lose control over the very software running on our computers. We are doomed to DRM hell.

The only real fix is his Rule #8: Trust in the server is everything in a client-server game. He left out the corollary: only build client/server games and make sure the server is a trusted computer.

Even then it’s not going to be perfect, as noted here with the cheats in Diablo II. Unless the server is perfect the hackers will find server bugs to make bad things happen. The whole problem is fascinating. It reduces to the same problem of distributed systems consistency in the face of Byzantine failure. Or copy protection, or DRM. This is what Microsoft’s Palladium is all about.

The game works pretty well so far, much better than the mess that was GTA3 on the PC. Some annoyances: it installed DirectX 9 without asking my permission, there’s a silly off-by-one display glitch in the cutframe letterboxing, and I can’t get Alcohol 120% to emulate their crappy copy protection. But the framerates are good and the graphics are beautiful. Even the intro is cool, retro C64 load screen.

Lots of laugh out loud moments. My favourite so far, the little putt-putt moped called the “Faggio”.

There’s CP1252, the almost-but-not-quite ISO-Latin-1 that is responsible for the evil breakage of “smart quotes” by encouraging web publishers to act like 0x93 is a valid way to represent a left double quote. At least it encodes É in a sensible place, 0xc9.

But why stop at one codepage? There’s also CP437, an ancient DOS codepage that is nothing like Latin-1 but contains Latin-1 characters like É at 0x90. Yes, that’s a different place than CP1252.

Apparently both of these evil 19th century codepages are still coexisting on my 21st century Windows XP system. I just dumped a bunch of MP3 files from my WinXP box to Linux and found the filenames hopelessly garbled. I finally guessed they’re in CP437. I’m a bit surprised Samba didn’t take care of it for me.

Python to the rescue:

Online games have always been victim to hacks, made even worse by the real world market on eBay for characters, items, and cheats. Blizzard was clever and went to a server based model to prevent hacks. Apparently it’s not good enough, because the game is awash with superpowerful obviously hacked items.

I found a good article analyzing the motivation and sources for these cheats. The motive is tens of thousands of dollars (real dollars). The source? Harder to say, because software and virtual property gets passed from hand to hand before it hits eBay. The author of the article above fingers a student at Tsinghua University.

From Beijing to the Bay Area, mediated by a virtual world.

The marketing strategy with Animatrix is great. Build excitement for The Matrix: Reloaded by giving away related content, then sell the giveaway too. You can preorder it on Amazon now, but it doesn’t ship until June 3.

May there be mercy on man and machine for their sins.

Bugs like this are incredibly common, usually not worth reporting. But Passport is different. Passport wants to be the single trusted repository of personal data, all your eggs in one basket. I worry they don’t have a fundamental systems security model to make that safe.

This isn’t the first time Passport has been hacked, either.

The protocol design is very clever. It’s as low level as it gets. The hardware device accepts commands like “write this MP3 data into memory” or “stick this byte on the I2C bus”. All the hard work is done in the server.

That means functionality can be added without hacking the device. Streaming Internet audio, Ogg Vorbis files? Have the server do the hard work and send MP3 to the device. Menus work by sending IR codes to the server, so you can add new functions. Some nut has added a calculator.

The server may be the barrier to the SLIMP3 becoming common. Anyone into MP3s has a PC in their house and may even have a network. But do they have a stable server machine? It’d suck if mom’s relaxing music in the parlour is interrupted because junior is playing SimCity.

I live in a quiet suburban neighbourhood of San Francisco, not the kind of place where you’d expect this kind of petty crime. And I always lock my car door. Did I forget?

I love movies like this, they remind me of my melodramatic Texas heritage. Baby Jane was difficult to watch again. 134 minutes is too long and the performances aren’t as good as I remembered. The most surprising thing on second viewing was the strong role of Elvira, the loyal maid, played by Maidie Norman. Complicated class and race relations.

The most astonishing thing for me about Baby Jane is that Bette Davis and Joan Crawford would agree to make the film. They are both so sad and Bette Davis is so hideously ugly. It comes across now as camp rather than horror.

I hadn’t realized there was a series of older actress exploitation films.

I was really hoping to find a wireless MP3 component. There are a couple, but none seem as good as the SLIMP3 with a wireless/ethernet bridge. The cd3o has a voice interface that sounds like a terrible idea. The Exstreamer sounds cool but doesn’t have any interface at all. Gloolabs’ Homepod is still vapourware.

What really sold me is the SLIMP3 is hacker friendly. You have to love a company that makes developer resources available via SourceForge. They remind me of Empeg.

I imagine SLIMP3 will be acquired by someone. I wonder if they can remain hacker-friendly while producing a product simple enough for the average consumer?

Foveon has a sensor technology called X3 that works around this problem by stacking several translucent sensors in a single pixel. In theory this should give sharper pictures. So far the Sigma SD9 is the only camera out there with the Foveon sensor ($1500). Alas, the reviews suggest this first camera isn’t quite up to the sensor.

We’ve killed around 2500 Iraqi civilians. That’s roughly World Trade Center numbers. No one has an estimate of how many soldiers we killed. Given that we’ve liberated Iraq, shouldn’t we care about the dead soldiers? They were victims as well.

The US is deeply into the arrogance of power. It’s wonderful that only 132 Americans have died in this war. But it is dangerous that our power is so disproportionate. We lash out across the globe without feeling consequence. The rest of the world is horrified at our violence, and in the US we are ignorant.

But the UI in the firmware is crummy. Enter RockBox, an open source firmware that was built by hacking the Archos. Works great: better UI, better music buffering for skip protection and battery life, and I think it even sounds better. iPod owners - it may be great design, but you’ll never be able to hack it.

Hardware companies should encourage this kind of thing.

And she’s going to move out of her boyfriend’s house at 5am? The boyfriend who lost his legs because of her? Cold.

The technology problem is fascinating. The right solution is to fix DNS to use UTF-8. But no one thinks we can update the zillions of programs that assume DNS is ASCII. So the recommendation is to encode Unicode in ASCII via Punycode. Slap xn-- in front of the encoded name and poof you have an IDN. crèmebrûlée.com becomes xn--crmebrle-20ap0r.com. Try yourself!

How are users going to see these names? The plan is to augment web browsers, email clients, etc. with software to handle Punycode. Internet Explorer is already halfway there, thanks to Microsoft’s interception of broken URLs. Verisign released a preliminary browser helper that augments IE for IDNs (with a now-obsolete encoding). Expect many more to follow. I’m not convinced it will work.

The political problem is also fascinating. Domain registrars stand to make a fortune registering new names. They’re impatient. Verisign has been selling IDNs for a year with the now-obsolete RACE encoding; there’s a debate whether to grandfather them in. Verisign even threatened to violate the DNS standard. And a lot of people still disagree with the Punycode approach. It essentially dooms non-American languages to second class status for a very long time. But it might be the expedient way to support IDNs. We’ll see.

I wrote my own tool to read the EXIF data in the image and rename and timestamp files appropriately. What a pain in the neck. EXIF isn’t a good standard; I decided exiftags does the best job of reading EXIF files, including Canon’s extensions. Converting timezones is a pain.

I was going to autorotate the images but as I note below the software that claims to do this losslessly doesn’t.

Maybe I’m obsessive? Well, now I have a Python library.

Only it doesn’t work, at least according to the Gimp. Above is the difference between the original image and the one passed through jpegtran. I’ve boosted the gain a lot - the difference histogram is the image on the upper right. The differences had pixels as much as 17/255 white! It should be all black.

Is this a flaw inherent in jpegtran? Is it an artifact of the Gimp’s jpeg decoder? It’s not just jpegtran; Canon’s own software has the same artifacts. And I don’t think it’s just the Gimp; I get the same results with djpeg and pbmplus.

I couldn’t tell if this is a spoof or not. The sybase-ads.com domain is registered to Evolution Bureau, a high-concept advertising firm that specializes in websites and software to go along with ad campaigns. All of the examples there are similarly on the edge between creepy and corporate, but there’s too much work there to be someone just playing around. The clincher is the page on sybase.com itself.

High farce that it’s not clear if “Compliance or Consequences” is a parody or the real thing.

I haven’t found a good review site, but looking around I did find PrintRoom.com. Their prices are better than ofoto, you can tell them not to do any enhancements, they have a FAQ for pro photographers, and they even provide an ICC profile if you want to colour match yourself. Worth a shot.

Unfortunately WinGimp doesn’t have any ICC support. Raph has a good page explaining why: it’s hard and there are patents in the way.

What surprises me is how steady the email stream is. There’s a gap for a vacations I took summer of 1996 and spring 2001. And a drop-off after I started working at Google in 2002 (I’m not indexing my Google email). But basically my email sent has been roughly the same for the last ten years. Weird.

Björk did a fantastic job, being both very Björky and Norma Rae. She’s like the anti-Madonna; equally creative and aggressive, but with talent.

If you like von Trier and the Dogme 95 thing make sure to see The Celebration (Festen) by Thomas Vinterburg.

I’m having a hard time finding a full human-readable list of what it does, but if you have the source I think grep 'tflags.*net' rules/* gives you a dump. It at least does a bunch of DNS lookups and checks against Vipul’s Razor, DCC, and Pyzor.

I don’t think SpamAssassin is evil. I am surprised that it does network checks by default. There’s the obvious privacy issue. And network overhead can be high, particularly with 60 second timeouts. It seems like spamassassin installations are vulnerable to denial of service; if an attacker can cause all spamassassin installations to wait 60 seconds to classify every email, it could cause chaos in mail delivery.

But network-based checks can be really useful. The DNS heuristics look great and the collaborative spam databases are a real solution to the spam problem. There’s an obvious commercial opportunity here.

For now I’m leaving network checks on. If you want, you can turn network checks off with the flag --local.

GipsySoft also makes Winspector, a handy replacement for Spy++ for digging into how Windows apps work.

What I like best is how easy this was. I spent weeks building Funes, a Java mail search program that never was useful. With Python and MySQL it took me just a few hours and the result is better! Goodbye, grepmail.

I’m not the only MySQL fulltext enthusiast: Jeremy Zawodny’s blog has a great entry with comments and Mitchell Harper has a useful introduction article. There’s also some performance discussion on a PHP forum.

One trick - for speed, run myisamchk -a on the table after building the full text index. And do your big load before creating the index; afterwards, inserts are slow.

• clicktrack: Clickthrough tracking
• imagesizer: Size tags for images
• autocorrect: Move posts without breaking links
• categories: Breadcrumbs at bottom of page
• foreshortened: Truncate posts in RSS feed
• asin: Macro for Amazon links
• file: Include file data; my QuickTopic, Referers, etc.
• rimg: Macro for my right-side images
• colour: Cute colour hacks

The image at right is one of my favourite cheeses, Epoisses de Bourgogne. It can be.. difficult when properly aged.

Most posts are before summer 1994, when I graduated from college. Usenet was a hugely important thing for me, both a technical touchpoint as I was learning Unix and the Internet and a social touchpoint as I was coming out. Now mailing lists, the Web, and blogs have replaced Usenet as the primary online social medium. I miss the exclusive club we used to have.

I learned about ISO 12233 on Imaging Resource, as part of their fanatically detailed page about camera testing. The review of the S400 has the 2272x1704 WG18 ISO 12233 image if you need. Also, the viewfinder test confirms I’m not crazy - the optical viewfinder really is way off center.

Waldo is one of the authors on the best paper I’ve ever read on distributed system design, A Short Note on Distributed Computing. Essential reading.

This spambounce storm reminds me of when I was sendsys bombed back in 1993. Back then, a couple thousand unwelcome emails was a real problem. Now it’s a typical week.

• Rip CDs and DVDs
• Burn CDs and DVDs
• Emulate a DVD drive

Got some SafeDisc bad blocks on your CD? No problem. Got some funky SecureROM 4.8 spiral tracks? No problem. Rip the disc to your hard drive, then either mount it with their emulator or burn it to CD-R.

Mostly I use Alcohol 120% to rip the games I buy so I don’t have to have the physical disc in the drive to play. No more downloading scary cracks. I also rip all discs before I ever mount them; the virtual device is much faster than the physical.

Obviously Alcohol 120% enables piracy. I haven’t seen their MDS files (ISOs with the out-of-spec data) turn up on file sharing networks, but that can’t be far away. But I hope they’re not tarred with the piracy brush; the software is great for legitimate use, too. Fair use has a posse.

Daemon Tools is a free hackerware predecessor. Alcohol 120% is a polished commercial product.

Wired, which in the past has breathlessly promoted the Media Lab, has a sober article titled The Lab that Fell to Earth, an update on the Media Lab that’s too long but well researched.

I was a PhD student at the MIT Media Lab 1996-1999. I have mixed feeling about my experience there. I learned a lot but ultimately got so frustrated I bailed on the PhD and have never regretted it. The master’s program was great, but the place lacks the academic foundation to be a good PhD program. And some of the stuff that went on there vis-a-vis students seemed borderline unethical.

Still, it was a fun place and I’m thankful for the opportunity to have done so much fun work there. And I’m proud to see friends continue doing great work there, like Cameron’s Blogdex or Raffi’s embedded networking work. The place was special, maybe it won’t be forever.

Property is Freedom
Property is Theft

PS: Marc? Phblllt.

I’ve had a lot of fun with the macro setting. Cliche flower photos, but they’re fun to do.

I continue to be disturbed by being entertained by depravity. “Psycho with a sniper rifle threatens people on a street in Manhattan” - bring on the popcorn!

I keep coming back to something from The Republic, section 377b:

Al Jazeera has been struggling to stay online in the face of persistent attacks from hackers intent on silencing them. Now Akamai abandons their customer right when they are most needed. Craven.

Old fans of Microsoft Flight Simulator know it as the startup airport. GA pilots know it as one of the nicest urban GA airports around. The mayor has been trying to shut the airport for awhile but wasn’t able to build the necessary consensus. Now he’s got what he wants by sneaking in at 11pm and unilaterally destroying the airport. Under the guise of “homeland security”. Lovely.

I just got back from a small road trip through northern California and was surprise to see how much anti-war sentiment was visible. The first photo is around noon on Saturday in Petaluma, CA (pop: 55000). The second is around 1pm on Sunday in Fort Bragg, CA (pop: 7000). In both cases you have protestors standing out in the hot sun in a small town, opposing the war.

Granted, Northern California is fairly liberal. But not too liberal - these are pretty small towns. About three minutes after I took the photo in Fort Bragg someone yelled “queer” at me from their pickup (I’ve still got it!).

I did see some pro-war or pro-Bush signs, too. Overall I was just surprised to see so much political sentiment out there in small town California. People think!

The feature that surprises me the most for being useful is the TV playback mode. Very comfortable looking at photos on a TV screen, and handy in a hotel room.

Trying to find an honest online camera store that will ship quickly and reliably is unnecessarily difficult. I ended up at B&H for rapid shipping. Amazon has it for $480 via J&R.

But the game makes me so damned tense! I find that’s true of many good video games. Hacking Python is more relaxing.

According to the paper Thousands of people roaming the streets in an organized/chaotic way, hundreds of cops doing battle to control the situation. In Portland in the early 90s when this kind of thing happened the cops went apeshit and started bashing heads. In SF it sounds like the police just calmly did their jobs. Could have been a lot worse.

12 years ago, that was me. It’s not me now and I’m of mixed emotions. Blocking traffic isn’t going to stop the war. But doing nothing encourages complacency. It is wrong that the US is off killing thousands of Iraqis and our biggest concern is which freeway offramp might temporarily be shut down.

Gillmor’s own concerns are expressed much more clearly than mine:

Watching the video you have no real sense that there are actual people dying down below. Just little bright blobs on a dark backbground. The soldiers are remarkable, too. So many people involved, so calm! War machine.

Listen to the voices at the end:

Thanks to Diffuse Shadows for the link and Obey the Fist for the video. I think it’s interesting how most blogs picking this up are pro-war.

Isn’t this what computers are for? Can’t the code remove the punctuation for me? And why do I have to tell them whether the card is a Visa, Mastercard, or Amex? The first digits on the card make that clear.

It’s like the site wants to make it hard for me to buy something. Is this some sort of Visa requirement, to only accept the number exactly as typed by the user?

The other day I was picking up my shirts at the Freedom laundry when I met a handsome man with his hair in a long Freedom braid. I invited him back to my house for some Freedom wine and before I knew it we were Freedom kissing. Turns out he’s into Freedom active; we had a lovely time and then he left, promising to write me Freedom letters. I sure hope I didn’t end up getting the Freedom disease. That’d be doubleplusungood.

These Tom Clancy games are creepy military porn. Fans love them for their realistic weapons, tactical planning, corpses, etc. I confess to an uneasy enjoyment of this kind of fantasy. Hitman 2 was awesome.

What surprises me is how much of the fan activity for Raven Shield is European. I would have expected something this militaristic would be most popular in the US. Maybe the realism of it turns off American Rambos.

PS: Amazon has a $10 rebate.

The review is a bit goofy but does have one insight I particularly liked:

gdchart is good software; simple code to draw graphs and save them as GIF or PNG files. The Python interface makes it really easy to do some serious charting.

In the meantime, the US has produced no compelling evidence of the alleged weapons of mass destruction in Iraq. Let’s ask another expert, Hans Blix, lead UN weapons inspector in Iraq:

I guess I was naïve to think that my government wouldn’t just lie about the evidence we claim to have.

A pox on DRM software, btw - I couldn’t get a decent screencapture of the video or even a link to the actual video. Grr.

The crazy thing about watches is you can easily spend $20,000 on one without getting gold or jewels. Just a lovely skeleton movement or a beautiful A. Lange & Söhne 1. The workmanship on the inside is as beautiful as the exterior. Alas, not in my range, so I just read along with the watch geeks.

I don’t want to be that smart to write simple scripts. I started learning Python a month or two ago and I’m a much happier person.

No fewer than 71 different hosts grabbed my robots.txt in eight days. Two creepy ones: NameProtect (trademark enforcement) and TurnItIn (anti-plagiarism). This doesn’t bug me too much. I do wonder why some of these crawlers felt it necessary to fetch robots.txt forty times in eight days.

The comic is fun - it neatly captures the wonder and mystery of Lovecraft’s epic fables. I like the everyman aspect of Mock Man, a nice rendering of Lovecraft’s own generic men. The thing that I think is coolest is that in the comic book is a URL and email address from 1997, and they still work! The Bits out of Time.

I was prompted to write this by Kottke’s post about a change Amazon made to URL formats. Used to be you could go to Amazon, copy the URL for a page, paste it into your blog and slap on your associate ID and you’d get the credit. No more, now you have to construct the URLs very carefully. This plugin makes it simpler.

I hope I got the URL format right. I’m sticking in the undocumented ref=nosim - I think it’s obnoxious that Amazon defaults my links to ‘buy more crap’.

There’s also a new version of the imagesizer plugin to automatically add size tags to your story’s images. Thanks to Todd for the Perl-fu to do this more cleanly.

Finally, I now have a place for all my Blosxom plugins. Apache stylin!

The game is very complex. The problem is it lacks any tools to help you understand what’s going on in the simulation. My cities get stuck at some low population level and I can’t find out why. It’s frustrating, not fun.

The documentation is awful; you have to buy the $20 strategy guide to even make the game playable. Even the folks who play obsessively can’t figure out how to get all the big buildings.

The recent much-lampooned Homeland Security graphics echo these designs and are beautiful in their own right. The images are more personalized, with clothes and hair. And yet they seem even more inhuman. And while they attempt to depict more dynamism, with arrows and cartoon vignettes, they seem more static than the Aicher designs.

I wouldn’t mind the occasional lost packet but this kind of network lossage is playing hell with some connection-oriented Windows applications we use. Very annoying to have your terminal session terminated. And web page loads fail a surprising amount.

Is this typical?

Ken noticed that all the outages are either 6 seconds or 11 seconds. Is that some sort of connection reestablish timeout?

His comments are insightful, but what he casts as problems with blogging are exactly why I like blogs. My blog is my own small world. It’s my one place to indulge the desire to talk only about what interests me. He finds the use of dates in blogs distracting. I like it; emphasizing the ephemeral nature of blogging frees me from the burden of writing for the ages.

Mostly I’m blogging this to highlight the painful irony of blogging about a non-blogger who doesn’t like blogs but wrote a blog entry to say so. I had a hard time wanting to read his whole page, too. I do like his found art.

Perl gives me a rash.

Well, sadness tinged with greed, for it seems if I just cooperate in some bank matters with my new friend JAMES EDWARD then I stand to gain access to a lot of money! I’m a bit confused as to why a long lost Czech relative would be in Mauritius, but it sounds safer than Nauru.

Back in the 90s Internet fantasy era I read a bunch of stuff about how the Internet would allow the third world to be entrepreneurs on equal footing. I find it sort of charming that folks in Nigeria and Mauritius are running the Spanish Prisoner. The best story on this stuff is Wendy Willcox, who manouevered a Nigerian spammer to appear in front of an Amsterdam webcam.

DoSomething only works with ID3v1 tags, not ID3v2. And the Einstein who defined ID3v1 decided that 31 characters was plenty for the title of every song written. So my CGI sees truncated titles.

I spent 20 minutes trying to find another Winamp plugin that would do HTTP messages with ID3v2 tags. No. The closest I found was What’s Playing, which looks nice but requires WinAmp 3 and doesn’t do HTTP.

I’m wasting my time.

The underlying problem is an impedence mismatch between new Unicode oriented tech like the Web and XML and old ASCII oriented tech like email and text editors. Browsers and RSS readers should mediate between the two but software often gets it wrong. For instance on a cut and paste MSIE translates U+201C into the byte 0x93, which I guess means something in some proprietary Microsoft encoding but is not valid ASCII. If it just turned it into a proper ASCII 0x22 quote life would be so much better. So because of buggy software we’re left with poor typography.

This may all seem a bit pedantic for a design nicety, but handling non-ASCII correctly really matters if you write in almost any language other than English. Unicode and UTF-8 really are the right things.

The ironic thing is that in this blog’s preferred font (Arial 12pt) smart quotes don’t really look any better than normal quotes. So for now I can ignore the whole issue.

In 1996 I wrote a P2P AI named Floyd. He lived in ccr, a research virtual world built by David Ackley. It deeply shaped my thinking about distributed systems. Floyd was the first P2P system I ever built.

Trillian is fantastic software. Clean and simple instant messaging, no ads or annoying crap. And it interoperates cleanly with all the major IM networks. The free version 0.74 skin stuff is silly, but Trillian Pro has a good no-nonsense default look. Lots of usability points are high: it imports your contacts from other IM networks with no fuss, autopatches nicely, etc. Excellent piece of software.

Usability on mod_gzip is fairly low. Original site is offline, docs are awful. Fortunately someone has taken on the task of making a decent support site. Even then the details of how it works are magic and opaque; honestly, this kind of server configuration should be much easier or automatic. Maybe it is in Apache 2.0.

Here’s the magic I’m using:

Blosxom has a built-in RSS mode already; the difference here is you have control over it via flavours. I took the opportunity to make the output smaller and to remove the <description> content. See for yourself.

You may remember recently sending an email to a Spam Arrest customer, and receiving a response asking you to visit our website and type in a word that was shown to you in a picture.
...
We are so confident you’ll like our product, that we’d like to offer you a 30-Day free trial.

Then I go to secure all the machines inside my firewall, so in case someone breaks the WEP it’s not a total disaster. Did you know that Windows XP enables “Simple File Sharing” by default? That’s the kind without passwords. You can turn it off by opening the “Folder View” option - the one that usually controls the visual layout of folders. And then, once you’ve got “Complex File Sharing” on (WinXP Pro only, no WinXP Home), you have to go to every share and set permissions. Only the permission setting interface is confusing, and there’s no easy way to configure permissions from the list of folders you’re sharing. So finally I decide to turn off file sharing entirely. How do you do this? By going to the Network Connection for the LAN, and selecting “remove Windows file sharing”. Seems sort of intuitive, only I still don’t understand - does that just turn off my file sharing server, or does it disable the client too?

What a mess. We’ve got a long way to go on the usability of security.