San Francisco has only one social issue that matters: homelessness. Our streets are filled with people in wretched conditions: mentally ill, addicted, just plain fucked up. We’re a city that thinks it’s compassionate to give a junkie $410 a month. We’re a city that’s incapable of offering real help.

The SF Chronicle is running a five part series on homelessness this week, presumably timed to go with the mayoral election. The first article Shame of the City: Homeless Island is heartbreaking excellent journalism. So are the photos.

One thing I do to save bandwidth on my bog is make tiny images. But what’s important is number of packets, not bytes. I just measured in a packet sniffer. For an MTU of 1500 bytes the first HTTP response packet contains roughly 1130 bytes of image data. Every other packet contains 1460 bytes.

Being conservative, optimal sizes for small images are 1100, 2560, 4020, and 5480. 3000 is no better than 4020.

Since learning Python ten months ago I’ve been a much happier hacker. I’ll never go back to Perl again, and I’m increasingly frustrated working with Java. Here’s some of what I’ve written in ten months:

• access.log reporting tool
• Converter to translate blog entries into Blosxom format
• Data scrapers for various websites
• Pinger for weblogs.com and blo.gs
• Email search engine (using MySQL)
• Parsers for various game datafiles
• CGI to record what music I’m listening to
• CGI for capturing data for my linkblog
• Photo management tool using EXIF tags
• MP3 management tool
• Cell phone data management tool
• Emulator ROM management tool
• Various BitTorrent hacks
• Windows systray app for my surround sound system

Next project: wine cellar software

After making a fuss for heritage turkeys, I have to confess my Midget White from Townline Farm was good but not great.

The most distinctive thing about our turkey compared to a freakish grocery store turkey was that the dark meat was dense and really, really dark. Much more like a game bird. The drumsticks tasted great but were very chewy. That may be our cooking fault.. The breast meat came out juicy and very flavourful, much better than the usual supermarket mediocrity. There was also a surprising number of tendons in the meat, not to mention a lot of fat in the skin. Good gravy! I’ll probably try a heritage turkey again, but it needs some practice. See also Slate on various turkeys.

Anyway, Thanksgiving dinner is more about side dishes and wine than turkey. The corn pudding with chanterelles and cranberry jalapeño relish were great, and the 1978 Santenay was elegant if a bit light to stand up to dinner.

Somehow, Macy’s survived.

Color scheme is a beautiful colour picker. Match a base colour to others with five different colour match types (contrast, analogous colours, ...). View your scheme in a simulation of various forms of colourblindness. Even has the ‘web safe’ palette of yore.

The tool design is great, very simple and clean (other than the color swatch display; more functional than æsthetic). I love the implementation: a single ‘live’ web page running a bunch of complex javascript.

The excitement of fanboy gamers is a double-edged sword; when disappointed they can be the harshest critics. Deus Ex is one of the best PC games ever, cleverly fusing FPS and RPG elements into an open ended game with a great story. There’s a lot of anticipation for Deus Ex 2, coming out next week (already gold).

Alas, the demo of DX:IW disappoints. Fans are up in arms that the game is dumbed down for consoles, lacks the subtlety that made the first game great. I found the demo unplayably laggy until Warren Spector posted some fixes. Even then it only seemed OK; nice visuals, boring gameplay, stupid UI. Kind of a shame: instead of building excitement the demo seems to have made me worried about whether this game is up to snuff.

Don’t miss today’s NYT editorial about US turkey production by Slow Food director Patrick Martins. It explains why your Thanksgiving turkey has no flavour. The accompanying graphic is great, too. The article spends too much time on the horrible conditions the turkeys are raised in. That’s a shame, but what I really care about is the destruction of flavour and individuality in our food. I did something about it: my direct-from-farmer Midget White should be here tomorrow.

My TiVo died: playback stuttering, menus slow, etc. Apparently this is common for TiVos of a certain age (mine is three years). The drives fail.

Thanks to Weaknees I was able to repair it in about half an hour, replacing the drive with a newer (4x bigger) one. I could have done it myself with the info in TiVo Hacks, but the extra $40 Weaknees charged me is worth the half day of aggravation saved.

The experience has bummed me out a bit about TiVo:

• Why did my TiVo fail after only three years?
• The setup process takes eight hours!
• I’ve lost all my TiVo state. Given how much data they track about their users, why couldn’t they have stored my season passes server-side?
• The CPU is too slow. My TV should never tell me “please wait”.
• My lifetime service isn’t migratable, so I won’t upgrade.

I really should build a small form factor PC and install MythTV, an MP3 player, a DVD player, and a bunch of game emulators. I like the way the TiVo is a simple appliance that just works, but I’ve hit the edges of that.

I watched The Matrix again tonight, partly because my TiVo is sick and partly to erase memory of the horrible sequels out of my mind. Now I understand what made the first film so magic: the interplay between real and simulation. The film toys with us, showing us little glimpses of the synthetic. Deja vu, the reality of bullets, the ability to do kung fu. And the lovely Cronenbergesque bits, the baby with the hoses, the bug, the sockets in flesh. The whole film is a discovery of unreality.

The next two films strand us, in the words of the script (and Baudrillard), in the desert of the real. Ugly boring world with stupid Zion hippies. And Neo is already brilliant and all powerful, so there’s no discovery. No pleasure.

This is reflected in the special effects technology. The first Matrix film is beautiful because it is, ultimately, a film. The FX are mostly film tricks (slowing, rotating) and the stunts are mostly human stunts, people on wires. The reality-based effects work highlights the gap between the Real World and The Matrix. By contrast the digital effects work of the second two films is just unreal. And boring.

There was no way to make a good sequel.

Massachusetts has temporarily allowed gay marriage (it won’t last), and already the Democratic candidates are panicking. Or as the NYT article says: This prevarication is what is so awful about America’s mediocre two party system. The Democratic candidates are all too craven to actually take a leadership stance on social issues. Bush isn’t afraid to be hateful: Why can’t a Democratic candidate be equally forceful, but in a humane way? One thing I admire about Dean is his willingness to actually take strong positions. But even he shies away from gay ‘marriage’. At least he favours gay partnerships with pretty much all the legal protections of marriage, and even enacted such in Vermont.

AOL Instant Messenger has a new feature: message routing. You can now log in AIM from more than one place and get multiple copies of messages.

I’ve used a slimp3 in my house for awhile now. Simple MP3 network appliance: small screen and remote control streams MP3 from your server. Slim Devices has now released a wireless version, the Squeezebox.

The biggest change is built-in 802.11b; no more need for an wireless/ethernet bridge. They also put in a digital out and wrapped it in a more conventional case. Looks like nice improvements.

After learning about Ericsson AT commands yesterday I wrote up some Python to program my phone’s contact list. It probably would have been faster to set up Outlook, but where’s the sport in that? You can download the unsupported, disabled Python T616 code to check out.

The real challenge here proved to be doing serial I/O. The excellent PySerial helped. But I’d forgotten how hard it is to write code to read data from a device that’s slow, particularly when you don’t know how much you’re reading. I should have tried out Twisted, like Matt Biddulph did, but asynchronous programming seems like overkill for a small project.

The Sony Ericsson T616 has a Bluetooth adapter that acts like a serial port. If you put the phone in serial mode, it understands a wide range of AT commands. Some useful references: R320s_WP_R1A.pdf, 888_r1d.pdf, AT Test commands, Google search for [CPBR ericsson].

I hope to use this to modify my contact list

I bet floAt’s Mobile Agent uses this protocol. Other folks have hacked their phones to be remote controls (Python, Perl). Phonefront is a commercial control app.

Similar to BulletML games like rRootage, Warning Forever is a pretty little shooter game. The interesting thing here is the articulation of the autogenerated bossed. Funky graphics, fresh and simple.

The site’s in Japanese. Poetic automatic translation:

I geeked out today with a Kill-A-Watt, a simple meter that lets you measure electricity usage of things plugged into ordinary sockets. I have two computers at home - a Linux box with an Athlon at 1000MHz and a Windows box with an Athlon XP at 1733MHz. How much power do they consume? Hours of fun!

System	Watts	$/month
Network	37	4
Monitor	90	9
Windows box	185	19
Linux box	47	7
Typical	300	30

My dollar calculations are off; I don’t know my real rate. What’s interesting is the incremental costs. Running CPU jobs at full-tilt takes another 30W on my Windows box, or about $3/month. AMD’s power management bug blows. athcool on my Linux box is saving me about 40W. If I could run VCool on my Windows box it’d save another 30W.

Bottom line - configure power saving on your monitor! And turning off your computer really makes a difference.

I had some of my Epoisses last night. It was perfectly fine and yummy, but it lacked the complex funkiness that makes the Epoisses I’ve had in Burgundy so challenging and delicious. Maybe it’s the aging or the cheesemaker, but I’m inclined to blame the pasteurized milk. Clearly I have to go back to France and do some comparison tasting.

I bought a new cell phone and upgraded my AT&T contract. I’ve liked their service and the deal was good, so why not?

Because their customer service is fubar, that’s why not. For the past two weeks AT&T has not been able to help GSM customers do things like, say, activate new service. Why? They just deployed a new CRM system and it doesn’t work.

I’ve been waiting for a week to use my new phone. I’m sure glad they have a 30 day refund policy.

I saw two young men in suits wearing hats in SF’s Financial District today. Wouldn’t it be fantastic if hats came back in style?

Men look great in hats.

The Matrix: Revolutions had a complex worldwide simultaneous release, ostensibly to fight piracy. Piracy happened anyway; 24 hours after the release, copies showed up on the net for download. Badly compressed movies of shaky camcorder copies. Pay the $9, folks.

Using the same /scrape URL that torrentspy uses, I tracked the BitTorrent activity of a 1.2G copy of the movie over the last week (from the day after movie release).

The main story this data tells is that a lot of folks tried to download the movie: 8500 at the peak, roughly the most people I’ve ever seen hit a single BitTorrent. The gaps are when the tracker was down; I’m surprised that so many people stuck around when it came back up. I’m also surprised at how many people stuck around to serve the file after downloading it; a steady 20% or more after things settle down.

For what it’s worth, this graph pattern is common for all the popular torrent streams I’ve monitored. The big variance seems to be how long the tracker survives.

Clay Shirky’s piece trashing the semantic web has stirred up quite a storm. True believers in The Power of Semantic Markup are all upset, the cynics are taking their potshots. I’m one of the cynics. I first worked with RDF back in 1998 in my Hive project. We used it to, well, describe resources. It was a total nightmare of complex syntax obfuscating some very simple data. I also side with Cory about Metacrap.

The RDF folks have had at least 5 years to prove how great the semantic web is. Where are the successes? There’s a few random database applications like rpmfind. There’s FOAF. And there’s RSS 1.0, where you can do the same simple thing you do with RSS 0.91 or RSS 2.0 only with twice the markup.

Each of the individual applications using RDF I know of could have been done more easily with plain XML. What’s the payoff for using RDF? Where are the fantastic semantic inference applications? I admit being fairly ignorant of RDF, so educate me. Point me to a practical example where the use of high concept RDF stuff has made an application significantly better.

Many of the most interesting cheeses are made from raw milk. But unless it’s aged more than 60 days raw milk cheese cannot be imported into the US. The result is Americans can only eat many fine cheeses by being travellers or smugglers.

One of my favourite cheeses, Epoisses, is in a grey zone. You won’t find raw milk Epoisses in the US: the cheese is only aged 4-6 weeks. And only 10% of Epoisses is made from lait cru. Is it the best?

One of the finest Epoisses producers, Fromagerie Berthaut, no longer uses raw milk. Berthaut is credited with rescuing Epoisses production from extinction in the 1950s. According to The Art of Eating Berthaut started heating the milk in 1999 after pressure when a nearby unscrupulous cheese maker had a listeria outbreak.

Epoisses can be difficult, strong smelling and tasting. I’ve had a hugely rewarding Epoisses at La Côte Saint-Jacques and I’ve had Epoisses that was mild and dull. I’m hoping the difference is aging because if it’s the milk, you can’t get the good stuff in the US. I have a Berthaut Epoisses (from Say Cheese) relaxing in my cellar now.

N-Gage games may suck, but people are already warez0ring them. A group called Blizzard is distributing cracked games and an installer. I’m not at all interested in running them, I just think it’s fascinating how the hacker community does things so quickly. It’s been exactly 32 days since the N-Gage launched. A quick search found some technical notes on how it works.

A few months ago I figured out how to configure Samba to work with non-ASCII filenames. Samba 3.0 changed all this, the new magic incantation is I sure wish I could just use UTF-8. But it’s remarkably difficult to make a Linux environment happy with UTF-8, so I’m stuck with Latin-1. I don’t really know why cp850 (a bastardized Latin-1) is the right thing on the Windows end, is that the default for US WinXP systems?

American Express sells itself on its customer service, but it has a lot to learn.

I agreed to pay a significant yearly fee for an AmEx card because it is supposed to have good benefits and customer service. My first experience? I call to activate the card and am connected to the worst telemarketer I’ve ever heard, struggling to read her script and sell me lots of crap I don’t want “while waiting for your data to load”.

In the mail with my card is the form I have to mail in to prevent them from sharing my private financial data with “valued partners”. When I try to register on the website I’m greeted with arbitrary restrictions. No “special characters” in my username. My password has to be 8 characters or less. Apparently email addresses can’t contain + in them, but I have to guess at that from the error “you have left the following fields blank or have provided an incorrect format”. After I give them an email address their stupid software will accept I’m told that I have to open “email preferences” to opt-out of the valuable offers they’ll send.

I can’t find the email preferences page ever again. I try hitting the back button to find it but am greeted with errors. The front page is full of offers to up-sell extra services. I finally find the opt-out page, only to be informed it takes them 2-3 weeks to update my preferences.

Stop selling me crap!

My home ADSL line seems to have gotten a speed upgrade; I get 256kbits/sec upstream now instead of 128. This is great - crappy upstream is my #1 complaint with SBC/Pacbell’s ADSL service. It looks like SBC upgraded all of San Francisco; SpeakEasy users got the news.

Two parts Highlander 2
One part Revelations

What’s the word for searching for an old acquaintance on the Internet and finding that the first result is his obituary? I spent time this morning reading digital ephemera; old email archives, archæology at the Wayback Machine. So much is gone, never archived.

White lily.

MSIE has a nice “Save as Web Page: complete” option. It saves not only the HTML but all the associated style sheets, images, etc giving you a fully standalone copy of the page. You can do this in Unix, too:

wget           # command line HTTP client
-q             # don't print out status
-p             # download related files
-k             # rewrite resources to local names
-e robots=off  # ignore robots.txt
http://URL/    # page to download

You may also want --user-agent='Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' The appropriateness of ignoring robots.txt is open to question; I think it’s OK for a personal one-time use.

It’s not perfect. The file may not be named .html, and some sites (Yahoo news) don’t download completely. But it’s pretty good. I’m using this to archive stuff I linkblog.

Little did I know so many others had linkblogs. I’m subscribed to a bunch of them with Trillian’s RSS plugin. Whenever someone posts a new link, it pops up like an instant message. It’s like surfin along.

I’ve added a new feature, my linkblog. It contains a bunch of links to stuff I accumulate. It’s over there on the left, in the sidebar. It’s also available as a very spartan blog of its own: useful for RSS. Here are the pieces:

linkblog.py

Python CGI that creates a bookmarklet and accepts input from it. Standalone tool.

Blosxom

The weblog engine for the linkblog

linknote

Blosxom plugin for my linkblog to read entries from the single file that linkblog.py writes out

linkblog

Blosxom plugin for my main blog that includes the output of the linknote-driven blog (see below)

include flavour

Blosxom flavour that prints out data with no Content-Type; the main blog calls this in linkblog

lastmodified plugin

Hacked up version of the lastmodified plugin

It’s handy for the linkblog to itself be Blosxom; that way you get RSS, plugins, archives, etc for free. This dynamic inclusion may not be worth the trouble; I could have made it a static blog and included it as a file, or I could have used an iframe. It works but it’s fragile. Particularly concerning the interaction with the lastmodified plugin.

Bob’s lastmodified plugin for Blosxom has a subtle bug. See the difference? Helps to have it lined up; there’s a space missing in the Last-Modified header.

Shouldn’t much matter, right? Wrong. Apache parses these headers from CGI scripts and tries to fix them up. It can’t parse the one without the space, so it silently replaces it with zero and all my pages look like they were last modified 1 Jan 1970. Argh!

I was so impressed with Maxim Stepin’s hq3x and related algorithms for scaling up sprite art that I ported his code to Linux and wrote a pnmhqxscale wrapper around it to use with the netpbm tools. The port is a hack, but it works OK. You can download it for yourself.

TorrentStorm is the latest good Windows BitTorrent client. It’s based on the experimental client that I liked, but with lots of extra features. Screenshots.

Good things it does:

• Displays seed, leech data like TorrentSpy
• Saves .torrent files
• Resumes downloads on startup
• Gives unsorted view of downloaded pieces
• Fast resume algorithm based on piece files
• Simple bandwidth configuration
• Excellent documentation

I’m disappointed there’s no source available. I think it’s funny there’s Python under there and no one can tell.