Ten years ago I wrote a pair of articles for O’Reilly on distributed systems topologies: Part 1 and Part 2. I wrote them because a lot of people were confused about what “peer to peer” meant in system design and so I tried to classify various options for how computers could talk to each other to make a single system. The article holds up pretty well, even if a lot of the analysis seems embarrassingly naïve. We don’t sweat the topological details so much anymore but I think there’s an echo of these themes in the discussion of how various NoSQL data stores manage data availability and consistency.

About once a year I get an email from someone who wants to start a company selling CPU time from idle computers, typically by running a screensaver or background job on public computers. Back in 2000 I co-founded Popular Power, a company to do just that, which failed about 15 months later. Here’s what I tell the people who write me looking for advice.

The primary challenge is finding customers. It is difficult to convince any company to risk sending their software and data out to untrusted third parties. There are no good technical measures for protecting intellectual property on untrusted computers. In addition, scavenged cycles are an awkward mode of computation and potential customers have to port their code before they can even try it out. A company’s primary value proposition is price, but you’re competing with the ever-decreasing cost of computer time. And the market is mostly $200,000+ enterprise sales. It always takes 12–18 months to close a contract at that scale, that’s a long time wait for a cash-poor startup.

A couple of things have changed in the past ten years for the better. Cloud computing is now mainstream, there’s no need to explain the concept. And distributed computing has improved with paradigms like MapReduce. Computers have gotten faster, although that doesn’t specifically favor scavenged computing. Unfortunately bandwidth has not gotten significantly cheaper so data transfer is still a barrier.

Fate has not been kind to this market in the last ten years: see my 2007 retrospective. Since then United Devices merged with Univa and is still operating, although presumably never worth anything like the $45M in venture investment it raised. Data Synapse got bought by TIBC for a rumored $28M, more than zero but not the $200m+ value of a real success.

I’m glad people keep trying to make this business work, but it’s a tough road. In the meantime free projects like BOINC are having some success providing low cost computation for public research projects. And of course the botnets are owning the Internet.

I had awful service from Virgin America recently. A flight cancellation, a software system disaster, and many mistakes from agents on the ground all worked together to make a truly awful travel day. I like what Virgin is trying to do, but until they get their act together after their disastrous switch to a new reservation systems I suggest avoiding them.

On Nov 2 Ken and I had Main Cabin Select tickets from JFK to SFO. That morning I called and paid extra for first class upgrades (score!). Then two hours before departure, the flight was cancelled. The scene at JFK was chaos because Virgin has just that week changed its online reservation system and it wasn’t working. The staff didn’t know how to use it, the system didn’t work, customers were sitting on the floor waiting for things to get straightened out.

Rebooking us was beyond the JFK staff’s ability to cope. The nice woman at the desk kept saying “we have this flight, do you want it?” and then spending ten minutes trying to book it, only to find some other agent had grabbed the seats while she was typing. She finally rebooked us on a different airline. Only they booked it wrong, twice, and I spent 90 stressful minutes going between terminals three times trying to get the flight booked correctly.

I will say the agent we talked to was pleasant and professional. She just was unequipped to make their own software work. And then they were so stressed and rushed they kept screwing up the booking on a different airline. Poor service is de rigeur for airlines but Virgin America claims to be something different. Sadly it wasn’t last month.

This is interesting: Google’s preview for my Wind History project shows they executed quite a lot of Javascript and SVG code. For Google to have rendered that, they had to load the page, asynchronously load 1MB of JSON data, then run roughly 1s worth of custom Javascript to generate and render a big SVG document. Doing this is no big deal if you’re willing to run a full web browser environment (say, a headless Chrome), but it’s still pretty sophisticated.

Related, a few weeks back Matt Cuts tweeted that Googlebot is executing some Javascript when indexing. (See this analysis and my earlier comments) It’s worth noting that the Googlebot indexer is almost certainly an entirely different system from the preview image generator. I suspect the index building crawl still runs very little Javascript code and almost certainly doesn’t render pixels; speed is important for freshness.

(A remaining puzzle: why the image tiles from Open Street Map aren’t in that preview. They’re also loaded in Javascript. They may have loaded too slowly, or maybe the preview generator doesn’t load content from external domains?)

There’s a big, ugly story that Zynga threatened to fire employees unless they gave some of their stock back (WSJ, cnet). The internal memo explaining things doesn’t really help. Zynga’s always been sleazy so if the story is true it’s no surprise, but there’s some nuances worth exploring. If you’re negotiating for a tech job argue for more stock, not more cash.

Tech companies give employees stock (or options) on hiring. The weird thing about that compensation is that the stock grant is generally fixed at the time of hire, before the company really knows anything about the new person. The resulting compensation is never really fair. Sometimes a great hire ends up not working out and ends up with more stock than they deserve. More often, an undervalued hire ends up doing great and it’s a shame he or she doesn’t have more stock. At Google my impression was most employees’ stock grants were entirely uncorrelated with their impact at the company; grant size had way more to do with hire date.

Companies can try to fix inequitable compensation by rewarding good employees with more stock. But those grants always come much later when the strike price is higher and are generally much smaller than the initial grant. So extra stock after being hired seldom really works out. It’s much easier to give a good employee a cash raise. The corollary to this is as a new hire, if you can afford to you should negotiate for more stock vs. more cash. Because while you can always get more cash later if you do well, you can seldom get more stock.

Another ugly aspect of the Zynga story is the use of “Google chef situation” as a metaphor for an employee getting too much stock. That refers specifically to Charlie Ayers, Google’s chef 1999–2006, who reportedly made a giant pile of money from his stock options. To anyone who suggests he doesn’t deserve that compensation: fuck off. Charlie worked incredibly hard at Google and did a great job growing a kitchen from one meal a day for 40 people to three squares for 10,000+ people. That kitchen had a huge impact on the success of the company; Google got an extra 200 hours’ work a year out of employees because we wanted to eat at work rather than go out. Charlie personally was responsible for the kitchen’s success. I assume “chef situation” is some sort of class distinction, that chefs don’t deserve as much compensation as the hallowed engineers. That’s disgusting.

The best meal we had in New York was at Eleven Madison Park. It was one of the most impressive meals I’ve had anywhere in the world and nicely complements one of the other finest meals I’ve ever had in my life, in 2004 at Campton Place. Same chef, Daniel Humm.

Eleven Madison is operating at a very high standard right now. If you live in New York or are going to visit make the effort to go: you need to reserve weeks in advance. They received four stars in the NYT two years ago and just recently got 3 stars and 5 forks in the NY Michelin guide. I had dinner there a year ago right after they switched to the current tasting menu and it was very good but not transcendent. In the past year they’ve found their stride and it is now as good as dining gets, anywhere in the world. Go.

Our meal consisted of some twelve amuse bouche followed by a four course tasting menu meal. The menu is deliberately abstract, just a list of main themes like “Lamb” or “Apple” that are elaborated based on the staff’s reading of the diner’s desires. An open mind is useful but they’re not playing any games, no need to fear some unwelcome surprises. The cooking leans towards traditional, not molecular gastronomy trickery, and excels by being very well executed.

Langoustine: Marinated with Grapes, Fennel, and Marcona Almonds
White Truffle: Tortelli with Fontina Val d’Aosta and Chestnuts
Guinea Fowl: Roasted with Pear, Salsify, and Foie Gras
Apple: Ice Cream with Toffee Cake, Walnuts, and Caramel

Foie Gras: Terrine with Plum, Bitter Almond, and Umeboshi
Lobster: Poached with White Truffles, Autumn Mushrooms, and Spinach
Guinea Fowl: Roasted with Pear, Salsify, and Foie Gras
Blue Cheese: With Pear, Chestnut Honey, and Bitter Greens

1995 Meursault Les Meix Chavoux (Domaine Roulot)
1996 Corton-Bressandes (Domaine P. Dubreuil-Fontaine Père & Fils)

Sadly I have no record of the various amuse, they were extraordinary in their variety and complexity. The main courses were nicely paced and undiluted, each enjoyable and complete without being overwhelming, Often at a fine meal like this by the time you get to the poultry you’re tired and just want to stop eating; Eleven Madison avoided fatigue.

But food is only half of fine dining, the room and service is the other half. And it’s extraordinary. Terrifically personal and friendly service, not too formal, but very professional. I’d gotten the reservation by gushing on about having been to Humm’s restaurant in San Francisco and was impressed the staff all knew my story. We even got a little visit to the kitchen, always fun, amazing to see as many cooks in the kitchen as there were diners in the room. All working precisely, neatly, for our pleasure.

Chef Humm is buying the restaurant from Danny Meyer, along with his General Manager Will Guidara. That’s probably good news, but I’m a bit concerned that they are also taking on food and beverage at a nearby hotel. The best chefs always expand their empires, it’s natural for his career, but I fear the risk of diluting his excellence. Did you know Wolfgang Puck used to do something other than frozen pizzas? They are also publishing a cookbook due Nov 11 (see video). We saw a pre-release copy. It looked beautiful and entirely impractical for the home cook, more of a monument to his art like Keller’s French Laundry Cookbook.

Time for the twice-annual public service announcement: Daylight Savings time is stupid. If you run Internet servers and/or collect logs be sure you do everything in UTC. Saves a lot of heartache.

Some bad software forces you to choose your time zone off a map, making it impossible to select UTC. A handy trick is to choose Reykjavik, Iceland time. Since 1968 Reykjavik hasn’t changed clocks for daylight savings and is identical to UTC. That could theoretically change in the future.

There’s an unpleasant social media company out there called Klout. It claims to measure your individual influence by analyzing your social networks on Twitter, Facebook, etc and then assigns a score of 1 – 100, your value as a social being. The basic metric itself is kind of interesting; what’s problematic is all the product around it.

The positioning of Klout is gross, turning ordinary social interactions into some sort of game with a competitive score. Of course there are meaningless achievements. But what’s most loathsome is the primary purpose of Klout is to offer “Perks” (aka “ads”) to you based on your Klout score. Yes, I can get a free sample of Axe Hold and Touch hair gel because I’m such a bro on the Internets! Even worse, you have otherwise-sensible people seriously suggesting we use Klout to decide if people’s comments have value. It’s like a high school popularity contest for the Internet. Yuck.

As with most social media you and I aren’t the customers of Klout, we are the product. Only with Klout you can’t opt out of being packaged and sold. There’s a Klout page for me with my name, my Twitter icon, a score, details about my social network. Klout refuses to remove it despite my persistent requests, because “your Tweets are publicly visible on your profile page”. Which is true (and an important principle I agree with) but ignores the sheer ugliness of creating scores and profiles for people who do not want them. I’m not the only one offended by this.

The Some Bits weblog is over 10 years old now, my first posts were in October 2001. The topics haven’t changed much: that first month was a random mix of technical content, tech news, cultural commentary, politics, and Victorian urinals. The format has changed, I used to write more short single link posts. Activity level has varied; here’s a calendar view of posts, one box per day.

My weblog is an old school blog, a public diary of things that personally interest me. I mostly write as a way to summarize what I’m learning about something new like living in Paris or flying airplanes. Sometimes I post to influence others with an opinion about Klout or Facebook or SOAP or the like. On rare occasions I troll. But even though this blog is mostly written to satisfy myself, I do write it for an audience. It takes me most of an hour to write these short messages. (See my secret work blog and defunct game blog for much less edited stuff.)

I love the old school blog format. I’ve got a big list of RSS feeds for other blogs like mine; some recently active favorites include Justin Watt, Jason Kottke, and Rafe Colburn. The list is much longer with less active people Mike Migurski or Andy Baio; I love it when a blog post pops up after six months of silence. I regret that “blogging” for most of the world now means crappy journalism; pro-blogs with 20+ posts a day salted with ad targeting keywords is tiresome.

The ease of Twitter’s short format has definitely sucked some of the energy out of personal blogging. But as Anil says, if you didn’t blog it, it didn’t happen. Go ahead, write something, it’s not hard! Even if no one but yourself ever reads it it’s worth your time. For hosted blogs these days I like WordPress.com. I keep meaning to check back on how Blogger is doing, too.

The New York Museum of Modern Art has an interesting exhibit right now, Talk to Me, showing recent digital art. Stuff like Jason Rohrer’s Passage, Aaron Cope’s PrettyMaps, Eric Fisher’s geovisualizations, Dwarf Fortress, Mike Migurski’s Walking Papers, Nicholas Felton’s Feltron reports, and Josh On’s They Rule. You know, all those cool things you look at when the links go around the Internet. In a real museum, bonafide art.

Only if you’re reading this blog post there’s not much point in going to the actual exhibit. Because the links above (and the exhibit’s site) are way more effective than seeing digital interactive art hanging dead on walls of a museum. I’m really glad that MoMA has this exhibit and Paolo Antonelli curated it very shrewdly. It’s great to see this work taken seriously and shown to MoMA’s audience. It’s just a shame the art carries over so poorly in a museum.

Passage is staged as a screen stuck to the wall with a joystick. You can play it yourself! But Passage requires a few minutes reading to understand and a few more minutes to play and no one was engaging that way in the museum. At best someone would pick up the joystick, fiddle a few seconds, then put it down again and wander on. The Tentacles art looked great, but the interaction required taking your phone out, getting on the MoMA WiFi, installing a custom app, then trying to figure out how to operate it. I actually went through all that trouble only to find the interaction server wasn’t working. The WiFi failed soon after. I felt really bad for MyBlockNYC, that exhibit had completely failed displaying an empty browser window with, I kid you not, Evony and Wizard 101 toolbars on it. One enterprising visitor later figured out how to bring up a weather website on the giant screen while other patrons raptly looked on at this exhibit of Art. (I resisted the urge to add 4chan to MoMA’s exhibit, although it honestly belongs there.)

Interactive exhibit tech is hard and it’s mean to pick on stuff that crashes. But then there’s the projects that didn’t exhibit well at all. Migurski’s brilliant Walking Papers is reduced to some printouts and a paragraph of text, I fear no one looking got the point at all. And the Square exhibit is particularly baffling: a credit card dongle stuck to a wall on top of a crayon drawing of an iPhone. Really?

For a contrasting approach to digital exhibition, IBM did a nice job with the design of the Think exhibit at Lincoln Center. Beautiful giant LED display at human scale, you could walk right up to it. And an interactive video room that while a bit too Hypercardy for me was genuinely beautiful and engaging. The content itself wasn’t so interesting, but the presentation was good.

I guess the real challenge here is how to collect and curate digital art with the imprimatur of a Legitimate Musuem. Video art and conceptual art have similar display challenges, it’s hard to make them compelling in a gallery. I think it’s a mistake to relocate online art to a museum; what works is art designed for the museum space. For the rest we need some other institution to help us curate and critique what’s going on in the Internet.

PS: the most effective interactive art display may well have been the ad for donations.

What strikes Ken and I most is how much better and how many more dining options there are in New York than San Francisco. In SF we like to think of ourselves as foodies, but it’s pretty provincial compared to NY.

Les Halles (Financial District). Atavistic French brasserie. Very basic menu, great if you want steak frites and not so much otherwise. Reasonable preparation, nice room, surprisingly inexpensive. I wish I had a place like this in San Francisco.

Dos Toros (Village). A taqueria serving SF-style burritos, in New York! Considered the best burrito option among many SF expats. I thought it was good; honestly as good as most of the places in SF. Nice spicy sauce, good carnitas, good cheese. Too salty, and the staff of four rolled sloppy burritos in twice the time a single person in SF can do. But it was good.

reBar (DUMBO). Didn’t eat, just drinks, but liked it. Comfortable place to meet after work, nice beer selection, cool art in the building.

Wolfgang’s Steakhouse (Tribeca). Ken wanted a New York steakhouse, this was pretty great. Really amazing quality steak, both the filet and the ribeye, although the portions are comically oversized. Better quality of beef than Ruth’s Chris, for what it’s worth. Good salads, OK service, kind of noisy room.

Bar Boulud (UWS). Part of Daniel Boulud’s empire, the Bar is a more casual cafe kind of place. Serious brunch menu, including very well made omelets (a lost skill in most of the US). Also a lovely charcuterie selection, many delicious meats. Great place for a smart lunch.

Chez Josephine (Hell’s Kitchen). 11pm dinner after theater, French, with nice live piano music. And a wonderfully welcome very gay staff. The food was, frankly, mixed quality; both our entrees were fairly dull meat+sauce, but the salads were nice. Really enjoyed the room and the service, though, a nice post-theater meal.

Highpoint (Chelsea). We were in the mood for a boozy brunch in Chelsea, Highpoint was perfect. Great breakfast bar, terrific Bloody Marys and an interesting list of other cocktails. The eggs benedict were nice. The food is quite inexpensive, too, good value.

My first few days in NY have had beautiful weather, so I’ve done a lot of walking through Manhattan’s open spaces. Here’s some notes and photos.

Roosevelt Island is this wacky bit of Manhattan borough, an island in the East River. There’s not much out there, condos and hospitals. The reason to visit is the open space, nice long walks on both sides of the island and great views of the Upper East Side. Also grass, parks, even barbecue grills for anyone’s enjoyment. It will be particularly nice to visit in 2013 when the southern park is fully open. It looks like they are preserving the creepy hospital ruins there as part of the park, a nice touch of gothic drama in an otherwise bucolic place. (Roosevelt Island is also a popular video game setting; several GTA4 missions start at that hospital.) The Tramway is strongly recommended; it’s a short ride but the view as you go sailing into midtown between the buildings is terrific.

The High Line is New York’s most famous new park, a nicely landscaped mile or two of abandoned railroad track. Having a park up above the city streets makes all the difference, a nice quiet remove. They also did a stunning job on the design with many great places to sit and watch other people. There’s too damn many people, actually, and the linear park design makes it hard to get any space. So it’s more of a people watching place than an open area.

Thanks for all the comments and suggestions in response to my round 1 post. New Yorkers are opinionated about dining! Thanks in particular to Marc for a list of more casual options near my hotel. I’m really enjoying the density and diversity of restaurants in Manhattan. Haven’t done much very fancy in the past few days, but we’ve got a list of ridiculous plans next week: Craft (again), 11 Madison, Del Posto, Daniel, Blue Hill Farm, Bouley, Mas Farmhouse, Le Bernardin. It’s too much, honestly, may need to pare that back.

Shake Shack (Flatiron). World famous burgers for a reason. They are very delicious thanks mostly to the large amounts of fat and salt in the burger. Eat them quick; I imagine in 10 minutes they’re disgusting. But on a nice day in Madison Park they’re awesome.

Sushi Yasuda (Midtown). Very widely recommended. It was good, but nothing particularly exciting or unusual. Much like the nigiri you can get anywhere, just very high quality. I’m told sometimes you get unusual things like toro from 4 different parts of the world. We got some great toro but it was unidentified. Yasuda has moved back to Japan, I wonder if the current staff is simplifying?

Pegu Club (SoHo). A nice bar with serious cocktails. Great drink list, nice room, silly uncomfortable furniture. I’ll go again but sit at the bar.

Emporio (Nolita). Italian picked at the last minute because Torrisi was awkward. Nice place, comfortable and quiet at lunch. The pasta was tasty but a bit overcooked.

Bar Pitti (Village). Another casual Italian, but very hectic. We had two courses and a bottle of wine and were out the door in 45 minutes, with crazy brusque service. But the boar pasta was just fantastic and it’s a good place, if I lived here it’d be one of my neighborhood joints.

Egg (Williamsburg). Lovely American breakfast / lunch place. Really excellent pork sausage, good duck hash, tasty comfort food prepared well. Sticking with the Williamsburg theme, the hipster service was comically incompetent but charming and friendly.

Spitzer’s (Lower East Side). Nice little corner bar with wide open windows on a good weather day. Plenty of beers on tap, also a good grilled cheese sandwich. The fries looked good. NY doesn’t have nearly as many pub-type places as we have on the west coast, this one was nice.

Ken and I are in New York for a couple of weeks. We are food tourists. Here’s some notes on places we’ve eaten.

Craft (Flatiron). Our first and, so far, best meal in NY. Perfectly cooked bits of delicious meat. My short ribs were excellent, as was my ballotine of suckling pig, a sort of charcuterie. All very simple, but also very solid. Good service, nice room. I may go back.

Le Gigot (West Village). A solid little French restaurant, of the sort you can find everywhere in Paris and almost nowhere in San Francisco. Nice salad, slightly odd cassoulet. The meats were all cooked perfectly but the broth was a bit weird, too dark and tasting of Kitchen Bouquet. Still, I liked the place.

Thalassa (Tribeca). Somewhat upscale Greek. It was OK, but not great. Very simple and spare cooking, which is appropriate: grilled fish with a bit of olive oil, grilled shrimp with a bit of olive oil. Interesting fish, good quality, but neither were cooked quite right. Execution is everything in this kind of place and it fell just a bit short.

Mercer Kitchen (SoHo). I have a fond spot in my heart for this American classic restaurant, one of the best cocktails I’ve ever had was a French 75 here some 10 years ago. Sunday brunch was uninteresting. Too crowded, boring menu, indifferent hamburgers (!) and omelettes. Resting on a reputation.

Soho Sushi (SoHo). Delivery sushi at a friend’s. Pretty good, really, but with delivery it’s hard to be sure what you’re really getting. Some of the rolls were ill advised.

Pigalle (Times Square). A solid little French brasserie, of the sort you find everywhere in Paris but almost nowhere in San Francisco. Not quite as solid as Le Gigot. I’ve been three times now and it’s reliable, spacious, and the cooking is adequate. Lunch was uninspired.

John’s Pizzeria (West Village). Glad I got the NY pizza out of the way. Entirely adequate, uninteresting, I can get as good a pizza in San Francisco only delivered to my house with my wine and my television. (Or in Zürich, or in Paris, or.. pizza is a commodity.) The best thing about John’s is it’s near Grom; save room for gelato.

Where I really want to dine are 11 Madison, Le Bernardin, and Per Se. Sadly they are all very hot tables and three weeks was not advanced enough booking.

All of my bash shell scripts start like

#!/bin/bash
set -eu

Why? Safety. -e means bash will exit if any of the commands it runs returns an unhandled error. -u means unset environment variables are errors. It’s a little safety net for shell scripting.

The iPad is a very useful gadget in the cockpit. This guide to iPad hardware and software is for pilots who know more about flying than computers and are looking to add an iPad to their general aviation toolkit. Most of these notes apply to the iPhone too. I will update this blog post periodically as things change. You may also find ForeFlight’s iPad Proficiency for Pilots useful.

What does the iPad do? The iPad is a computer with a revolutionary new user interface. It’s ideally situated for the cockpit: easy to use one handed, even balanced on your knee. There’s a variety of excellent aviation software for the iPad. It’s also a nice companion for Internet while travelling and for entertaining passengers.

Electronic Flight Bag plus more. The key aviation software I use is ForeFlight. It contains most everything you need for flight planning in the US: VFR and IFR charts, NACO terminal procedures, flight planning, official weather briefings via DUATS, supplementary weather (radar, METARs, etc), flight plan filing, A/FD, even fuel prices at airports. See their demo video for more. ForeFlight is very complete: I planned and flew an 8 day trip from California to Florida and back entirely using ForeFlight. ForeFlight costs $75 / year for full chart updates for the entire US. An extra $75 for a pro subscription gets you georeferenced approach plates so you can see your actual position via GPS. There are alternatives to ForeFlight with some different strengths: WingX is the best known.

Other aviation software. Pilot Wizz and E6BPro are both good calculator/converter apps; Pilot Wizz has a great weight and balance screen. Mr. Sun is a handy little sunset calculator. SkyCharts is a simple alternative for chart display with good enroute display. Jeppesen Mobile TC lets you use Jepp charts, but the app is very limited. OffMaps v1 is a good way to cache street maps for viewing while airborne. MotionX is a nice app for recording GPS tracks to look at your flight later on Google Maps. X-Plane is a fun simulator. I also use a bunch of web pages regularly when planning flights: Fly2Lunch, 100LL, etc.

iPad hardware. You have two choices when buying an iPad: how much storage and whether to get 3G. ForeFlight itself takes 6GB for full charts. Double that for updates and a 16GB model is barely sufficient. I suggest buying at least 32GB, more if you want to bring video and music along. 3G is a toss-up. If you get the WiFi only model then it will have no builtin GPS, but see below about external GPS. I like the 3G because it gives you the chance of having Internet access in the middle of nowhere if there’s no WiFi. Verizon vs. AT&T is a push, but the AT&T model will work better in Europe. (The original iPad 1 isn’t missing anything essential, although the extra CPU speed on the iPad 2 is nice. If money is tight, consider a used iPad 1).

GPS. The iPad with 3G has an assisted GPS receiver built in. In my experience it’s not useful in the air; some pilots say it works for them but it doesn’t work reliably for me. For about $100 you can buy an external GPS. I have a Bad Elf GPS (Amazon) and its performance is excellent, even when the antenna is sitting in the back seat. I’ve also heard good things about the GNS 5870 (Amazon) and XGPS150 (Amazon) as wireless options. Note that GPS is not necessary to get a lot of value out of the iPad. But if you want a backup GPS in the cockpit or you want to display your position on charts for situational awareness, get an external GPS.

Things to avoid. If the iPad overheats it shuts down, a potential crisis if you’re flying an approach. Keeping it out of the sun seems to be sufficient although it’s worth noting Apple says the iPad only operates to 10,000′. The iPad2 has magnets in the tablet itself and in the case that can swing your magnetic compass from at least 10″ away, possibly including when sitting on your glareshield. Do not use the iPad as a substitute for an IFR GPS or for any other required equipment. It is neither accurate nor reliable enough to trust your life to it. That goes double for the apps that emulate an attitude indicator. And of course don’t get distracted: if something goes wrong put the toy down and fly the plane.

Accessories. People seem to like this clip for knee mounting, also this aluminum case. The iPad battery is rated for 10 hours, so charging in flight may not be necessary. Cigarette lighter chargers work well. Remember that planes are often 28V and the iPad wants to draw 10W. The iPad fits very nicely inside the best flight bag in the world.

Alternates: iPhones, Android. For most purposes the iPhone works identically to the iPad; ForeFlight is great on the iPhone too. The big difference is screen size: the phone is too small for plates and working with charts takes a lot of squinting. For alternate tablets I’m hopeful that Android will catch on and offer some competition for Apple, but so far the iPad stands alone. Android requires totally different software; ForeFlight does not have an Android version, but WingX does.

Conclusion. The iPad is great for pilots and can replace most of the flight planning you currently do with paper charts and computers. There’s a lot of options: my bottom line recommendation is a 32GB + 3G iPad ($730) with ForeFlight ($75/year). If you want a GPS backup, spend $100 on a Bad Elf GPS.

Ken and I are headed to New York for the second half of October. No particular reason, just to hang out and see friends and eat well and maybe see a show or two. If you’re in New York or on your way through, drop me a note and we’ll get together! Recommendations for restaurants and things to do are also welcome. We’ll be staying down in SoHo.

I was playing around with the FADDS aviation database and came up with a report that shows the top 1000 airports in the US as ordered by number of general aviation operations a day. The FAA source data is not entirely accurate but gives a general idea of activity.

No huge surprises: the biggest GA airports are near LA, Phoenix, Daytona Beach, and Minneapolis. A lot of these ops are probably training related; every single touch-and-go practice landing gets logged. I was surprised my little home airport of San Carlos is #71 at 350 ops / day. I have a hard time believing that number is really correct, that’s roughly 30 ops / hour of daylight.

Ken and I ate well on our trip up to Vancouver, BC and back. Some notes on where we went.

In Vancouver, on the fine dining end we were impressed by West which is still excellent despite the changes over the years. The most inventive cooking we had was at L’Abattoir, really precise and interesting food in a trendy, slightly too noisy room. For more old school we enjoyed dinner at Hy’s Encore, classic steakhouse with terrific service and very good food. Also enjoyed a more casual lunch at Joe Fortes, seafood in a businessman’s bar and grill setting. The awkward middle ground was Le Crocodile, the service and room were not up to the promise (and expense) of the classic French menu. Not that we suffered, mind you, and the wine list is excellent. Regrettably we didn’t get to Vij’s despite many strong recommendations.

Portland treated us well on two quick overnights. Jake’s Famous Crawfish was great; it can be hit or miss but we had a hit that night with fantastic crab claw dish with mushrooms and artichokes. And an interesting evening at the Gilt Club, a sort of hipster late night restaurant / bar. The menu was a bit odd but the cocktail selection was fantastic. I had the BLT, “House Roma tomato vodka, basil & lemon olive oil and fresh lemon - up with bacon salt.” It was amazing.

Ken and I flew ourselves up to Vancouver for his birthday last week. It was our first international flight in at least ten years, with all the new US security stuff, and frankly it seemed so complicated I suggested we just land in Bellingham and drive across the border. But we figured out all the paperwork and it wasn’t so hard after all. There’s a variety of detailed guides for flying to Canada (I like AOPA’s), here’s the gist.

Weeks before the flight we got the plane documented for international flight. Proof of insurance, a radio operator permit for the pilot, a radio station license for the plane ($160), a customs decal ($30). Plus the usual plane documents for any flight. Nothing difficult but it requires a couple hours figuring out government web sites, then possibly weeks before the documents are shipped (ours came in a week). No one ever asked to see anything but the customs decal.

The Canadian border is pretty easy. When you arrive, they want a phone call 2 – 48 hours before the flight saying when you’re coming. Then on the ground you call again from the customs box. In theory they may come inspect you and your documents, but for us they just gave us an arrival number over the phone. There’s no specific border requirement when leaving.

The US border is more complicated. The big requirement is filing a passenger manifest with eAPIS for both your departure and your arrival. The web site is awkward (and broken in Chrome), so expect to spend some time. Fortunately you can file days in advance. The eAPIS filing gives you permission to leave the country via email; I also called the local customs office and they seemed confused that I was asking them. You do need to notify customs at your arrival airport 1 – 24 hours before you come back in, mostly so there’s someone there to meet you. We arrived right on time and were done in 10 minutes at the plane: radioactivity check (!), passport check, customs decal check. Pretty easy.

Flights across the border need to be on a flight plan, both for the US and the Canadians. We went IFR so that was natural; if you go VFR you need a flight plan with a specific squawk code. Canada does charge for ATC services, apparently the bill is in the mail.

I went to FOSS4G this year, the primary conference for open source map hackers. I’ve been playing with geographic software for awhile but have little expertise in the area, so this was a fun and quick way to get up to speed. It was a good conference, well worth my time, here are some notes.

My primary takeaway from the conference is there are a lot of options for open source geo and it’s still difficult to know which to use. So many choices. For now I’m keeping my head down and learning GDAL, PostGIS, Mapnik, Tilestache, and Polymaps. PostGIS is the big one, a set of spatial extensions to the Postgres database. The book PostGIS in Action is a big help. Also one small takeaway: a whole lot of people are using Node.js for real stuff, not just the fashionable web kids in the Mission.

I met or heard some very impressive people and companies. Development Seed is building a lot of great mapmaking software, including the TileMill tool I played with recently. Stewart Long of Public Laboratory is doing inspirational work with citizen aerial mapping. Vizzuality got a lot of buzz as a Spanish company doing beautiful map design. Michael Byrne gave an exciting talk about the work he’s doing inside FCC to make data available. And Brian Timoney gave a strong talk on how far government agencies have to go in publishing data effectively.

The rest of this post is a link dump of some of the people and things I saw at the conference. I’m no “curator,” just a typist, sorry for the lack of organization.

People

Tom MacWright
Peter Batty
Frank Warmerdam
Schuyler Erle
Tripping the Bits
Howard Butler
Paul Ramsey
Dane Springmeyer
AJ Ashton
Stewart Long
dbsgeo.com

Companies

Cirrus Technologies
Development Seed
The Grassroots Mapping Forum
The Public Laboratory
Vizzuality
Paragon Corporation
skobbler
SimpleGeo

Code (non geo)

Socket.IO: cross-browser WebSocket
Inkscape.
Apache JMeter
Node.js kuler
Color Trends + Palettes :: COLOURlovers

Code

Carto: A CSS-like Map Styling Language
Virtual Machine for TileMill
GeoNode
GeoScript
GeoServer
GeoTools The Open Source Java GIS Toolkit
glob3, a 3D GIS framework
Imposm
PDAL - Point Data Abstraction Library
libLAS - ASPRS LiDAR data translation toolset
Cartagen MapKnitter
MapMill: Crowdsourced map processing
Opticks
WKTRaster – PostGIS
SpatialSearch - Solr Wiki
Arc2Earth | Publish your GIS data instantly to the Cloud
Geospatial Indexing - MongoDB
cloudhead/cradle
couchbase/geocouch
mapbox/landspeed.js - GitHub
vmx/geocouch-utils
OSX Downloads | dbsgeo.com

Code with a Web bent

mod-geocache - apache module for caching map tiles
Leaflet - JavaScript library for interactive maps
MapBox
MapBox / Wax
MapFish — MapFish
Mapnik C++/Python GIS Toolkit
Mapnik2
MapQuery
MapServer
OpenLayers: Home
TileStream
MapMint - new Web GIS experience
Mapnik and Node.js
tinyows
AppGeo/geo - GitHub
chelm/FOSS4G-Node.js-and-Polymaps-Workshop
stamen/modestmaps-js
Vizzuality/Windshaft

Data

Geography Markup Language
CartoDB Blog
Dutch open data
Earth Observation Day
Geospatial One Stop
Grassroots Mapping
The National Map
RTMM Home
Open Database License
Developer - National Broadband Map
Aerial Photographs of the California Coastline
U.S. Census Bureau - TIGER/Line
FCC Developers
OS OpenData - Ordnance Survey
Simple Feature Access - Part 2: SQL Option)
OSSIM

Products and Demos

WebGL Earth
Map Chat
dthompson/geotrack - GitHub
dthompson/mapchat
sabman/geochat-example
Welcome to OSGeo-Live 5.0
MapQuest Open
NYC hurricane evacuation map (web)
NYC hurricane evacuation map (PDF)
Geospatial analysis tool for Red List assessment
This Tract
uDig : Home
WorldMap
National Broadband Map
CartoDB

Portals

OpenGeo : Home
Geospatial Information & Technology Association – GITA
Welcome to INSPIRE geoportal
OSGeo.org | Your Open Source Compass

Just hit a weird problem on my iPad: trying to install an app or upgrade existing apps resulted in a mysterious “Error 1004”. Apple’s official help is useless and the user forums are full of amusing cargo cultism (“reset your time zone!”)

The solution seems to have been flushing Safari’s cookies. Or maybe its cache or history. One way or the other, the iPad App Store had the wrong idea about my login. And now I have to log in to everything again.

Homebrew is good software, a straightforward way to compile and run traditional Unix tools on MacOS. However it’s got some sharp corners, particularly for Python, and you need to be paying attention. Long story short: if you’re using Homebrew’s version of Python, be sure you have /usr/local/share/python in your PATH. If you don’t your environment is broken and wrong. In particular, pip and easy_install will be installing things in Apple’s version of Python, not Homebrew’s.

If you’re the type of guy who reads directions, you already have that odd directory in your PATH. Because brew install python told you to do that. You can see it again, go run brew info python. See? This is all well documented, too. I’m not the kind of guy to read directions. Years of using Debian has conditioned me that the package manager will install things cleanly and for the very few things that need intervention I’ll get a little console warning in bright colors. Homebrew’s not like that, many packages print caveats and you’re expected to read them. (In the same vein, if you’re doing geo-hacking and use gdal, you might want to check brew info gdal and whether your PYTHONPATH includes /usr/local/lib/python. Also see these notes on building numpy on Lion.)

I opened a github issue suggesting brew doctor diagnose this missing Python path. I’d just write the patch, but the hurdle of doing Ruby and github right is a little high for me right now.

TileMill is good software. It’s a turnkey system for designing custom slippy maps for web publication. It’s very easy; in four hours I was able to create a simple map of US airports and airspace without having much background in the software or GIS concepts involved. Making a map requires a bunch of things: data, style, publication. There’s been open source ways to do this for a few years but it’s pretty complex tech, TileMill makes it much simpler. I put my project up on GitHub, the map is live. For data, TileMill comes with useful datasets from Natural Earth like country outlines, roads and rivers, and urban areas. I added custom data: airport locations from GeoCommons KML and airspace definitions from FAA shapefiles. Easy to add, just a bit of sweating the missing projection metadata from my sources.

Designing is where TileMill really shines, a beautiful browser-hosted GUI for making maps. You style the map in Carto, a simple CSS-like language (here’s my style file). The development environment includes a live view of generated tiles and a great little editor. Really a joy to use, especially considering all the complex Mapnik stuff going on behind the scenes. TileMill is capable of more complex design than I used, like labels and interaction, but it’s also deliberately simplified compared to using the underlying tools directly.

Publication is simple. You can create single PNG or PDF images like this. Or you can generate an MBTiles file, a bundle of all the pregenerated tiles for a web map. Hosting a map at web scale yourself is a bit complicated. But you can upload your tiles to the inexpensive TileStream hosting service or just copy it to the free iPad app for personal use.

I’ve been learning about open source geo for a year now and it’s easy to get overwhelmed. TileMill is impressive in how it combines so many powerful tools into an easy to use product. Great work by Development Seed, the company behind TileMill. They’re supporting a lot of interesting humanitarian projects with great mapmaking tools.

In preparation for heading off to the mapping conference FOSS4G I made a roundup of the map hacking projects I’ve done in the last year or so. There’s more there than I realized. Things that are more like real products are at the top.

• WindHistory.com, a map of prevailing winds.
• Fisk Map, a simple web view into an amazing historical map of the Mississippi River.
• An unreleased project I’ve been working on that maps GPS tracks from airplane flights.
• iPhone location map, before Apple fixed the location data leak.
• SRTM overlay drawing a stark raster of high elevations over a base aviation map. (Big static image).
• A bunch of base maps, a hacking framework for trying out different raster maps.
• Minecraft polymap, a quick hack showing a loupe to let you see an underground layer

Map data has become much more accessible to journeymen hackers; I did most of this work without using or, honestly, understanding most traditional GIS tools like GRASS and the various ESRI products. A lot of my hacks have been motivated by Polymaps and how easy it is to make maps in Javascript with data overlays.

I just launched an update on my Wind History project at windhistory.com. The big new feature is month filtering. You can now choose to look at prevailing winds for a specific month or animate a map of the shifting winds over the year. Try it out in the Bay Area, you can clearly see how the winds change in the winter. The rotating Kansas winds are interesting, too.

I also snuck in a couple of minor changes. The main map page now uses the geolocation API to initially position the map near where you are. And I improved the station page a bit, adding some much-needed tick marks on the radial axis.

The majority of web users still use IE7 or IE8, but the majority of web designers don’t have access to those crappy old browsers. That makes testing web sites awkward. Fortunately there’s a free and legal solution for occasionally running IE using virtual machines. Microsoft makes virtual disk images for Windows with IE available specifically for testing web sites. Boot those disk images in a virtual machine and you have a full virtual Windows system with an old IE for testing.

Microsoft provides the images for Virtual PC, but they work in the free VirtualBox VM as well. Here are detailed notes on getting Vista with IE7 running on a Mac running Lion. The idea should be the same for other systems.

1. Download and install VirtualBox. It’s pretty painless on MacOS.
2. Download the disk image you want from Microsoft. They’re large, several gigabytes in a multipart RAR archive.
3. Unpack the archive using The Unarchiver (Macs), IZArc (Windows), or your tool of choice. Out will come a large VHD file, 14 gigs for Windows Vista.
4. Launch VirtualBox and press “New” to create a new virtual machine.
5. On page 2 of the Wizard, name your machine something memorable and set the Operating System to “Microsoft Windows”, Version to “Windows XP”. (The Vista pre-set will not work: it uses SATA drives which the Windows system doesn’t support. Choosing XP forces IDE drives.)
6. On page 3, configure RAM. I have lots of RAM so I gave it 2 gigs.
7. On page 4, choose “Use existing hard disk” and click the little green arrow folder to browse to the VHD file you unpacked in step 3.
8. Click “create” and exit the Wizard.
9. In the VirtualBox Manager, right click your new VM and choose “Settings”.
10. In Settings, click Network, then Advanced, then change Adapter Type to “Intel PRO/1000 MT Desktop (82540EM)”. (The Windows system doesn’t have drivers for the default PCnet card.)
11. Exit settings and double-click your new VM to launch it.
12. In VirtualBox, choose Machine / Disable Mouse Integration so you can see your mouse cursor.
13. Wait patiently. Your Vista virtual machine should boot and give you a login screen. (If it blue screens and reboots, you have a driver problem: are you sure you told VirtualBox WinXP and not Vista?)
14. Log in as user “Admin”, password “Password1”.
15. Congrats, now you’re running Windows in a VM!
16. Inside the virtual Windows, you can mostly ignore or cancel all the boxes asking about drivers, activation, security, etc.
17. (Optional) Choose Devices / Install Guest Additions to put some extra drivers in the Windows machine to make the VM more pleasant.

A few caveats.. I’m no VirtualBox expert, there’s probably cleaner ways to configure the virtual machine so it runs better. I never could get the Windows XP system to boot. Also the virtual Windows boxes are still subject to Windows Activation, so you’ll need to rebuild your machine from scratch every 30–90 days.

By the way, there are other ways to try a site on many browsers. A service like browsershots or BrowserLab is great for a quick check. But if you want a full interactive experience, a VM is a good way to go.

If Chrome is your default browser, then when an external app opens a link Chrome opens it in a new tab in an existing window. We can argue whether that’s the right behavior in general, but it’s absolutely wrong if you’re using Spaces and the Chrome window is in a different space. Chrome jerks your whole desktop to the left, forcing you to the Space where it happens to have a window open, completely ignoring the Mission Control preference “When switching to an application..”. It’s terrible.

Fortunately there’s a fix. Long story short: you set your MacOS browser to a little program that tells Chrome to open a new window. Tabs still work like normal in Chrome, but external links now always create windows and, as a side effect, avoid the horrible Space switching.

That little program is in AppleScript, an ancient, opaque language. (Compiled scripts? Really?) Here’s the nut:

on open location theURL
  tell application "/Applications/Google Chrome.app"
    make new window
      activate
	set URL of active tab of first window to theURL
  end tell
end open location

It’s a bit promiscuous with the windows if you’re used to the new tab behavior. The ideal thing may be to re-use an existing window if one is visible in the space, otherwise open a new Window. It’s a bit of a mystery why Chrome lacks a new window preference.

Since launching my Wind History map about three months ago, the site has had about 6000–9000 pageviews, or about 100 visits a day. Not exactly blockbuster, but I never expected the site to be a big hit. It’s as much a tech demo of advanced map datavis techniques as anything and at that it’s a success.

The most popular airports are San Carlos and Santa Fe, no doubt because I use them as examples. Folks are also notably interested in Snyder, TX (from an AOPA discussion); Camp Bondsteel (in Kosovo) and the NAS Whidbey Island and neighboring Oak Harbor in Washington.

The nice thing about a static site like this is it takes no maintenance. But I’m playing with some new features now. Geolocation API to move the map to where you are, fancier wind rose diagrams, and static HTML for dumb ol’ Googlebot.

The new Deus Ex game is fantastic. If you like intelligent, complex games run out and buy it right now. I played it on Xbox which was fine, PS3 is about the same and PC may be better.

The new developers absolutely nailed what made the original Deus Ex so great: gameplay options. You can sneak through levels avoiding the bad guys entirely, or go in guns-a-blazin and murder everyone, or hack your way through security, or find the hidden air ducts. It’s amazingly fun and they really got the gameplay right. The story’s quite good, too, with some good characters. Also some beautiful set design despite some limited graphics capabilities.

The game is a success in almost every way. The one failure is the boss fights, they’re awful. Weirdly, they were made by a third party. There’s various ways to cheap you way through them, best to just kill the boss quickly and move back to the fun game. And there’s a whole lot of fun in the game. May end up being the best game of the year.

I’ve taken a lot of crap for my various blog posts about how TechCrunch is not journalism. So I’m feeling a little vindicated today that the CEO of TechCrunch’s parent company agrees with me, in the newspaper of record of all places:

We have a traditional understanding of journalism with the exception of TechCrunch, which is different but is transparent about it.

The new CrunchFund is the big drama of the blogosphere today. If you’ve followed Arrington’s career it won’t surprise you, of course he thinks he can be the editor of a major tech publication while also investing in tech companies. He did it for years, with varying levels of disclosure quality. It’s part of the package along with the shoddy editing, the ignorance of basic concepts like off the record, the crazy personal vendettas. Arrington’s a blogger. A very effective, powerful blogger. He’s not a journalist. (Not that he disagrees: NYT, TechCrunch comments).

The frustrating thing is there is some actual good journalism at TechCrunch. I feel bad for the good reporters there: apparently they feel bad, too. I’d like to call out MG Siegler (aka parislemon) in particular for regularly writing well researched stories with insight and accuracy.

Maybe a real news outfit will hire the good folks out of TechCrunch. Both the Wall Street Journal and the New York Times have excellent blog-style tech coverage with proper journalistic methods and ethics. They should hire everyone away from the AOL clown car as soon as their acquisition lockups expire.

I just finished playing this computer game and I feel like crying. Is it time? is a little art piece (in Flash) about growing old and lonely. It’s really beautifully done, there’s a lot of hidden elements that work well. Some thoughtful discussion on RPS.

Adobe Lightroom is good software. Adobe rethought the digital photo workflow a few years back, tossing out the chaos of Bridge and Photoshop. And they got it right. Lightroom is like Picasa, Aperture, and iPhoto in that it’s as much about managing a library of photos as editing any individual image. But from what I’ve seen Aperture is the only other tool close to touching Lightroom’s professionalism.

If there’s one downside to Lightroom it’s that it’s a bit confusing to learn this new way of doing things. I still don’t really understand what all it can do. But it does a good job easing new users in and the basic things you need to do are easy and, well, basic. Import photos. Mark the good ones. Edit them a bit. Export them to Flickr. All easy to do.

I have special love for Lightroom today because I just migrated all my stuff over to my Mac and it couldn’t have been easier. My Windows serial number activated the Mac copy, no fuss. All of Lightroom’s metadata about everything I’ve ever done in the .lrcat catalog. My edits, tags, titles, all in this one single file. It’s portable. I simply copied the catalog and my image files to the Mac, told Lightroom where the photos had moved, and I was done. Stress free and transparent.

Lightroom’s not cheap, often $300 retail. But it goes on sale occasionally; right now it’s available for $145 on Amazon, as good a price as you’re likely to see. There’s also a generous free trial program. There’s also a good third party geocoding plugin.

My fancy new iMac is really power efficient. It averages about 70 watts when I’m using it; that’s about like a lightbulb. Impressive, considering it’s a 27″ LCD monitor. It’s less than half my old Windows desktop and about the same as my stupid idle A/V rack. Assuming I put the iMac to sleep half the day, that works out to 25kWH or $9 a month.

The iMac is basically a laptop bolted to a giant LCD monitor, with efficient mobile CPU and GPU keeping heat and power down. The biggest constant power draw is the LCD backlight; the automatic dimming helps quite a bit. Here’s some numbers (top end 27″ iMac: 3.4GHz i7 quad CPU and AMD 6970M GPU).

• 60-100W ordinary usage (depends on room light)
• 60-150W going from dimmest LCD to brightest
• 130-150W playing Starcraft / Team Fortress
• 210W maxing out CPUS + GPU with Bitcoin
• 50W monitor dimmed
• 35W monitor off, computer still on
• 1W sleep mode

Spinning up the hard drive adds about 5W. Maxing out a single CPU adds about 10W. The screen is most amazing to me: between 25-115W just for the display, depending on brightness. Weirdly, adding a second monitor via the display port adds 25W to the iMac’s power usage, even when the screen is asleep.

As efficient as the iMac is, it’s still worth the trouble to let it automatically sleep when you’re not using it. I’ve yet to have a Windows desktop where sleep mode actually worked; totally solid on the Mac, of course.

I’ve long thought my linkblog has better content than my real blog. It’s just a simple list of things I find interesting online: URL, title, short comment. About 5 links a day and I write for an audience; this isn’t just my personal bookmarks.

I just set up @somebitsLinks, a copy of my linkblog on Twitter. Tweets are a good medium for linkblogs, I hope some folks appreciate being able to see what I write there. Try it out, tell me what you think!

The Twitter feed is implemented by sending the Pinboard RSS to Twitter via dlvr.it, a third party product. It’s a nice service, has a variety of options for filtering and routing stuff from A to B. I’m hopeful it’ll have Google+ support as soon as Google gets its act together. They support Facebook now but that seems spammy. The one drawback is dlvr.it rewrites and shortens my links before they get to Twitter, doubly annoying now that Twitter in turn rewrites and shortens the links again. Sorry about that.

Why don’t modern PCs come with error correcting RAM? Bit errors in RAM happen, whether due to cosmic rays or hardware failures. Estimates vary: an error every 3 days on a typical PC or maybe every month. A survey of Google’s servers came up with 8% of DIMMs seeing at least an error a year. Whatever the exact rate, odds are a typical modern computer is going to see some.

Do RAM errors matter? Sure! For example, at Black Hat Artem Dinaburg introduced bit squatting, where you register domain names one bit off of popular names and wait for random errors to send traffic your way. He reports getting close to 100 hits a day off of just 31 domain names. Sam Pullara reports he’s able to duplicate the attack.

More generally, RAM errors cause random data corruption. They are entirely preventable with error correcting RAM. Indeed, the early PCs all had parity memory (error detection, not correction). ECC adds a little access time overhead and some expense for the extra RAM and ECC logic. But isn’t it worth it?

I just had an unwelcome scare on my new Mac, when I rebooted it took awhile and ended in a black screen saying “No bootable device -- insert boot disk and press any key”. Yikes! I’d set up a Boot Camp partition earlier but hadn’t installed Windows to it; apparently that broke things. I’m not the only one to have this problem, and thanks to this blog post I was able to fix it.

The fix was rebooting while holding down Cmd-Option-P-R. That resets the PRAM (aka NVRAM or BIOS settings). After that the system booted fine. It doesn’t look like anything else important is in the PRAM, although I’ve probably screwed up the Boot Camp boot chain.

Macs have a bunch of magic keys at startup. I’d have thought holding down Option for Startup Manager would have been the right fix, but it didn’t seem to do anything. Apple’s support entry on this problem is unhelpful (step 1: Start up to MacOS. Um, that’s the problem). I wish Boot Camp were more transparent; there’s a bunch of conflicting and confusing information about how the Apple boot chain works.

Apple’s Setup Assistant and it’s little brother Migration Assistant are total miracles. They make it easy to set up a new Mac by copying various settings, applications, and data from another Mac. Such a valuable feature; saves users literally days setting up a new computer. I don’t think Microsoft’s ever gotten this right. I have to think Apple’s sold a lot more new computers to users who know the upgrade will go easily.

I like my Air so much I went out and bought an iMac. I needed a monitor anyway; why not pay a bit extra to have a computer bolted onto it? So easy to set up. Plug in my Time Machine backup drive, say “import everything”, and I’m done. All the sensible settings were carried over. All my apps. All my weird stuff, like the Canon printer drivers and my personal Homebrew install and the odd hacks that use undocumented APIs. The only thing that didn’t just work on the new machine was Xcode; it was all there but for some reason my PATH didn’t pick it up.

I realize Macs have been able to do this trick for awhile, but it still seems like a miracle. The new App Store should make it even easier by streamlining licensing. If I understand right, if you buy an app on the App Store once you get the right to install it forever on any of your machines. I hope I have that right; that’s how the iOS store works and it’s how all my Mac apps so far have worked. That’s a pretty user friendly policy, also easy to understand. I wonder how it works in offices?

Apple’s magic Time Machine backup system works great. But it has one flaw: it’s inefficient with large files which change frequently. It makes a whole new copy of the file each time, there’s no diff mechanism. My Time Machine backups are 300 MB every hour despite me actually changing very little. The Time Tracker app helped me figure out what’s going on.

Chrome stores your web history in one giant file per month: 100 MB in two weeks for me! That file is constantly changing. Arguably history should be backed up, it’s valuable user data, but I’d rather not if it’s this inefficient. Unfortunately there’s no easy way to exclude just the history database; the filename changes every month and the directory it is in has other useful stuff.

My entire Dropbox repository is being backed up whether I change anything or not. I’m not the first to see this problem but it’s not clear what’s going on. (My guess: v1.1.40 and Lion.) My solution was to simply stop backing up Dropbox (why bother?). A related issue is Dropbox maintains databases in .dropbox that can also get unwieldy.

Under the hood Time Machine is pretty simple; it’s mostly doing the same hard link trick that rsnapshot uses with some help finding changed files from the filesystem. The GUI and easy setup is the magic, the sort of product Apple is uniquely good at building.

After years of using Windows and trolling my Apple-loving friends I finally bought a Mac. I love it. I last used MacOS five year ago (and before then, in 1993), so I’m coming to Lion with fresh eyes. This blog post is long for me and comes in three parts: the good parts of the Mac, the bad parts, and a list of software I find useful.

The good

Emigre has an underappreciated slogan: “Design is a Good Idea”. Apple gets design. The MacOS UI is good, generally obvious. The supporting documentation is excellent. Text and graphic rendering is beautiful: not blurry, it looks like paper on a high quality LCD. The hardware is a marvel, years ahead of what any Windows manufacturer can turn out. Of course I knew Apple had good design, but I’m still surprised how much I regularly appreciate it.

The other nice surprise is how handy it is to have Unix at hand. For years I’ve worked with two machines: Windows in front of me shelling in to Linux boxes for real work. It never really worked fluidly. With the Mac I can edit Python code and run it sensibly right on the same machine without weird filename collisions, network file system oddities, or Cygwin atrocities. I can augment my system with shell scripts. I can use most good old Unix software. I’m surprised how much this affects my daily work.

One pleasure of learning MacOS is finding all the grace notes. Preview and Quick Look are both lovely ways to just look at a file’s contents. There’s a real builtin dictionary with a handy Wikipedia client. There are buttons on the keyboard that pause my music or turn the volume up and they actually work. System sleep restores reliably. Windows claims to do most of these things but they never seem to work right.

MacOS also has some surprisingly sophisticated stuff. Time Machine is a terrific backup tool. The new Lion document model is brilliant; finally I don’t have to press some stupid “save” button. There’s automatic typo correction which makes me look smarter without being annoying. And Apple has figured out how to build systems around SSDs, a significant speed upgrade.

One other pleasant thing about the Mac ecosystem is the third party application market. Macs have a long tradition of solid, well designed software that is not free. I love that the three most popular text editors for MacOS all cost $40+. It’s great that Skitch, Alfred, and Reeder are beautiful apps that charge enough to support the developer’s lifestyle. The jury is still out on the Mac App Store, but given the success of the iOS market I’m optimistic we’ll see more independent developers making a living making beautiful apps.

The bad

As much as I admire the Mac, when something doesn’t work quite right the sharp corner sticks out. Here’s some of the things that have bugged me in my first couple of weeks with MacOS.

Installing software is poorly managed. Download a .pkg file, mount it as a virtual drive, then drag the app to the Application folder, then unmount the package. How awkward! Some stuff comes with an installer script, but then there’s no standard uninstaller. Mac fanboys say “just drag the app out” but that only works for self-contained apps, not printer drivers or packet sniffers or anything that installs in several places at once. I’m amazed this doesn’t cause more problems.

There are too many keyboard shortcuts. The Home and End keys are broken. And there are way too many modifier keys: do we really need four? Windows generally gets by with two. (Hilarious example: Ctrl-Shift-Q is “play macro” in SublimeText. Cmd-Shift-Q is “force quit”. Ask me how I know.)

There are too many preferences to customize display. Why do I have a choice of two graphical animations for minimizing windows to the Dock? Can’t Apple pick one? Third party apps can be really bad, with 50+ display preferences. Good designers picks one good design, they don’t leave it up to individual users to cobble together their own. This opinion may be controversial.

The split between applications and windows is confusing. Some apps like Preview stay running even with no windows open, others like Font Book close when you close the last window. Lion has decided it’s OK to quit applications automatically if there are no windows. Chrome has a special hack to stop you quitting accidentally, there’s also a general solution. It’s just strange. I do like having a unified application menu bar, though.

iTunes is a fucking disaster. The UI works like the poorly managed work of summer interns and ignores the standard MacOS UI conventions. And it hangs when importing music; is it really single threaded?! The database is a joke, poorly managing data and regularly corrupting itself. The sync model with iOS devices is confusing and broken. iTunes is the gateway to Apple’s flagship consumer devices, it’s astonishing it’s been this bad for so long. Fire is the only way to be sure.

Software

One of the fun things about getting a new computer is finding and installing all sorts of software on it. Here’s the stuff I’ve found particularly useful.

Chrome, the web browser. The best browser choice and since most of us spend 90% of our time in a web browser it’s worth picking the right one. Chrome benefits greatly from AdBlock and a hyperlink underline fix.

Alfred, a fantastic search UI and app launcher. It’s a replacement for the Spotlight UI (and the abandoned QuickSilver). I use it some 30 times a day already and have only started to learn what all it can do.

Growl, a desktop notifier. This little UI shim belongs as part of MacOS, it’s amazing it’s not integrated yet. Fortunately a lot of third party software supports it.

MenuMeters, a system monitor. Handy for tracking bandwidth and CPU usage. Apparently animating in the menu bar is a no-no these days but I find it useful.

Dropbox and Cyberduck for accessing remote files. Dropbox is a marvel, particularly good for sharing files with your iPad. I’m not wild about Cyberduck but it does a decent job with SFTP and is free. If I did a lot more of this kind of file sharing I’d look into Transmit.

Adium, a multi-protocol instant messenger. Now that iChat supports several protocols it’s less necessary, but it’s nice that Adium does IRC too.

Twitter, an interface for Twitter. I like having it open instead of visiting the site a couple of times a day.

Reeder, an interface for Google Reader. I love the iPad app and so far am liking the Mac app as well. Google’s web interface is good too but the native app feels cleaner and faster.

Skitch, a screenshot tool. Makes life easier when taking a lot of screenshots and the integrated web upload is handy.

Sublime Text 2, a text editor. I’m still evaluating this, the other contenders are TextMate, BBEdit, and TextWrangler. I like Sublime because it’s simple, just a text editor and not an IDE-wannabe. Also it’s under active development; I’m not about to drop $60 on a text editor that hasn’t even bothered to work right on Lion.

iTerm2, a terminal emulator. I don’t really know why this is so much better than Terminal but everyone else seems to love it, so I use it instead.

Homebrew, a Unix package manager. Makes it relatively easy to download and compile the random Unix programs MacOS is missing. Alternatives are MacPorts and Fink, the experts I trust all say Homebrew is better. I sure miss Apt and the robust Debian software library.

Save some money and power and repair bills: go clean your refrigerator’s condenser coils. You should be cleaning them every few months and you probably never have. It’s really pretty easy, takes just a few minutes. Find the coils and/or grill. In old fridges they’re on the back, but modern fridges put them down below, near the floor. Cut the power. Brush or vacuum the dust off. Turn it back on. Done! I just removed a mat of dust for the first time in far too long and suddenly my fridge is much quieter. Not just running less often: my guess is the fridge has a variable speed fan that was working way too hard. (Computer heat sinks have a similar issue). Wish I’d thought to do this sooner; this kind of maintenance is the sort of life skill I’m sadly missing. Now I’m off to go replace my furnace filter.

The main destination for our Nordic trip was Norway, driving up the coast from Bergen to Tromsø (flying over part of the 1800km). The west coast of Norway is incredibly rugged, full of mountains and fjords. Lots of beauty along the twisty slow roads. As you get further north it gets sparser and quieter and colder but never truly isolated. There were too many places we went to write them all up, here’s the highlights. My favourite place was Lofoten, a remote peninsula and islands that’s very rural, mostly old fishing villages. It’s a big tourist destination and well worth it, beautiful and quiet. We stayed at the fantastic Reine Rorbuer, a hotel made out of old fishing cabins with a very good restaurant attached. I could easily spend a relaxed week or two there.

Most of the cities we visited were nice. Bergen’s great (do the Norway in a Nutshell tour), Trondheim’s pleasant, Bodø is tiny, Andennes is sad, and Tromsø is surprisingly sophisticated for being so far north. Oslo’s a lovely big city, the Aker Brygge is an example of urban redevelopment working successfully and the Vigeland sculptures in Frogner Park are terrific. But while we enjoyed all these places none were really exciting, nothing so great I have to go back. I was particularly missing a good cultural museum that explains Norway’s history; being under Danish rule for so many centuries seems to have limited the historical imagination.

I can’t talk about Norway without remarking on how terribly expensive it is. We’re used to high European prices but between Norway’s robust oil economy and its insulation from the eurozone financial crisis it’s way out of hand. Crappy paper cup of coffee? $5. Pint of beer? $12. Main course at a decent restaurant? $50. Pretty much everything cost double what it does in San Francisco and some 50% more than most of Europe, making it hard to recommend Norway as a tourist destination right now.

This post seems negative. Truthfully, Norway was a bit boring and the cuisine was a bit disappointing. But it is a beautiful country, the people are friendly, it’s sophisticated and comfortable. I’d gladly live there (at least in the summer) and I’d be happy to go back as a tourist.

Some interesting stats on MacOS and gaming, from Valve’s Steam hardware survey. The population is self-selecting but somewhat representative of the gamer community. Steam has been available on the Mac since May 2010.

• 5% of Steam installs run MacOS.
• 48% of Macs are MacBookPros. 31% are iMacs.
• The most common GPU (24%) is the NVidia 9400M. Or not: the MacBook Pro contained both a 9400M and a 9600M GT and Valve may not report that accurately. This benchmark database says the 9400M is dog slow but the 9600M is somewhat reasonable for low-end games.
• 29% of Mac users have 512MB of VRAM or more (compare 86% of Windows users)
• 74% of Mac users have 4GB of RAM or more (compare 52% of Windows users)

What does this mean? 5% is a tiny minority of desktop computer gamers. And very few of those 5% have serious gamer graphics capabilities. Given that, it’s hard to see how it’s cost effective for mainstream games to support the Mac. No surprise, honestly, but there’s some data to back up the assumption.

I’m loving my new Mac but I’m coming to the conclusion MacOS is not suitable for an omnivorous gamer. Blizzard and Valve have both committed to the Mac but most PC games never make it over and the few that do are often low quality ports that come out months late. And graphics libraries are a mess. Of course DirectX is a non-starter, but even OpenGL and OpenCL support are years behind.

The Faroes was the most fascinating stop on our Nordic trip. It’s a tiny country of 50,000 people with its own distinct language and culture. Just a few islands between Iceland and Norway at 62°N, a Viking outpost. The terrain is mostly mountains and fjords that go straight into the sea with so little flat land that gardens are built on beaches and giant holes are gouged out of the hard rock to make room for buildings. It’s the most remote place I’ve ever travelled but it’s also firmly modern European with a workable (if nutty) airport and good roads. Really cool place and while there’s almost nothing to do I can totally recommend it for visitors. The main thing we did for our three days in the Faroes was drive around. The road system is great thanks to tunnels built between the islands in the 90s. The whole place is astonishingly beautiful. There’s no trees but everything is covered in rich green sod, including the roofs. The sparse settlement makes for plenty of natural landscape; when there is a village it’s just more picturesque.

The only town of any note is Tórshavn, you’ll want to stay there for a short visit. We had a fine stay at the Hotel Torshavn but if I were doing it again I’d stay at the Hotel Føroyar; a bit out of town but beautiful and also home to a terrific restaurant. There’s not much to do in Tórshavn but there’s almost no services elsewhere; we could barely find a place serving lunch in the rest of the country!

I’m not sure what sustains the Faroes as a country. Fishing is the big industry and there’s some hope of oil. Their wool and knitting are also famous (see the Faroe Knitting blog). There’s a little tourism, particularly campers and hikers but I imagine that only works for a couple of months a year. The joy is visiting some place remote and beautiful. After our quick visit I started thinking how pleasant it would be to spend a month with some good books and an Internet connection.

Awhile back I came up with a Windows hack to make Caps Lock search Google. I missed that ability on MacOS so I cobbled together a hack to do it. It’s crazy how in every OS that big useless key under your left hand is so hard to use!

1. Download my surl AppleScript and put it somewhere. It’s a very simple script which opens the clipboard contents as an URL or Google search. (First ever AppleScript program, comments welcome!). This script should work however you can convince MacOS to run it.
2. Install Alfred and buy the Powerpack. Alfred runs AppleScript with (almost) any hotkey. It seems silly to pay $20 for this feature alone, but Alfred’s awesome for other reasons too. (Alternatives for MacOS hotkeys are welcome!)
3. In Alfred, go to Features / Global Hotkeys and click the +. Add the hotkey of your choosing (but not CapsLock yet) and make the action be the surl AppleScript you downloaded earlier. Copy some text and press your hotkey: should open a browser window.
4. CapsLock is special. Install PCKeyboardHack, reboot, then open PCKeyboardHack in System Preferences. Choose “Change Caps Lock” and bind it to a new key code: I chose 105, which is F13 or PrintScreen. You can use Key Codes to find another option. (Note that 110 won’t work: Alfred doesn’t support it.)
5. Go back to Alfred. Edit your working Global Hotkey from step 3, changing the binding to your Caps Lock key. Voila!
6. (Optional). Go to System Preferences / Keyboard / Modifier Keys and bind Caps Lock to “No Action” in MacOS. I’m not sure this is necessary.

Alfred is such a powerful generic search doodad that it may make more sense to make Caps Lock open Alfred instead. But I really like having one button access to open URLs and Google searches, so I haven’t done that yet.

Ken and I started our Nordic trip with three days in Reykjavik, Iceland. It’s interesting; definitely part of Europe but feels somewhat American, open and friendly and culturally omnivorous. Worth a visit; a few days in Reykjavik if it’s convenient or a couple of weeks to explore the whole country.

Reykjavik is a small town but quite pleasant, particularly the cafe area in the middle of town. The Hotel Borg was quite nice and we had a fine, world class dinner at Sjavarkjallarinn. The new concert hall is a stunning piece of architecture. I could take or leave the Blue Lagoon, but it’s a nice way to wash off the jet-lag if you arrive too early for your hotel room to be available.

A real highlight for us was renting a plane and going on a sightseeing flight over the Þingvellir. Fun to put BISR and BIRK in the logbook, also a great way to get a sense of the land. The flying was easy; TF-GUS was a familiar 172SP and with an instructor handling the radios and navigation, we could just fly. I’d like to do more plane renting on trips; not nearly as complicated as I feared.

Lodi, CA gets a lot of shit, it’s a medium sized Central Valley town without much to distinguish it at first glance. But really it’s kind of a pleasant place, and it has a nice airport with a good mechanic. And the region is getting serious about producing quality wine, too.

Ken and I just had a nice overnight stay at Wine and Roses, a hotel / restaurant / spa on the northwest side of town. Very pleasant place. Our room was spacious and comfortable with a nice outdoor patio. Dinner was great, the place is often billed as the finest restaurant in Lodi. Pricy food but wine was inexpensive, and while the service was a bit irregular it was also professional and friendly. We didn’t try the spa, but the pool looked good.

One Lodi place we didn’t get to try is the Woodlon Diner, an unassuming little truck stop diner with a trained chef. Recommended by a friend, we tried to go but all eight tables were full. Note that Google Maps has the wrong location; it’s actually just north of the airport.

One of the challenges in understanding Islamic terrorism is how foreign it seems to us Americans. Norwegian terrorist Anders Breivik’s manifesto is interesting in that it gives Westerners religious and culturally inspired terrorism “on our side,” in plain English with familiar Christian and Western imagery. Breivik’s actions and rhetoric seem extremist to sensible Westerners; jihadist literature must seem equally extremist to sensible people in the Islamic world.

Breivik’s document is rational, well researched, coherent. At 1500 pages it rambles a bit, you can get a sense of the ground he’s covering via the table of contents (Scribd, Pastebin). He writes a history of the conflict between Islam and Christianity, a detailed rejection of various European modern philosophy, a curated collection of essays about the “threat” of Islam spreading in Europe. And then a detailed, apparently accurate manual about how to become a one man killing machine complete with armor, weapons, bombs, pharmaceutical enhancements. Tragically, all this research actually worked for Breivik and he was able to murder 70+ defenseless people on summer holiday.

See the emblem on Breivik’s germ warfare suit? That’s the Knights Templar, a 13th century Crusader military group that Breivik latched on to for his 21st century holy war. It’s a pretty florid bit of mythology. His manifesto is quite explicit about creating a modern crusade to defeat Islam: “Crusading is not just a right, but a duty according to Canon Law”, Breivik says, and after quoting the Bible for several pages concludes

Each Christian must now make their own personal decision on all of this. You can either choose to learn how to rise up in the power of your Lord and Saviour and learn how to become a true warrior in the Lord, or you can continue to keep your head in the sand and oppressor after oppressor keep beating you down.

Breivik here sounds exactly like a violent jihadi. Later in his manifesto he addresses this criticism:

By propagating and defending Christendom we simply mean that we want to halt the cultural Marxist/multiculturalist attacks and systematic deconstruction on our Christian cultures and the Church itself and to reverse the de-Christianisation of Europe. ... Western Europe has grown weak and decadent and will be completely annihilated culturally unless we succeed to implement a second European renaissance and reverse the damage done.

Reading this screed helped me better understand the violent Muslim equivalent, the rhetoric around a new caliphate and the establishment of pure Islamic states governed by Sharia. Those Middle Eastern concepts sounded foreign to me but Breivik’s similar ideas are comfortably English and Christian and European. And just as hideous.

A lot has been written about the emergence of the hashbang convention for web applications, URLs like https://twitter.com/#!/nelson that use # anchors to do fancy Javascript things. But most people I’ve talked to don’t know the real purpose of the #! convention, making Google’s life easier.

Googlebot is essentially blind to Javascript content. It (mostly) treats web pages as static text and refuses to execute any code on the page. That means that Google is incapable of indexing pages that rely on Javascript to render correctly. Not just fancy AJAX web apps, either. Google can’t index web pages that render pages client-side, or customize the display in Javascript, or do much of anything dynamically. My recent wind history project is essentially invisible to Google, for example.

Enter the hashbang convention, well documented by Google. The #! part is incidental; the spec is all about how site designers need to create a whole second set of statically generated HTML pages just for Googlebot, all behind URLs that include _escaped_fragment_ in their paths. There’s a 1:1 mapping between the #! URLs for humans and the _escaped_fragment_ URL for robots. My Twitter page at https://twitter.com/#!/nelson, for example, also exists at http://twitter.com?_escaped_fragment_=/nelson. A special page just for bots, which Google dutifully translates to #! URLs for humans. It’s all a big, gross, well-intentioned hack to work around a fundamental limitation in Google’s indexing technology.

I don’t quite understand why Google hasn’t tackled this core problem and figured out a way to run Javascript while indexing. It seems bad that Google can’t see the web the way users see it. Google has an amazing Javascript engine, of course, and more than enough skilled engineers to apply it to web indexing. It may be a scaling problem; running code for a page would probably increase their indexing workload by 10x to 100x. Or it may simply be that Google feels with its market power they can require every modern web site in the world to build a special version just for them. Hopefully I’m too pessimistic and Google is working on indexing Javscript content already.

The Eye-Fi guys have a new trick, uploading photos from a digital camera wirelessly to your iOS or Android device and from there, to the Internet. The core tech is an SD card with a wifi network magically shoehorned into it.

I had a lot of fun on my recent trip posting photos via Twitter. It was like sending postcards, some 92 images with bits of text day to day. Only problem is they’re crappy camphone images. Now I’m home and have a stack of 500 good photos taken with a real camera but without the enthusiasm of immediacy to edit and upload them. If only I had a way to upload photos from my real camera during my trip! Well I did, via the Eye-Fi, but I only found out about it when I got home.

The magic is direct mode. The Eye-Fi SD card not only can join wireless networks, it can form its own ad hoc wifi network solely to beam photos to other devices. The Android and iOS apps are smart enough to find this little private network and get your data. It’s a total hack but a clever one. Makes me wonder a bit what the point of Bluetooth ever was.

I should note that Eye-Fi has had the ability to upload photos to the Internet for a lot longer. The problem is with only your digital camera as a user interface, you had no good way to choose and edit the photos before uploading. The iPhone (or better, iPad) is a perfect solution. Also this route lets you upload via any 3G network your device has.

The Internet is at a dangerous inflection point. Facebook Connect is quickly creating a monopoly on identity. Sites are increasingly requiring Facebook logins now: Techcrunch comments and turntable.fm early access are two examples. And many more sites like TripAdvisor now promote Facebook over their own logins.

As a user the Facebook Connect experience is great. I see a familiar blue button, I click it, and I’m done. No creating an account, no coming up with a new username and password, no entering specific data. And it’s not just a login, many Facebook integrated sites give me a better experience with access to my Facebook social network. For site owners the advantage of Facebook connect is clear: good user experience, less code to manage, and access to Facebook data.

The problem is that Facebook is creating a monopoly. That’s a huge risk to every other company on the Internet. It’s bad for users too, we’re losing the ability to use pseudonyms online. And while Facebook’s technical execution is excellent the company has demonstrated over and over again its willigness to act unethically towards their users. We don’t want them controlling user identity.

There is a terrific technical alternative to Facebook Connect: OpenID. The tech works well and it’s open, letting users and companies choose their identity provider. But despite some four years’ headstart it’s never succeeded in being adopted widely like Facebook Connect has. And while I like competing login systems like Sign in with Twitter, identity is too important on the Internet to let any proprietary solution dominate. Our ecosystem needs a productive open standard. I still think OpenID is sufficient, but I’m in a dwindling minority.

I’m at my Reed College reunion, reconnecting with old friends. We’re all about 35 – 40. I’ve noticed there’s a real difference in my conversations depending on whether we’re connected via Facebook and Twitter or not. I post a lot online: folks who follow me already know I live in San Francisco, that I’m a pilot, that I was in Slovakia last year. It’s a great place to start.

Before the Internet we had casual social information sharing. Christmas letters, alumni class notes, outright gossip. But online social networks make it way easier and more fulfilling, you can learn as little or as much about someone as you want.

The best thing has been meeting someone I haven’t talked to in fifteen years, finding we have a lot in common, then making a point of finding them online while standing there with them. So we can stay in touch. Social media really works to keep a community together.

Apparently it’s news to almost every web developer out there, but in the real world people’s names have spaces in them. My name is “Nelson Minar”. It is not “Nelson_Minar” or “NelsonMinar” or “NelsonM” or “Nelson397” or any of the other nonsense I have to use to work with some website who’s decided to constrain names to some 1980s software-friendly character subset.

The hardest part of signing up for a new site these days is picking a unique user name. It’s annoying to have to remember different names. And it’s really obnoxious when my janked up UserName is also used as my display name. The right way to do logins right now on the Web is use email address as the login name and let the user choose their own display name which does not need to be unique. That’s not ideal (email addresses can change) but it works pretty well. If you absolutely have to not use email as the login name, please at least let my login name have a space in it.

While I’m delivering the news, here’s something for you ignorant American backwoods motherfuckers. Some people’s names have “special characters” in them. Like François Rabelais or Björk Guðmundsdóttir or 艾未未. It’s 2011; the only software that can’t handle Unicode properly is Perl. (As if you needed another reason not to use Perl.) Stop limiting your code; there are only two languages that can even be written in ASCII.

A big motivation for my Wind History map project was building a tech demo showing off dynamic SVG generation. I’m pretty proud of the implementation: it’s 100% static files, no active server code at all. The whole thing is two 3k HTML pages, 20k of custom Javascript, and 24mb of JSON data. All the hard work is done by Mike Bostock’s brilliant Polymaps and D3 Javascript libraries. Here’s some details.

I’m most proud of the map view, wind roses on top of a slippy map of OpenStreetMap tiles. I originally loaded pre-generated images on top of the base map but each PNG was some 2kb and that’s a lot of HTTP requests. Each station only needs 12 bytes of data, so I rewrote the site to render the diagrams in Javascript in the browser all driven by a single 200k JSON database. It’s a very cool technique and works great.

Naïvely drawing 2500 SVG diagrams on the map is slow and cluttered. Mike came to my rescue and wrote a tiler for me so that I only render SVG when the location is visible. I also replaced the wind roses with simple circles when zoomed out too far. It’d be better to declutter by only drawing the most important weather stations at high zoom levels; I fear many people never realize they can zoom in on the map.

The station page is equally important, a detailed wind diagram for every location. It’s pretty aggressively Javascripted, I have a single template HTML page that gets filled with JSON data. I really like this client-heavy development style, it’s very productive and scalable. The drawback is my site is largely invisible to search engines. I’ll probably go back and pre-generate station HTML for all the pages just to cater to Googlebot.

The station page is pretty simple: I had to cut a lot of planned features to meet my deadline. I’ve got code in development for filtering by month with animated transitions to show winds shifting over a year. Also plans for fancier histograms of wind speed, more airport data, etc. Lots to do here still, will be fun.

I’m really excited by the ability to build complex visualizations entirely in Javascript. Browsers are really capable software hosts now. Between SVG, Canvas, and WebGL we have three beautiful rendering options. It was particularly liberating to decide not to care about MSIE before 9.0; what a terrible tax Microsoft has placed on developers.

I’ve just launched a project I’ve been working on for awhile, windhistory.com. Check out the prevailing winds in California or see how strong the northeast winds are in Honololu. One goal of the project is to help pilots understand typical winds at their airports. It’s also a neat demo of what modern web development technology can do. I’m particularly proud that the wind rose diagrams in the map are rendered entirely client side in SVG from a single 200k data file. More tech details in a followup blog post.

Another Sony hack yielded a database of 1,000,000 plaintext passwords. Why does Sony have plaintext passwords? Because they’re idiots and deserve to suffer a civil lawsuit. But Sony’s negligence is security researchers’ gain: check out this analysis of the password haul. The most astonishing result:

Two thirds of people with accounts at both Sony and Gawker reused their passwords.

Passwords are a broken mechanism of authentication. They are weak, dangerous, and difficult for naïve users to use correctly. It’s time to end passwords.

The SF Chronicle has a heartbreaking story about how a married gay couple of 19 years may be split up by US immigration.

Starting June 13, Makk, 48, faces possible deportation if he remains in the country illegally when his current visa expires. If he leaves, he would not be readmitted, the couple would be all but permanently separated and Wells, who has severe health complications from AIDS, would be left without his spouse and sole caregiver.

“We’re at the end of our rope,” said Wells, 55. “Ever since we met, all we’ve tried to do is be together. The focal point of our lives, everything we’ve done, is just so we could be together.”

They are legally married (in Massachusetts) but thanks to a federal law passed and signed by the Democrats their marriage has no status for immigration. The Australian in the story isn’t even a second class citizen: he’s no kind of citizen at all.

Woah: Silk Road is an Internet narcotics market. Visit the market via Tor, pay in Bitcoins, and receive your LSD by mail! The Silk Road web site only exists as a Tor hidden service, but you can sometimes see it (non-anonymously) via this proxy. On the public Internet, here’s the founders’ feedback thread on Bitcoin’s forums.

I’m fascinated by how it works. I’m going to quote their “About” page in full, because I’m guessing it won’t be online much longer.

Silk Road is an anonymous marketplace where you can buy and sell without revealing who you are. We protect your identity through every step of the process, from connecting to this site, to purchasing your items, to finally receiving them.

Connecting to the site: Silk Road uses the Tor network to anonymize all traffic to and from the site so no one can find out who we are or who you are from your internet traffic. All traffic is encrypted and relayed through Tor nodes accross the globe, each one not knowing the origin or destination of the traffic. To learn more about the Tor network, visit torproject.org.

Purchasing your items: The currency used to buy stuff on Silk Road is called Bitcoin. Bitcoin uses encryption and a system of peer-to-peer double checking to create a completely digital currency. No personal information is associated with your bitcoins at all, making them ideal for anonymous transactions. Additionally, Silk Road employs a built-in tumbler that mixes all incoming bitcoins through a series of dummy transactions before they ever leave. Click here for instructions on how to get Bitcoins, or visit Bitcoin.org to learn more.

Delivery: Absolutely none of your personal information is ever required here. However, an address WILL be needed to accept delivery of any physical goods. Even so, it is stored encrypted, and is deleted as soon as your transaction is complete, so there is no record of it. And, because Silk Road is a Tor Hidden Service, the address remains encrypted within the Tor network when it is transfered.

For more details about improving your anonymity and making purchases, please read our buyers guide.

I’m particularly struck by their statement they provide a BitCoin laundering service. I’d noticed BitCoins didn’t seem particularly anonymous, here’s proof that someone with a vested interest in real payment anonymity agrees.

Twitter just announced a Twitter enhanced Firefox. It lets you type @nelson in the address bar to go to my Twitter account or type #twitter to search for the #twitter hashtag.

You can do this in Google Chrome too.

1. Click the Wrench icon
2. Choose Options
3. Click “Manage search engines”
4. Scroll to the bottom of the list to add entries
5. Add a search engine named “Twitter people”, Keyword “@”, and URL
   https://twitter.com/%s
6. Add a search engine named “Twitter hashtag”, Keyword “#”, and URL
   https://twitter.com/search?q=%23%s

Now type @ nelson in the Chrome address bar and voila! You go to my Twitter account. Or type # twitter to see tweets with the twitter hashtag. Note the space is necessary; not sure it’s possible to eliminate that.

The steps up above are awfully manual but work fine. There’s probably a way to automate this installation in Chrome; I know there’s a discovery protocol for websites to automatically add suggested searches.

It’s time to have a grownup talk about hot sauce. If you go to a generic American restaurant and ask for hot sauce, you will be brought Tabasco. And it’s a terrible choice, the worst kind of hot sauce.

Tabasco is a Louisiana hot sauce. Like its cajun brothers the primary flavour in Tabasco is vinegar. There’s chile, too, but it’s secondary. Vinegar sauces have their place, particularly in barbeque and cajun cooking. But Tabasco is the worst of the Louisana options: the vinegar is some industrial product with a one dimensional acid taste and the chile barely adds any flavour. If you want a Louisana style hot sauce try Crystal, Lousiana brand Hot Sauce, or Wintzell’s sauce (made by Panola). They all have much better flavour: interesting vinegar and decent chile.

Vinegar is a poor seasoning for most American food, particularly anything with eggs and any of the Mexican-derived food we get in California. American restaurants should generally offer Mexican hot sauce. Mexican sauces are made primarily with chiles and water. There may be a little vinegar and spice for flavour but the acidity doesn’t dominate. There’s a zillion options for Mexican hot sauce but a few good common options are Tapatío, Melinda’s, and Cholula.

I also want to give a shout out to Sriracha, the modern, Vietnamese-American hot sauce whose thickness, sweetness, and intense garlic flavour give an interesting third option. I find its seasoning too strong to use as a general condiment but it’s a delicious alternative to ketchup or mustard for flavouring sandwiches.

Summary: you probably want Mexican hot sauce, not a vinegar sauce. If you want Louisana hot sauce, avoid Tabasco.

The US military is on the Internet march. Obama said of two new military appointments “Between them they bring deep experience in virtually every domain ... Land, air, space, sea and cyber.” Also today the WSJ reports “The Pentagon has concluded that computer sabotage coming from another country can constitute an act of war, a finding that for the first time opens the door for the U.S. to respond using traditional military force.”

I used to think “cyberwarfare” was an inflated threat. But in the past couple of years my thinking has reversed. Stuxnet was a big deal, presumably a deliberate act of cyberwarfare by the US and Israel against Iran. China’s hacking of Google was also a big deal; aimed at individual activists, not nations, but still important. The recent attacks against RSA SecurID and now Lockheed Martin are troubling. We’re beyond script kiddies stealing some Warcraft accounts, this is focussed espionage against the US military.

One of the problems with cyberwarfare is it’s not clear how to apply international law. What is “proportionate response” to a network break-in that disables a radar installation? How do you even identify an attacker when the attack was a virus that was planted six months ago on USB sticks? These statements says the Pentagon is moving to take these questions seriously. It’s about time.

There’s a scary trend happening in computer hacking; it looks like the bad guys are escalating from attacking random weak sites to targeted attacks against companies that provide security infrastructure.

Back in March RSA disclosed that they had been compromised and their SecurID system had been targeted. SecurID is a product that’s widely used by companies for login security. RSA never came clean on exactly what happened, but they did admit that the incident could “reduce the effectiveness” of SecurID. Now the other shoe has dropped with the NYT reporting that the recent Lockheed Martin compromise appears to be related to the RSA breach.

In May LastPass warned users they had a “network traffic anomaly” that they could not explain. LastPass is a password management tool that stores their customers’ passwords. The company’s disclosure was quite comprehensive but vague because they don’t really know what was taken. I still trust them, but it makes me nervous.

There’s been a lot of other recent high profile security incidents: the Gawker fiasco, the Google incident with China, the Sony outage. But the attacks on RSA and LastPass feel different. Those companies are not easy targets, they are sophisticated security companies. And they make security systems, a very valuable asset if you’re in the business of attacking protected targets.

The buzzword for this kind of attack is Advanced Persistent Threat. The reassuring thing is these attacks are focussed and purposeful, not random vandalism. Personally I fear random Eastern European password hackers much more than I fear the Chinese government or a US / Israel cooperative. But it’s a bad thing if our industry’s most sophisticated security companies are no more effective at protecting their customers than a crappy PHP site.

After reading up on Bitcoin I ran the software for a day and mined some coins. It’s a fascinating decentralized system but has some significant practical flaws.

Cash flow isn’t private. The protocol requires all transactions are public: you can see them all on Block Explorer. Here’s the transfer of 0.02 from the Bitcoin faucet to my account. Here’s 50 bitcoins I helped mine along with some other random transactions in that block. Here’s my miner’s wages from slush’s pool. You could do some fascinating visualizations of the flow of money. It’s hard to remain anonymous: you can have multiple identities, but because every single coin is traceable if even one identity is known the rest can be discovered via traffic analysis. The Bitcoin wiki recommends laundering your money through a third party.

Latency is a problem. I received 4 payments last night; each took about 5 minutes to show up in my wallet. I made a payment of 0.02 today that showed up in about a minute, but I was required to pay an extra 0.01 transaction fee. Communication latency seems to be a result of the peer to peer network and presumably could be improved. But there’s design latency, too. You can’t really trust a payment is valid until it’s been confirmed, which boils down to waiting for about an hour’s worth of other transactions to be done after yours.

Scalability looks like a serious problem. The core innovation of Bitcoin is the block chain, a signed history of all transactions. Bitcoins are validated by tracing the history of a bitcoin all the way back to block 0, the Genesis Block from January 2009. I don’t see any process for short-circuiting this validation. It’s like every time you got a dollar bill, you had to talk to everyone whose hands that dollar ever passed through to work its path back to the bank that printed it. Also the current client needs a copy of every transaction ever. That takes a couple of hours to download but presumably could be optimized so you only keep the blocks you personally care about.

Coins are brittle. My money entirely exists in a file named wallet.dat on my hard drive. If I lose that file or it gets corrupted or it gets stolen, I lose all my money. That’s no worse than real cash under your mattress but it’s significantly worse than other online payment mechanisms like credit cards.

I don’t mean to hate on Bitcoin, it’s an impressive cryptocash system. And it actually functions: at market rates there’s some $35 million in Bitcoins in circulation. That valuation may not last for economics reasons, but the crypto and decentralized system design is pretty sound. Still it’s got some rough edges, interesting to see where they are.

Digital currency Bitcoin is the subject of this week’s hype. And while I agree with Adam Cohen’s analysis that it’s likely a scam, I don’t think it’s inherently a scam and it’s an interesting implementation of cryptoanarchy.

The basic idea of Bitcoin is you have a file on your hardrive that, through the magic of cryptography, is unique and unforgeable and says essentially “Nelson owns 5 bitcoins”. Bitcoins can be created by users out of thin air but via more math magic there will only be 21 million bitcoins ever. Bitcoins can be traded and verified and double spending is preventable (math, is there nothing it can’t do?). As long as people agree that Bitcoins can be exchanged for dollars then we have a new currency. (At current rates, a Bitcoin is worth about 7 USD).

The economics and politics of the Bitcoin idea are anarchic. The primary advantage of cryptocash is it exists outside the control of any government. Untraceable, unregulated currency, I wonder who benefits most from that? The history of previous electronic currencies like e-gold suggests the main use of a cryptocurrency will be fraud. It didn’t have to be that way. Back in the mid 90s when Internet commerce was just taking off, e-cash had a real chance to be an alternative to credit card payments. If things had gone differently, maybe we’d be buying our socks on Amazon with DigiCash instead of Visa.

But the credit card networks are expensive, scalping roughly 2% of every transaction. There’s legitimate value in having a cash-like system for the Internet. And the Bitcoin technology seems to be an honest implementation. I didn’t spend the time to understand the protocols, but at its core it’s a fully decentralized system with peers exchanging signed messages about the flow of money. I fully believe the math can be made to work (although Bitcoin’s protocols no doubt have problems).

The implementation has some drawbacks. The default Bitcoin client is not suitable for casual users. And there’s a lot of latency. Two hours later I’m still waiting for the data to synchronize. I claimed my free 0.02 bitcoins from the Bitcoin Faucet but it took over 10 minutes for the transaction to show up on BlockExplorer. And the money still haven’t arrived.

One fascinating bit of the tech is Bitcoin mining. It takes weeks or months for a single system to generate Bitcoins, so long the mining option is being hidden. Instead people pool their efforts and use GPUs to do the math. A GPU is some 200 times faster at the crypto than a CPU, using OpenCL code like poclbm.

I expect Bitcoin itself will be forgotten in five years or else consigned to being a currency largely used for fraudulent transactions. (Here’s an interesting bet: $1000 in Bitcoins against $1000 in gold, to be paid in 5 years.) But I hope the idea of cryptocash continues to be developed and improved; it’d be nice to have a cash equivalent for the Internet that wasn’t PayPal.

It’s finally happened: someone’s made a World of Warcraft clone that’s a credible competitor to Blizzard’s megabusiness. A lot of companies have tried the MMO market and failed, most notably Warhammer Online, but Rift is a success. It’s as good as WoW. And that’s a disappointment.

Rift is a fantasy MMO game by Trion Worlds, a serious new company that’s raised $100M in venture capital. It’s been pretty successful since their launch two months ago. The buzz is good, a lot of people are enjoying it, and while it’s a bit early to tell how big the game will be I think they’ve got enough momentum to recoup their investment.

I just finished a week free trial, levelling up a Cleric and a Rogue to level 13. It’s fun. The graphics, networking, and user interface are all good. Not quite as solid as WoW but close enough. The game offers two big innovations. Rift Events are big impromptu battles against bad guys that pop up, basically Warhammer’s public quests but with a dynamic element that means there’s more likely to be people nearby to play with. And the flexible class system (souls) allows more choice in play style, although there’s already a forming consensus of useful builds.

If I were picking a new MMO solely on the basis of gameplay I’d probably choose Rift over WoW. Unfortunately, Rift is so like WoW that there’s nothing to distinguish it. Gameplay still boils down to pressing a button once a second to activate one of your forty abilities in the right order. Group play still is the basic tank/healer/DPS trinity. It’s yet another bucket of gaming from the DikuMUD well and it’s getting stale. I question whether the world really needs a WoW clone when WoW is still good enough.

$100M is a lot of investment and congratulations to Trion for producing an excellent product. But it’s so expensive to make an MMO now, it’s hard to see how a trully innovative and risky game can get funded. We need more odd innovative designs like Eve Online. I’ll be curious to see how Glitch evolves.

With the 2011 floods of the Mississippi river there’s real concern that the Mississippi might jump its channel at the Old River Control Structure and take a shorter path through the Atchafalaya basin, changing the structure and economy of half of Louisiana. In the Internet discussion an astonishingly beautiful set of maps from 1944 keeps turning up. The maps are from the “Geological Investigation of the Alluvial Valley of the Lower Mississippi River”, a report by Harold Fisk (Geology Professor at LSU) for the Army Corps of Engineers. They did a huge amount of research on the history of the river course, tracing 20 “stages” of the river course going back some 2000 years. Accompanying 170 pages of sometimes dry text and tables are these fifteen beautiful maps, a tangle of ropes showing the historical Mississippi. Each colour (or stipple) is a former river path and with some squinting you can work out the specific flow of the river in, say, 1765. The real message is the aggregate of all those historical courses, the unmanageable chaos of river meanders.

The report and maps are available for download from the Army but the files are big and pretty unwieldy. I’ve resized the rectified TIFFs to 25% and uploaded them to Flickr: you can browse them in this Flickr set. The one single image I recommend is Plate 22 Sheet 13: it shows the adjacency of the Mississippi and the Atchafalaya, connected by the Old River where the current water control system is threatened. For a different perspective, check out all the maps assembled into one long picture. It’d be neat to see a zoomable version of this to full resolution.

Please try my historical map of the Mississippi River courses!

I was so taken by the 1944 Army Corps of Engineers maps of the Mississippi River that I made my own zoomable map. It works well in Google Chrome, reasonably well in most browsers, and not at all in older versions of MSIE. (The tiles aren’t perfect; I did this in about three hours’ work.)

The map is truly a beautiful bit of geologic history. For more info on it, see my previous blog post. So many amazing swirls and details in the river’s course. Rendering it as a slippy map makes it easy to see the map in great detail, for instance the Old River Control Structure, a site threatened by the floodwaters of 2011. (Interestingly, the modern channel was built on relatively dry land.) The opacity slider (or text box) in the upper right lets you look through the Fisk map to a contemporary Google map. Check out this flood plain, for example. The satellite view contains echoes of the various old meanders, too, like these curved fields.

Credits:

• Army Corps of Engineers for the source map
• Google for the base maps
• GDAL for the geographic software to make the tiles (See also my secret work blog notes)
• Polymaps for the Javascript library

I like maps. On my recent Texas trip I had my iPad with a good GPS and four different map applications. You can see two sets of screenshots from the various maps. Long story short, I’m impressed with You Need A Map (and it’s free!). The four apps I tried were MotionX, a GPS tracker with terrain maps from OpenCycleMap; OffMaps with their CloudMade/OpenStreetMap maps; You Need A Map with a variety of raster and vector sources; and ForeFlight, an aviation specific map with VFR sectional charts. All of these apps support cached offline map access, I store some 10 gigs of map data on the iPad. The two locations I compared are near the Guadalupe Pass in Texas and near the Cajon Pass in LA, California.

The main thing I learned is how different maps can be. MotionX’s terrain layer is designed for hiking with very detailed countours. But the shading isn’t very good and it’s easy to lose the big picture zoomed out as much as I do in a plane. OffMaps spare road maps have served me well driving around Europe but don’t convey much terrain. Oddly they do indicate airport positions very well. ForeFlight is showing AeroNav’s excellent VFR charts; great for pilots, not so much for anyone else. The real generalist is You Need A Map, whose combination of a terrain raster map with vector layers gives some really impressive maps. I’m particularly impressed with their dynamic terrain shading.

Configurability is another interesting difference. You Need A Map is amazing: you have a choice of some 20 map layers to composite with different presets for emphasizing roads, natural features, points of interest, etc. It’s the only iPad app I’ve seen to extensively use vector map data rendered in real time. MotionX, OffMaps, and ForeFlight all offer your choice of a handful of different raster layers but basically you take what they offer. Which can be perfect if, say, you’re flying an airplane and using ForeFlight.

One final difference is extra features. OffMaps and You Need A Map basically display maps and nothing else. MotionX is really a very fine GPS track recorder; the maps are just gravy. And ForeFlight is a full flight planning system.

My conclusion from this experiment is that for flying trips, in addition to ForeFlight I’ll be using You Need A Map. OffMaps will stay with me on my iPhone, its maps are really good for road navigation. I don’t have much use for the MotionX maps but that’d change entirely if I were a hiker.

One of Google’s big announcements this week was the launch of Chrome Angry Birds, a port of the hugely popular mobile game to Google’s browser. But calling it “Chrome Angry Birds” is missing the point because what’s really interesting is that it’s a real-time multimedia cross-platform HTML 5 app. It runs fine in MSIE 9 and Firefox on Windows (sound and save games included) and I’ve heard reports it works in Safari on Macs, too. And because the game is based on open browser technologies, we can easily pull it apart and see how it’s built just like we’ve been pulling apart web pages since 1993 via the magic of “view source”.

The key file is appcache.nocache.manifest: it contains a list of all the other stuff in the app. There’s a whole lot of files: sound, art, levels. Here’s the image sprite sheet for birds and pigs, here’s the slingshot sound, here’s the data file for level 1. The only thing that’s opaque is the code itself, 390k of GWT compiled Javascript. People often call this style of code “obfuscated” but the compilation is as much about making it run faster as it is hiding the details.

The part that impresses me the most is the sound: that’s been a weak point in browsers. Over on Metafilter someone pointed out that it was loading a small Flash shim, what appears to be gwt-voices. So still not a 100% Flash-free app, but very very close. The caching is quite sophisticated too.

Last week I finished a 10 day flying trip to Houston and back. I grew up in Houston but hadn’t been there in years. Part of my purpose was reconnecting with old friends and part was having a flying adventure on my own. 25 hours and 3000nm over some six days, worked out great. Here’s some photos from the trip. I followed a southern route. Overnights were in Palm Springs CA, El Paso TX, Houston TX, San Angelo TX, and Palm Springs CA again. Good weather most of the way, necessary since I’m not instrument rated yet. Lots of light chop which gets tiresome. Also some alarming mountain wave over Palmdale the first day: 40kt tailwind turned into dramatic up and downs, at one point I was at my best climb speed and still losing 500 feet per minute. Altitude is your friend! I also discovered after six hours of flying I’m pretty tired and not as sharp on the landing as I should be. Nothing unsafe, but I have a better of idea of my limits.

I had some good food. Bad weather made me drive to Austin but on the way I enjoyed Black’s Barbecue in Lockhart with excellent brisket and some of the best smoked sausage I’ve ever had. Then that night I had phenomenal nouveau sushi at Uchi in Austin. Houston Tex-Mex wasn’t as good as I remembered, but El Real Tex-Mex was quite good. Danton’s in Houston was good for fancy-pants Creole and Ragin Cajun was good down home bayou cooking, including a bucket of crawdads.

The best thing of the trip was reconnecting with old friends, particularly high school buddies. Amazing to reconnect with people for the first time in twenty years and feel perfectly comfortable. They all turned out to be interesting and nice people, too! I also had a good time meeting some new people, including a hilarious gay night out in San Angelo, TX with some friends-of-friends that turned a dreaded West Texas overnight into a real hoot.

I’ve moved my linkblog to Pinboard. If you follow it already you’ll want to update your RSS subscription. If you don’t follow it, you should! My linkblog has better content than this text blog. Links are on the sidebar at left but best enjoyed via your RSS reader of choice.

Many thanks to the Delicious and Yahoo crew; they served me well for hosting for four years. I decided not to follow along with the AVOS purchase, not for any particular reason but because it seemed uncertain. A big thank you to Joshua and Delicious for having a fantastic, easy export of all my data. That’s an important user right more sites should support and more users should demand.

Pinboard works great and is simple. I like that I’m a (modestly) paying customer. I particularly appreciate that the search function works so well: Yahoo never got that right. I was also impressed by Maciej’s postmortem when the site got crushed after the Delicious shutdown rumour. He’s a smart engineer. All the cool kids seem to be on Pinboard now, I’m in good company.

Awhile back I wrote about the religious debate about operating airplane engines. I finally got a chance to run lean of peak on my recent Texas trip. Compared to rich of peak, LOP was 20% more fuel efficient, 5% slower, and about 20° cooler. But I’m still running ROP.

The plane is only now capable of running lean of peak thanks to our engine monitor, fuel totalizer, and GAMIjectors. The stock injectors had one cylinder peaking 1.0 to 1.5gph sooner than others; now our “GAMI spread” is 0.4gph, a win whether we ever run lean of peak or not.

On my recent flight I was at 9500′, 20.5″ manifold pressure, 2400 RPM. Flying at my typical 50 ROP I was getting 140kts TAS, burning 10.3gph, with EGTs at 1357/1378/1352/1324 and CHTs at 303/332/337/290. Leaning out to 40 LOP I was getting 132kts TAS, burning 8.3GPH, with EGTs at 1401/1441/1416/1352 and CHTs at 281/316/311/271. So the difference there is running lean of peak I saved 2gph, ran my engine 20°C cooler, and gave up about 8kts to do it.

Saving gas but going slower: is LOP really a win? Cessna didn’t publish performance tables for lean of peak, but they did publish tables for saving gas at the recommended 0.5gph ROP. At 10,000′ Cessna says full power gives 145kts TAS at 9.2gph. If you want to save 2 gph they have you dial way back on power to get 126kts TAS at 7.1gph, for a 19kt slowdown. So roughly speaking, LOP saves me just as much gas as going slow ROP, but makes me 11kts faster.

Despite the good results I still flew over the hard, unforgiving mountains of the Southwest running ROP. The engine feels wrong in the transition to LOP, you can feel the sluggishness. I’ve also had various salt-of-the-earth mechanics all warn me against running LOP, and while their reasoning doesn’t make sense it still makes me nervous. And of course LOP really does make you slower and with four hours of West Texas ahead of you, you want every knot. The cooler operation seems like a real benefit though.

(A caveat: those EGT numbers are a bit wonky. The actual measured peaks with some experimenting were 1462/1494/1466/1410 at 8.8/9.1/8.9/9.2 gph, so it’s possible I was running more like 90 ROP than 50 ROP and could have saved a bit of gas while ROP. There’s some lag in the EGT response time, I was doing stuff too quickly.)

One of the amazing things about visiting Texas is how every single person I’ve seen has offered me a place to stay. Whether an old friend I hadn’t seen in 20 years or someone I barely knew, pretty much everyone offered me a guest room in their house. Such lovely hospitality! I think that offer is much less common in San Francisco; homes are too small to have a guest room and people generally seem too busy to casually offer that kind of sharing. That’s too bad.

I recently went to my doctor for a regular checkup. he ordered some labwork. Nothing special; basic blood and urine tests to see that I’m doing OK. Here’s the thing: the billing is entirely insane. I have individual health insurance from Anthem, and the insurance adjustments are completely incomprehensible. Here’s an itemized bill.

Billed	Savings	My co-pay	Insurance pays
44.00	36.51	2.25	5.24
51.00	40.81	3.06	7.13
99.00	86.08	3.88	9.04
68.64	63.06	1.67	3.91
61.36	56.37	1.50	3.49
78.00	68.64	2.81	6.55
$402.00	$351.47	$15.17	$35.36

My health care provider charged me $400 for the tests. They only got paid $50, an 88% discount, because that’s what Athem has negotiated. I paid $15 of that $50. Huh? How do these numbers make sense? In no way is this free market rational pricing.

When my mother died, in 2000, she went to the hospital with urgent problems. With no insurance. They operated on her for a few hours, she spent a day in intensive care, and she died. It was pretty fucking awful. It also cost some $50,000, billable to her estate. The hospital gave us a 35% discount for paying the bill in cash.

I’m fortunate to be able to afford to buy my own health insurance and healthcare. My family was fortunate that my mother could get the care she needed and the medical bills didn’t bankrupt us. In neither case does the pricing make sense. It’s particularly bizarre that the primary value of insurance seems to be pre-negotiated lower prices for healthcare. Instead of, you know, insurance.

I like my iPhone tracking where I’ve been; my only complaint with Apple is they didn’t disclose this useful feature and give me access. But it’s not much hacking to get at the data, so I made a map. Red is cell phone towers, blue is wifi nodes. The map looks correct to me; I did a lot of small plane trips in the last year. My guess is the iPhone is keeping a cache for geolocation. Your iPhone knows where you are by looking up cell phone tower IDs and WiFi MACs; I think the recently discovered file is just a history of those lookups. It’s not a very useful tracking file; it doesn’t record how often you’ve been near those points. It does keep timestamps (last seen?) and of course even the cache history is pretty personal data.

Technical details on my secret work blog. The main thing is two tables called CellLocation and WifiLocation which map tower ID or WiFi MAC to a latitude and longitude. The table’s big, some 32,000 cell towers and 170,000 wifi nodes. I wrote some quick code to convert the data into GeoJSON and map it with Polymaps. I’d link the zoomable map, but it’s personal data and the code isn’t optimized.

Our culture has not yet figured out how to handle privacy in the age of digital data. We’ve had a couple of recent reminders that cell phones are people tracking devices: iPhones are recording a history of your location and so are cell phone companies. Both law enforcement and any person with access to your phone or computer can find out everywhere you’ve been for the last year.

The problem is this lack of location privacy surprises people. Telco tracking has been happening for several years now. Here’s the thing: I want a record of everywhere I’ve been. I want access to that data. A big problem with the telco tracking is I don’t have access to my own location data; only AT&T and the cops do. The big problem with the new iPhone tracker isn’t that the phone is tracking my position, it’s that I don’t have easy access to that track.

A basic law of the digital age is that any information that exists is going to be collected and archived. Once some fact is available, like the location of my cell phone, that data is going to end up in a historical database. Either by design (telco towers) or by accident (iPhone). It’s hopeless to stop personal data from being recorded. Instead we need to develop good laws about authorized access to that data. And we need to develop good tools for people to access and use their own data. There’s no value in hiding our head in the sand and pretend that iPhones recording location data is some anomaly. Tracking personal data is the inevitable future and it should be made a good and useful thing.

PS: American cell phone companies also keep records of all your SMS messages and provide them to the police when subpoenaed. But you can’t get a copy of your own SMS history from the telco.

I’ve been on a productivity kick lately, trying to figure out how I manage so few hours of hacking on my projects and so many hours wasting time web surfing. I mentioned a failed experiment offloading web surfing to the iPad. The other thing I’ve been doing is running time tracking software, the web based RescueTime and the Windows app TimeSnapper. They’re both useful.

Both programs watch what what software you use on your computer during the day and then give you reports of your usage classifying time as productive or non-productive. TimeSnapper’s special feature is recording screenshots every 10 seconds and letting you review your day. RescueTime’s special feature is community-based classification so that it already knows that github.com is productive time whereas 4chan.com is not. Thanks to these tools I now know (for instance) that I spend nearly an hour a day on Metafilter. Ouch!

The virtue and the failing of these tools is they are mostly passive recorders; they are great for monitoring behaviour, not modifying it. A lot of productivity guides suggest some sort of action to say “I am working now”; punching into a time clock, starting a 30 minute timer, etc. That works for me at times but it also makes hacking feel like more of a chore than I would like. (I envy Ben Franklin.)

One thing I dislike is the monitors often report bad news. TimeSnapper’s primary report is a “Productivity Grade” where some days I rank 20%, 30%: talk about a failure! RescueTime’s report shows the average user is “0.52 efficient”, not a very auspicious goal. It’s just a language problem but it seems important.

Another drawback of these tools is privacy: they record every program and website you use. There’s ways to mitigate disclosure: temporarily switch monitoring off, encrypt TimeSnapper screenshots, tell RescueTime not to record specific websites. RescueTime has found a market as a tool to let managers snoop on their employees (with disclosure). Useful, but also a bit creepy.

My free month of TimeSnapper is up and I won’t be paying $25 for it. It’s a good tool and the screenshots are nifty and occasionally useful. But it takes a lot of disk space and the reports aren’t so useful. I’m going to invest a bit more time (and $6/month) into RescueTime, I think I have more to learn from it.

Google Maps’ tiling has become a de-facto standard for Internet maps. You can see how it works in this interactive demo. Basically every map tile is named by three numbers: the zoom level and an X,Y coordinate. For example, most of San Francisco is contained in tile z=11 327,791. Oregon is in z=5 5,11. Google, Bing, and many open source tiles use the same tile cuts, albeit with different names. The maps are based on the spherical mercator projection EPSG:3857.

How big are the tiles, what scale is the map? The best explanation I’ve seen is Microsoft’s. Tiles are 256x256 pixels (by convention). At Z=0 a single tile covers the whole earth (except the poles). Each progressive zoom level is 1/2 the scale. Tiles are always square, but get smaller the further you are from the equator. Tiles “naturally” get less wide (in meters) because the lines of longitude narrow at the poles. The conformal projection we use for tiles also causes the tiles to get less tall (in meters) so that each tile remains square. I’d love to see a 3d globe with this tile mesh draped on it!

Here’s a table of map scales for various zoom levels, see also my spreadsheet. “Feature” describes something that will generally fit in a single 256x256 tile. “One pixel” is miles or feet per pixel. All values are calculated at 37N, the latitude of San Francisco. Tiles at the equator are roughly 20% bigger on a side than in San Francisco; at 60N they are roughly 40% smaller.

Zoom	Feature	One pixel	Scale (100dpi)
0	Earth	78 mi	1:500,000,000
2	Continent	19 mi	1:120,000,000
4	India	5 mi	1:30,000,000
6	Utah	1 mi	1:7,700,000
8	Urban sprawl	1600 ft	1:1,900,000
10	City	400 ft	1:480,000
12	Town	100 ft	1:120,000
14	Neighbourhood	25 ft	1:30,000
16		6 ft	1:7500
18	Block	2 ft	1:1900
20	House	5 in	1:470
22	Car	1 in	1:120

I’m particularly interested in elevation and terrain tiles. SRTM has global coverage at roughly 90m or z=11. NED 1/3 is US coverage at roughly 10m or z=14. Google’s terrain layer provides tiles globally to z=15, but it looks pretty averaged in remote locations.

Caveat: I’ve found a lot of conflicting tables and explanations of map tile scales: it may be I’ve got something wrong. See also OSM and this discussion. I also ran into an interesting spreadsheet, the time and storage requirements for a tileset (from Geosolutions). The globe to z=12 takes about 3 gigs of disk and a day to render.

I saw an interesting talk by Sha Hwang and Zain Memon of Trulia, the real estate data company. They described a clever way to let a user manipulate a map visualization interactively in their client. I don’t know of a demo to link yet, so please bear with the prose description.

Say you want to show a heat map of average home price by zip code. The traditional way would be to render a raster image (or tile set) on the server with each zip code coloured by price. Very efficient for the client, but you’re stuck with a single visualization. Or you can overlay a bunch of vector polygons on top of a raster map: efficient, manipulable, but you are drawing complex shapes in the browser and it doesn’t scale well.

A third way they described is rendering the zipcodes into a raster map, then repainting in the client. The basemap tiles don’t look like much, a harlequin of colours. Yellow for 94114, red for 94117, black for 94121, etc. Then on the client, Javascript goes in and applies a colour map: turn all 94114 pixels orange (medium expensive), 94117 pixels red (very expensive), etc. The advantage is the colouring happens entirely in the client. The user can easily interact and try out different visualizations (say, convolving with crime rate) without going back to the server for new image tiles.

Sha had a great way to describe this idea: in their image tiles, the pixel colours do not represent the data values like price or crime rate. Instead they represent the data keys, the zip code, and then they join that key to various values in the client dynamically. It’s a nice generalization of an idea I’ve seen various folks play around with (myself included).

I’m not aware of an online map yet using this technique but Sha said he thought the technique was fast enough to be practical for consumer use. I belive the easiest way to implement it would be using canvas. Unfortunately that’s a poor match to the SVG implementations I’ve been favouring lately, but no doubt there’s a way to make it work by embedding canvas elements inside SVG or the like.

I did an experiment using my iPad for my two hours of daily reading of the Internet. It didn’t go great, mostly because of UI limitations. But it’s definitely workable.

I’ve got a daily list of things I read online. RSS feeds in Google Reader, a few community sites like Metafilter and the AOPA forums, Twitter and Facebook, and some random image blogs (notably Chan4Chan NSFW). Doing a full read takes an hour or two and I consider it time well spent. Reading those same sites a second, third, fifth time the same day still takes about an hour and becomes procrastination. So I thought I’d shift that work to my iPad, leave my desktop for real work.

The good things.. It’s nice having the iPad in your lap in a comfy chair. The touch scrolling and navigation is comfortable for casual reading. The display is beautiful; bright, high enough resolution, and generally fast enough except for the monster chan4chan page.

The main bad thing is the single-tasking user interface of iOS. I’m surprised how much multitasking I do when reading the net. Metafilter opens up into a forest of 20 tabs, sometimes several new windows as I explore sidelines. Google Reader URLs get copied and pasted into Delicious for linkblogging. Forum responses get written, sometimes over 30 minutes as I research related things. Videos are watched in the background. Task switching is possible on the iPad but it’s awkward. App switching is clumsy and cut and paste is a real chore.

In general I like the idea of single-tasking interface. But my random daily reading is something that seems best suited to a multitasking windowed UI. What I really need is to be able to write code on my iPad. Only that too is multitasking: an editor, the software itself, docs, version control.

The iPad is best at single task focus, like games or videos or books. (Related: I have a Kindle for sale.) It also seems to work better with new custom apps like Reeder or Twitter; traditional web sites like Metafilter don’t quite fit the iPad as well.

The La Cross BC-9009 Battery Charger is good hardware. Nothing fancy, just charges AA and AAA batteries. But it’s really good at charging. You can choose low current if you’re not in a hurry and want to extend battery life or use high current for a quick charge. It has a mode where it discharges the battery first as well as a reconditioning mode that cycles the battery a few times. It’s $46 on Amazon.

We use Eneloop NiMH batteries; they seem well rated. I’d love to use lithium batteries but there are few options for rechargeables; NiMH is more of a developed market.

I used to never bother with rechargeable batteries but we go through enough batteries in the airplane it’s worth it. A hidden benefit is you can replace the batteries in your device whenever you feel like it; no need to risk running flat because you’re trying to get every last milliamp out of your disposables.

Flying from San Francisco to Las Vegas or other points east is tricky; between mountains and restricted airspace there are very few ways to go. The usual VFR route is something like KSQL KWJF KDAG KLAS, 436nm at 9500′+. That carries you out of the mountains near Palmdale and south of all the restricted space in California and Nevada. It gets boring; I recently learned of another route via a desert town, through the Trona Gap.

The route is roughly KSQL L05 ODGEE SERUE L72 KLAS, a series of wiggles through a narrow corridor between R-2505 and R-2524. That’s 390nm, saving 46nm or about 10% of the trip. The mountains are awfully high west of L05, I’d want at least 10,500′ crossing there or else go south towards Bakersfield first to come up the river valley (adds 20nm). Either way it’s more time over rough terrain than the Palmdale route, but at least it’s something different.

I’d only try threading this needle with flight following and a trustworthy GPS. R-2505 and R-2524 are active at all altitudes all the time. R-2506 is only Surface to 6000′ during the daytime so you can shave a few miles off the route above. Apparently it’s not uncommon to be cleared to enter R-2505 above 10,000′ which could save some 17nm. On the other hand the Edwards AFB info says they sometimes close the Trona gap entirely via NOTAM. Plan accordingly.

Another alternate route I’ve flown in the area is via Yosemite, the Tioga Pass, and the Owens Valley. Something like KSQL E45 O24 KBIH O26 L06 KLAS (394nm). It’s a fascinating trip through a part of California I’d never seen before and KBIH is a nice stop for fuel and excellent Thai food. Not the simplest route: the Sierra crossing is tricky and wind is a concern both in the mountains and in the valley.

Disclaimer: I’ve not flown most of these routes and have not confirmed altitudes, airspace, etc. The terrain and weather in this area can easily be dangerous. Don’t take my word for these route ideas; you’re responsible for your own flight planning.

A big part of learning to fly on instruments is procedures: quickly reading a chart and making the appropriate turns, radio changes, power changes, etc in the 5–10 minutes you’re flying an approach. It’s important to train in a real plane but an instrument student can learn a lot with a simulator, even a basic PC game.

Simulator: For 20+ years Microsoft Flight Simulator was the gold standard, but Microsoft has abandoned the product and recent iterations like FSX didn’t work very well. X-Plane is the new hotness and works on PCs, Macs, and even iOS devices. It’s pretty and has neat aerodynamic simulations but what really makes it work for IR training is that it simulates the panel instruments pretty well. CDI needles move smoothly, VORs warble just like the real thing, and it has a good simulation of clouds and the moment of breaking out of them. You can easily customize your plane’s panel, too. X-Plane is pleasant software where FSX always felt awkward.

GPS: X-Plane has some fancy avionics simulations but the GPS options are terrible. The solution is Reality XP’s Garmin 430W/530W. It’s a bridge to Garmin’s standalone 400W/500W trainer, simulating the most popular high end GA GPS. The realism is fantastic; it feels like it’s emulating the hardware and running the actual firmware. The only drawback is the navigation database is from 2007. Not a big deal, but newer approaches may be missing and sometimes it’s out of sync with X-Plane’s database.

ATC: Learning to fly IFR requires learning to work with air traffic control; you can’t really simulate that. So enthusiasts have been operating VATSIM, an ATC network staffed with volunteer controllers. VATSIM’s quality apparently varies, there’s a new modestly commercial effort called PilotEdge that looks promising.

Other addons: X-Plane has a robust third party developer community creating model aircraft, better scenery, and various plugin code. Unfortunately X-Plane.org is very poorly organized, hard to find things. I haven’t seen anything there I’ve had to have although I am excited for RealScenery NorCal. There’s also rumours of software that imports Google Maps aerial photography to replace the default scenery but apparently it’s banned. One neat thing is you can use XPlane’s network mode to drive a live display of SkyCharts on your iPhone or iPad.

Hardware: X-Plane has a mouse mode but you really need a controller of some sort. I’m reasonably pleased with the Saitek yoke. I also have an older CH Products yoke which is nicely smaller but doesn’t feel right. Honestly, neither feel right, I have a very hard time controlling the plane in the pitch axis. I’m beginning to think a fancy joystick would work better, if it doesn’t feel like a real plane anyway why not go for something totally different?

Getting fancy: There’s a whole world of very fancy amateur simulation products. People spend hundreds to thousands on avionics simulations, plane models, scenery, and flight controls. And if you’re serious you’ve got 3–6 PCs networked together running X-Plane so you can look out the side window, too. That’s all overkill for learning instrument procedures and for that kind of money I’d start looking into FAA Certified equipment. It may make sense if you want to “fly” a heavy jet, but for my purposes I’m happy with the PC game and then jumping in a real plane.

I continue to look for one great iPhone game. But there’s a a big market now and Apple’s Game Center gives us data on sales. Some notable iOS games:

• Angry Birds: 9,300,000
• Flight Control: 1,300,000
• Tiny Wings: 270,000
• Canabalt: 78,000
• Space Invaders Infinity Gene: 38,000
• BIT.TRIP BEAT: 20,000
• 100 Rogues: 7000
• Forget-Me-Not: 1100

Angry Birds is the #1 iOS hit ever and 9 million people is a lot. For comparison The Sims is the top selling PC game at 16 million. But then again Angry Birds costs $1, so total revenue for the top iOS game is (I’m guessing) 2-5% of the big PC game. Maybe up to 10% with related ads, in-app purchases, and variable pricing.

It’s hard to compare more generally; iOS games tend to be platform exclusives unlike big PC and console games. Here’s sales stats for Xbox 360 games with a whole bunch of 1 million+ games. Even modestly successful AAA games like skate sell 500,000+ copies. Then again they need to, the budgets are well over $10 million.

What I love about the iOS game market is the long tail, all the weird little innovative games. Like Canabalt, a marvelously perfect mobile game. 78,000 sales isn’t bad for a small game, particularly at $3 a sale. (Related: I wish Apple had set the floor on prices at $2 or $3.) 100 Rogues is a great example of a gamer’s game, intricate and clever and beautifully designed. It deserves more sales, but I’m glad it at least has some. Forget-me-Not is tiny at 1100 sales but it’s brand new and has sharp corners, I’m curious to see how it fares.

Caveat: the data above undercounts. It leaves out people who’ve opted out of Game Center as well as people who haven’t played the game since Game Center support was added. Angry Birds claims 100 million downloads, but I’m not sure how much of that is iPhone sales.

Cropper is good Windows software. It’s a simple, modern screenshot utility that makes it easy to save images to disk or upload them to various services. I’ve got mine feeding Flickr. Mac users have Skitch, now we don’t need to be envious.

The key thing is it’s simple. Run it in the background and it overrides PrntScrn, Alt-PrntScrn, etc to automatically save screenshots to a folder, upload them somewhere, email them, whatever you’d like. If you click the tray icon you get a simple UI for selecting a rectangular region of the screen instead. Windows’ default UI is copy to clipboard, paste into some paint program, crop, save, launch an upload program, upload. This is one step.

In addition to Cropper itself you want the plugin pack. The SendToFlickr plugin works great and I imagine the Twitter, Facebook, etc integration is good too. (Don’t install TFSWorkItem; it crashes Cropper). The only feature I’m missing is playing a sound on screenshot.

Scalable Vector Graphics is good technology for drawing images on web pages. It’s sort of a forgotten tech; SVG dates back to 1999 but lack of IE support hamstrung adoption. But it works great in every modern browser, including IE9, and it’s beautiful. A lot of the focus for HTML5 drawing has been about canvas. But SVG is very capable and better suited for document presentation.

SVG is a vector graphics format: you write a description of objects in the drawing as a set of declarations: “red square at (50,50)”, “the word vector rotated 30° and flipped”. Then the browser renders the scene graph. The underlying XML isn’t too verbose and the syntax for transformations and paths and the like is a shorter text format. SVG is remarkably capable in what it can draw and browsers do a beautiful job of rendering with anti-aliasing. Given how rare SVG usage has been it’s amazing how good the implementations are.

This blog post contains some simple SVG: if you don’t see a wind rose in the upper right click through to my blog. Should work in IE9, Firefox 4, and Chrome. (Doesn’t work in Safari and older Firefox because of XML namespace issues unique to my blog). If you view the source you can see the code for the diagram. It’s basically a bag of <path> elements, each of which draws a wedge and translates it.

I first understood SVG’s value when talking to Mike Bostock about his genius Javascript visualization libraries Protovis, Polymaps, and D3. SVG is a declarative document, so it’s very easy to manipulate the scene graph via DOM and Javascript. Mike’s libraries have turned that capability into beautiful interactive visualizations.

SVG is pretty easy to program. You can write it by hand, but for building SVG in Javascript I recommend D3: it does for SVG what jQuery did for DHTML. A key debugging tool is DOM inspection. Browser developer tools let you easily examine and modify the generated SVG on live documents. For docs, the resource I keep going back to is the SVG spec; it’s remarkably well written. O’Reilly also publishes a free online book.

Any language compiler or runtime that requires me to manually optimize out the length of an array when iterating said array is not worth my time to program in.

I’m looking at you, ancient C compilers. And you, Javascript. Particularly you, shame on you for not having a humane syntax for iterating over collections.

I’m assuming V8 and any other modern fast Javascript runtime can handle figuring out that the array length isn’t changing inside the loop. Boy, I sure hope so. I’ve got better things to do than program in assembly language.

We need to have a 25th reunion for the first generation of Web developers. Sometime, say, 2021.

Back in 1995, 1996 I would visit San Francisco and one of my favourite things was to drop in on Adam and Marc at Organic Online, an early web studio. It was in the same grimy SoMa building as Worlds, Inc, some clothing sweatshops, and Wired. It was an exciting time, full of promise. We knew the Internet was important but it still seemed kind of nerdy and specialist. I think few people realized how quickly the entire world would be made closer by digital networks.

Recently I was talking to a friend who said he’d run into Jason and Meg and they’d spent some time talking about me. Which was flattering and seemed natural even though I haven’t talked to them in months. And this week I was lucky to have lunch with Merlin who mentioned Ben and Mena and that got me thinking of Carl Steadman. Also, related, I sure wish I saw Matt’s talk at SXSW. So many bright, creative people.

That last paragraph is blatant name-dropping, but I do it to convey my nostalgia. Those are all people I count as friends and colleagues but I don’t have regular relationships with them. It’s like we were college classmates. We’re a community defined by a shared experience of helping birth the Internet. We were in the right place and right time and we (and many others) have had the opportunity to transform the world.

Part of my sense of generation was crystalized by this headline: 4chan founder: Zuckerberg is “totally wrong” about online identity. Christopher “moot” Poole and Mark Zuckerberg were both born in the 1980s: not quite post-Internet, but still a different generation than we are. And here they are arguing about online identity. A lot has been written about Facebook reflecting a new generation of startup and I admiringly agree. I’m excited to see what else comes out of the generation ten, twenty years younger than I am.

Awhile back I posted some quick thoughts on a new kitchen gadget we got, a sous vide cooker. We’ve had it a couple of months now and my final opinion is it’s useful but I’m not sure it’s worth the expense and space. Ken does the cooking; he likes it.

Sous vide is the process of cooking food, particularly meat, slowly at low temperature in a sealed bag. The great appeal of sous vide is it’s really simple to get consistent results. Want a medium rare steak? Set the temperature to 55°C, toss some beef in for 2-72 hours (depending on cut), and out comes a perfectly cooked medium rare beef. Hamburgers, fish, poultry, eggs, all easy to cook to a consistent texture.

In some sense sous vide cooking is the mirror twin of barbeque. But what it doesn’t do is add flavour. Traditional hot cooking sears, caramelizes, browns, burns, and generally imparts complex flavours to food. Sous vide doesn’t do that. So you have to brown the meat after cooking it. But the browning is tricky; you can’t do it too long lest you overcook it. (Spicing is also tricky; some spices taste different after being cooked sous vide). We’ve yet to get a really good crust on a sous vide steak.

If we had no idea how to cook a steak properly the sous vide would be a miracle. Or if we needed to turn out forty steaks a night consistently, it would be a huge help. But Ken already knows how to cook a great steak on a grill, in a skillet, or in an oven. So now we’ve got a fourth way, with some drawbacks, and I don’t quite see the point.

One strength of sous vide is that it’s not a dry process. Chicken and pork come out beautifully tender and moist. It’s a great way to salvage the flavourless fat-free industrial meat we get in the US. I also shouldn’t understate the value of the convenience; Ken really likes not having to worry about timing the meat completion along with everything else. And it’s easy to cook something, store it in the fridge, then finish it with a quick sear days later.

The home appliance of choice is the SousVide Supreme, $400 on Amazon. It’s a well made appliance, good construction and works well although I wish they’d done a better job on the lid insulation. It’s far too large for meals for 2–4 people, but the demi version they offer isn’t significantly smaller or cheaper. You also need a vacuum sealer; any air left in a bag at all will cause it to float and cook inconsistently. The package deal is $470, a significant expense. If you don’t already own good knives and cookware, buy those first.

Flying a plane when you can’t see out the window feels remarkably different than flying visually. When you’re in a cloud you never quite know if you’re about to fly into a mountain. All the instrument training is to make sure you avoid the places where the mountains are, but the psychology of not being able to see the mountain you know is nearby is significant.

One key piece of how we stay safe in IMC is air traffic control radar. ATC generally knows where your plane is and helps keep you off the rocks. Often in NorCal the controller gives you vectors, tells you to fly a specific heading for a few minutes while they route you somewhere convenient. Once you’re on vectors you are 95% trusting the controller; you’re off the charted path.

I’ve been very impressed with the quality of ATC, particularly here in NorCal. But controllers are human and make mistakes, sometimes mistakes that get pilots killed. There’s a training crisis in ATC right now, the controllers hired after Reagan busted the union are reaching mandatory retirement and they can’t train replacements fast enough. The rate of mistakes is going up. Flying is still very safe but it’s in the pilots’ interest to look out for themselves.

The best defense for a pilot is always knowing where you are. Situational awareness is remarkably challenging with traditional radio navigation, where all you may know is you’re somewhere north of a beacon 30 miles away. I’m at the stage where I still regularly get confused: am I west of the airport or east of it? Fortunately my airplane has GPS, a map with a terrain database that shows my position at all times. If I’m being vectored into a mountain, I’ll get ample warning. But the FAA still treats GPS as an add-on in instrument flight, an extra, it’s not necessary equipment. So we learn how to fly without GPS too. I wonder if I’ll ever be truly comfortable flying without my electronic map.

Every few weeks I get unsolicited blank checks in the mail from my credit card company, Capital One. “Instant cash!”. Of course it’s a trap; the cash is a loan at some poorly disclosed usurous rate, probably 20%. So I dutifully tear the checks in half and throw them away.

Now Capital One has gotten more aggressive and is sending me regular emails asking me to transfer balances. So convenient! “Keep in mind, you’ll pay a transaction fee of 3% of each transaction amount”. Another unwanted offer. I asked them to stop emailing me.

Unfortunately, the option to be excluded from our balance transfer solicitations is not available at this time. We’re sorry for any inconvenience this may cause.

I only gave them my email address so I could get emailed statements. Apparently there’s no way for me to opt out of their scammy attempts to trick me into borrowing money from them at outrageous rates. Awesome.

I’d cancel the card, but they’re the last US card that doesn’t charge a 2-3% scammy foreign currency conversion fee .

Google Chrome has a great feature, incognito mode. Press Ctrl-Shift-N or select Wrench / New incognito window and you get a browser window with no state. No cookies, no cache, no browser history, no addons, nothing loaded before the page and nothing saved after. (Firefox, Safari, and IE can do this too).

This mode is half-jokingly called “porn mode”, but I find it very useful as a software developer. Some ways I use it:

• Testing links before sending them out. Check whether the link you’re about to send your friend gives them the same page you see or if the page requires a login or cookie to look as intended.
• Debugging Javascript. Chrome extensions confuse Javascript debugging; the scripts show up in the developer console and you never can be sure whether some addon is modifying things. Incognito mode loads no extensions unless you explicitly enable one.
• Bypassing caching. It’s hard to test a webapp reliably, particularly for performance, when so many of the assets in the page may be cached. A new incognito window starts with no cache. The window seems to keep a memory cache, so you can also test your caching behaviour after the initial clean load.

Porn mode was invented for browsers to not store state after ending a session. But I find I mostly use it to not load state before starting a session.

I keep seeing this icon in software: Wunderground, Google Docs, random Mac and PC apps. I can tell from context it must mean “save”. But how does a line drawing of a house with a giant window above the garage mean “save”? Is that blue square my files, stored in a box in the garage? And why is a corner of the house missing?

For that matter, why am I manually saving data anyway? Maybe this icon really means “publish” and the blue square is the printing press in my garage.

I’ve started training for my instrument rating. The IR is essentially the second half of a pilot’s license; it lets you fly in clouds when you can’t see the ground and have no visual reference to navigate or even stay upright. It’s a pretty demanding rating and takes about the same amount of training as the basic private pilot’s license. More book learning than the PPL, so I’m hoping it will go relatively easily. Half the challenge is the skill of flying blind, using the 6 (or 8) primary flight instruments to keep the plane level and flying in the right direction. It’s not too difficult: use the attitude indicator to keep the plane level, center the CDI needles to keep the plane flying the right way, and cross-check your instruments to make sure nothing strange is happening. As long as you have your full concentration on the instruments it’s not hard to go where you want.

The other half of the challenge is conceptually understanding where you’re going, particularly when flying an instrument approach. You better fly the course right or else instead of being in a cloud in a valley you may end up inside a mountain. The extra workload of dealing with charts and radios means you can’t fully concentrate on the instruments. It’s amazing how quickly some mysterious force will cause your plane to drift 30° off course and lose altitude when you spend 10 seconds looking at a chart. I’ve got some 5 hours of simulated instrument training so far and I’m pretty overwhelmed.

The solution is practice. Basically a brain workout, training it to be able to handle the extra workload. It’ll get easier to keep the plane flying the direction I want. And as I get familiar with approach and radio procedures it will require less thinking to do everything right. Maybe some day I’ll even stop confusing my left from right!

It’s a bit frustrating that half my instrument training is for skills that are on the way to obsolesence. In the real world I’m going to do be doing all my flying with a WAAS enabled GPS and a very capable autopilot, where flying an approach is mostly a matter of punching the right buttons on the computer. But I still have to learn the older radio navigation technologies. And, of course, learn to do it all by hand in case I don’t have the fancy gear.

Do you store important things at Google? Maybe private documents on Google Docs, or all your email on Gmail, or even just your search history? Do yourself a favour and enable 2-step verification for your Google account.

Two factor authentication makes passwords stronger, requiring the user prove themselves with an extra code in addition to the password. The codes are time-limited so they can’t easily be stolen and reused. I’ve been using two factor for my bank and my Warcraft account for years now and I’m glad I can finally protect my Google account. Email accounts are particularly sensitive since so many other websites let you reset your password via email.

Google’s implementation is pretty good. Most users will set it up so their second code comes from a simple program on their smartphone and enter it once a month for every new computer. If you lose your smartphone there are backup login options available; a printed code you can carry in your wallet or a backup phone number. There’s a bit of extra awkwardness for some applications that don’t know to ask for the second auth code; the workaround is reasonable.

The interesting thing is Google is also an OpenID provider. All the pieces are now in place to end passwords. Most web sites (like, say, Gawker) shouldn’t have a user password at all, just use OpenID to authenticate via Google. And Google authentication is now quite strong, thanks to two factor. There are business and product barriers to widespread adoption of OpenID logins but it’s undeniably more secure.

I’d like to end this post with a shout-out to my friends at Duo Security, a startup developing two factor authentication for easy integration into any site. Check out the demo; it’s very slick and the guys running that company are some of the smartest security people I know. Google’s gone and built their own system, as they always do. But if you’ve got a company looking to do two factor yourselves check out Duo.

Twitter is a lot of fun while watching a sports game. You can tweet your witty comments to your absent friends. And Twitter lets you participate in a global zeitgeist. Twitter has been around for five Super Bowls now and has published statistics from three of them: 2008, 2010, and 2011. Each graph is roughly similar; a sustained increase in activity and tweet spikes around the big plays.

One fascinating difference is 2011 is the first year when Twitter activity went up during the halftime show. In 2008 and 2010 twits took a relative break during halftime. In 2011 halftime is the biggest sustained tweet activity for the whole game.

Why more tweets this halftime? One guess is it’s because the Black Eyed Peas were completely awesome. Despite the terrible sound the show was fantastic. Or maybe it’s demographic. The Black Eyed Peas presumably skew to a younger, more connected audience than Tom Petty in 2008 or The Who in 2010. Also Twitter is now more an integral part of American entertainment, all the way up to will.i.am planning to tweet during the half-time show (foiled by #attfail).

Another interesting datum is the impact of the Super Bowl on Twitter traffic. In 2008 tweet traffic during the game was sustaining 1.25x a typical day with spikes up to 2.5x for big plays. 2011 is the same: 1.25x sustained increase with spike of 2.5x for the ending of the game. Surprisingly consistent pattern.

This year Twitter also released absolute numbers: sustained 2000 tweets / second, up to 4000 at spikes. That’s a hell of a lot of traffic for a real time distributed system. A lot of database updates, cache invalidations, message propagations, and message deliveries. Compared to the fail whale months Twitter has been doing awfully well recently. If that kind of systems problem interests you, Twitter is hiring.

Ken’s birthday present was a day’s flying in a Cessna Chancellor. The 414 is a pretty big small airplane: twin turbocharged engines, 6750 pounds, comfortable seating for seven people. It’s a small plane compared to commercial aircraft, but it’s big and fast enough that it can be used for charters. And Ken and I got to fly it. Well, our CFI flew it, but we got to play along. It’s a lot of airplane.

The FAA requirement for me to be legal in the Chancellor is laughably small, 10 hours of flying time to learn how to recover if one of the engines fails. That’s a crucial and difficult operation but apparently it doesn’t take long to pass the required skill checks. But the MEL is the least problem; I’d need a lot more time and experience before I’d feel safe. And the insurance companies want some 100 hours in a similar plane before insuring a pilot.

The main challenge of flying a bigger plane is it has more energy. It’s heavier, it’s faster, it flies higher. It takes experience, planning, and fine control to go from 200kts at 18,000′ to stopped at sea level on a 2450′ long runway. More energy is more fun, of course, particularly being able to climb at 1000 feet per minute or more above the Sierras. But everything goes faster, and heavier, and you have to know your plane.

The other big challenge is systems complexity. Bigger planes have more stuff: cabin pressurization, de-icing, turbochargers and intercoolers, complex fuel distribution, etc. It all has to be managed by the pilot, mostly manually. Every new task adds pilot workload. The only way to develop that extra bandwidth is experience.

The fact that there are two engines out front seems the least of the challenges. There’s a bit of extra work tuning the RPMs so they run harmonically but other than that a twin is all gravy. More power. More lift from the prop-wash over the wings. And a great view out the nose with no engine or prop in the way. Babysitting the turbocharger temperatures was more work than worrying about having two engines. As long as both of them are working.

I’m not in a hurry to move up to a twin. I just started my instrument rating, that’s the next big task. Then more experience flying, only then maybe something bigger and faster.

I’m a happy Briggs & Riley customer. They make suitcases. Well made and functional cases, but heavy, expensive, and not particularly attractive. What makes them worth the price is a true, no bullshit, lifetime warranty. “If your Briggs & Riley bag is ever broken or damaged, even if it was caused by an airline, we will repair it free of charge.”

We own two of their smaller size suitcases. Both have had to have significant repairs, one several times (a flawed wheel design, since re-engineered). Each time the repair has been completely stress free. Ship or bring them the case, get the case back in a week or two. Done. I’ll be a loyal customer for life.

PS: luggage is one of the most heavily spammed Google searches. I wonder what will come of this ad-free, commerce-free, genuine customer testimonial?

Minecraft has been a huge indie game success, some $12M in sales. There’s an enormous modding community for the game: new creatures, a complete magic system, better lighting, server management frameworks, etc. All sorts of creative mods. Here’s the crazy thing: all these fans are studying and modifying obfuscated code.

Minecraft is Java, obfuscated with ProGuard. In response the Minecraft developer community has created the Minecraft Coder Pack. MCP decompiles the code, maps symbol names to something meaningful, and spits out editable source. It even has a handy re-obfuscator so your compiled code is symbol compatible with the official release. It’s a remarkable piece of work to enable fans to modify a game they love.

The mod community is clearly good for Minecraft. The game is notoriously buggy and the development team is small, so the community fills in gaps. For example, server management: hMod and its successor Bukkit are essential for public servers to protect from griefers. The various game enhancement mods serve as a testbed for new gameplay ideas. The developer team is even taking patches from modders, for instance fixing a ridiculous variable swap bug. I can’t think of another example of obfuscated software being so heavily and publically modified.

So what benefit does the obfuscation give? It clearly doesn’t keep the code secret and I doubt it helps stop people from avoiding paying the 15€. What obfuscation does do is set a very clear legal barrier to anyone legitimately modifying the code. The mod community is kept off balance, in a sort of unstable detente where they’re officially discouraged but also productive and vital.

I took advantage of our amazing January weather to do a quick overnight on California’s Central Coast, specifically at Shell Beach. Great trip in an airplane: instead of a boring four hour drive I had a beautiful 75 minute flight down the coast in to San Luis Obispo. I stayed at the Spyglass Inn, an unremarkable but pleasant beachfront motel with a nice view. It’s inexpensive and conveniently next to the Dolphin Bay Resort. I would have felt ripped off if I’d paid the $400 to stay at the resort but being next door I had the same view and could walk over to Lido for some of the finest dining on the coast. Nice dinner: abalone from Cayucos, also dungeness crab. Also good breakfast the next morning nearby at Zorro’s.

There’s not a whole lot to do on this part of the coast. Off-roading on the beaches and sand dunes down at Oceano. Enjoying the views from the cliffs. Butterflies in Pismo Beach (in winter). The beach itself, although Shell Beach is too tiny for that; you want Pismo Beach or Avila Beach for a big expanse of sand. Mostly it’s just beautiful. Pismo Beach is also near to San Luis Obispo and Paso Robles, two nice towns inland in low valleys. There’s good wineries in the area, particularly around Paso Robles. It’s hot in the summer but on a sunny winter day it’s terrific. Basically the Central Coast is a really nice, mellow part of California. I like it every time I go.

The whole coastline from Avila Beach to Oceano is one big stretch of tiny beachfront communities. Pismo Beach is the center and the biggest town, with the most bars and cafes and the most sand. Next time I visit I might stay at the Kon Tiki Inn or maybe the Avila Lighthouse Suites for something more remote.

Ken and I flew out to Florida and back for Christmas and New Year’s. We visited my sister in Palm Beach, then down to Key West for New Year’s. Big flying trip: 8 solid days of travel, some 5500nm and 45 hours. Lots of photos. Some of the highlights:

• Flying around Texas where I grew up. We landed at 06R in Bellville, TX, where I was born. Tiny little airport but adequate, also had fun circling the town looking for our old house. On the way home we also stopped at KJDD near Mineola, TX where my grandfather was from. While at that airport I met Wayne Collins who knew my family and had flown a little Bonanza all the way to Adelaide, Australia.
• Visiting St. Mark’s, Florida, a beautiful wet part of northern Florida with a surprisingly rich history. Totally worth a visit if you’re in the area.
• Hanging out in Key West for New Year’s. Ken and I were surprised at how much we liked it; beach towns aren’t usually our thing. Key West is just busy enough to be fun without being overwhelming, it’s laid back, there’s great restaurants, and a definite non-conformist streak to the town. Would be glad to go back.

Our full route was

KSQL San Carlos, CA
KCRQ Carslbad, CA
KSDL Scottsdale, AZ
KPEQ Pecos, TX
KAUS Austin, TX
06R Bellville, TX
KNEW New Orleans, LA
KCTY Cross City, FL
F45 Palm Beach, FL
KEYW Key West, FL
KBFM Mobile, AL
KJDD Wood County, TX
KLBB Lubbock, TX
E80 Belen, NM
KSDL Scottsdale, AZ
KWJF Lancaster, CA
KSQL San Carlos, CA.

Pretty much all those places worked out pretty well, I was pleasantly surprised by both Mobile and Lubbock. The big disappointment was New Orleans, we didn’t have enough time to enjoy the city and Bourbon Street has gotten pretty unpleasant. Next time around I’ll head to Frenchman St for music. My only real regret on the trip was we were in such a hurry to get across the country we didn’t see much of the country; I’d love to do a trip like this where I only need to fly three or four hours a day.

Awhile back I endorsed Rebit as a Time Machine like backup for Windows. I just deleted it from my system and can no longer recommend it. I don’t think it’s terrible, but the reasons I gave up on it are sort of interesting.

• It’s noisy. Rebit is designed to run all the time, which means it’s constantly writing to a chattering external drive. It was irritating enough I started turning off the drive most of the time.
• It’s intrusive. Rebit installs a Windows Explorer extension so that when you right click on a file, it offers a menu of options. They did it wrong somehow and it added 300ms+ of latency to interaction.
• It’s buggy. At some point an upgrade of Rebit failed. It kept trying to “upgrade” to an older version. Their support folks tried to help me (“delete this magic file!”) but it didn’t fix the problem. Also, maybe related, I lost confidence it was actually making backups.
• It’s confusing. I bought “Rebit Backup Software”. That product is no longer offered, they now sell “Rebit SaveMe”. I think it’s a different program, but I’m not really sure, and can’t be bothered to find out.

These kinds of problems are typical for software. Geeks like me will often spend a lot of time figuring out how to work around the problems, find the right version of the software and the quiet hard drive and understand it. But for Rebit I couldn’t be bothered. So I uninstalled it and switched to manual backups using the built-in stuff in Windows Vista.

It amazes me there’s not a really good versioning filesystem for consumer computers. We’ve been doing it in Unix and VMS for 10+ years. Apple’s Time Machine gets mixed reviews from what I’ve heard.

APRS is an interesting system, a network for wireless transmission of data packets to the Internet. It’s global, free, and low latency. APRS is often used to provide location tracking of vehicles but can route any short text message. It’s a great example of amateur radio hackers meeting Internet hackers and echoes other much loved networks like Fidonet or Usenet.

APRS lets you send short messages from a handheld radio to the Internet through the simple expedient of broadcast and repeat. A user broadcasts a new APRS message on a VHF band. Nearby fixed stations (“digipeaters”) hear the message and rebroadcast it to other nearby digipeaters, so the message floods the digipeater network (within path and hopcount limits). Eventually an APRS iGate hears the message and forwards it into the Internet where it is published to the world as part of a single, global APRS message stream. Not very efficient, but reliable and well suited to short messages.

What is APRS used for? A big application is location tracking, with radios beaconing out location and speed every few seconds. Here’s a friend’s training flight. Location packets are a big part of the APRS spec, but there’s also arbitrary text and email and SMS-like applications. aprs.fi gives a good map-based view into APRS activity, and here’s what The Matrix really looks like.

So what’s the drawback to APRS? It’s amateur radio, which means strictly non-commercial and you need a license to use it (10 hours of study). Private data is not supported. And it’s all hackerware, some assembly required, although recent radios like the Kenwood D72 are pretty turn-key.

The real limit to APRS is that it’s the sort of gentle hacker network that’s maintained by love. I suspect it wouldn’t handle the strain of 100x as many users or commercialization. The broadcast design makes for limited throughput and the community process probably wouldn’t handle business interests well. Sort of analagous to the pre-1993 Internet. Nothing wrong with that, but I suspect we’ll never see APRS-based location tracking for the mass market.

A funny thing about becoming a pilot; you start hearing a lot more airplanes. With this unseasonably wonderful weather we’ve had a lot of little planes sightseeing over San Francisco. I used to be barely aware of it, now I hear every one.

It’s a little obnoxious. What’s hardest is I can’t help but listen to whether the engine sounds healthy. Twice now I’ve run outside in alarm only to look up and realize the pilot just slowed the plane down in order to enjoy the view at a more leisurely pace.

We bought a sous vide cooker. We only got it yesterday and all I’ve cooked in it was an egg, but already I’ve got some thoughts.

The appeal of sous vide cooking is easy precision. You dial in a specific temperature to a water bath and let your eggs, or meat, or veggies hang out for an hour or two at that temperature and you get perfection. The variance in temperatures is amazing. Eggs go from runny (but pasteurized) to hard between 58°C and 70°C. Beef goes from rare to medium between 51°C and 60°C. Traditional cooking on a hot stove requires the cook control the final temperature with time. Sous vide lets the cook just set an exact temperature and forget it; there’s no risk of overcooking. For someone like me with no kitchen skills, that’s very appealing. But so many caveats.

First, sous vide cooking is slow. A hamburger takes 2 hours. A big chunk of meat can take 2 days. It cooks entirely unattended but you do have to plan ahead. A related problem is you can only cook to one temperature; no cooking both your meat and your veggies in the same bath. And when the sous vide step is complete you’re not finished. Meat still has to be browned or seared, and seasoned, and finished. And while an egg yolk may come out creamy at 63°C if you want a firm white you need a final hot boiling step (carefully timed).

The other issue is equipment. The Sous Vide Supreme we bought is pretty well built, but at $450 it’s an awfully expensive gadget. It takes up a lot of space. And you really want a vacuum bag sealer to go with it, another gadget. I fear if we store the kit away next to the mixer, the food processor, and the blender we’ll never use them.

I’m optimistic the sous vide cooker will get some good use in the house. It’s simple enough even I might be able to cook well with it. But it’s got some drawbacks.

One thing I’ve never quite understood is why there’s such a divide between the Internet / Web tech world of companies like Google or Twitter and the game development world of companies like Blizzard or EA. Computer games have often had terrible Internet experiences, awful web sites, etc. The game companies are catching on now but there’s still a huge culture divide.

So that’s why I like my friend Chris’ job posting for an online engineer. It’s a job at a top tier game company, Irrational Games (of Bioshock fame, also System Shock 2, Freedom Force, and the upcoming BioShock Infinite). But it’s for a deeply Internet nerdy set of skills: scalable web backends, NoSQL storage, protocol design, etc.

It sounds like it’d be a fun job for an Internet / web developer looking to do something new. Games have a totally different interaction with users than web sites, real time and intensely emotional. Different technical environment, too.

For nearly two years now I’ve been using various hacks to remove the useless left navigation bar from iGoogle. I’m currently at an impass and can’t turn it off; none of the Chrome extensions I can find work. Even Frugal iGoogle, a recent and well designed extension, fails.

The underlying problem is changing page layout. My last workaround broke when I was put in an experimental iGoogle layout littered with icons. Deleting cookies fixed that but iGoogle still redirects itself to the URL http://www.google.com/ig#t_0. The left nav is now implemented in a table element, of all things, with a class of GL-BX4-BFS. That seems to be new.

The real problem is the iGoogle design itself. There’s at least 4 Chrome extensions to remove the left gutter. It’s a small nuisance but a regular one for a default start page.