Seven years ago I wrote about XML security problems, the XXE vulnerability. This flaw is the gift that keeps on giving: someone exploited Google with it this month.

XML is a ridiculously complicated data format. And XML parsers implement all the features, including the obviously dangerous and useless ones. And engineers keep forgetting to turn those features off. It’s just terrible.

  2014-04-11 18:29 Z

Unison is good software. It’s a command line program to synchronize filesystems, to keep a directory tree identical on multiple computers. I use it to sync about 40G of files across two Macs, to keep my home directory and source code and various applications in sync. The neat trick is I sync those two Macs through a portable hard drive so I don’t have to wait for hours for files to go over the Internet. Unison can also work online so changes are propagated automatically.

Unison is a lot like rsync. But Unison is designed to be bidirectional. Rsync always syncs one way: copy A to B. Unison will look at the differences between A and B and merge them, including a limited UI for conflict resolution. This protects me from the case where I modify something on both machines without syncing beforehand.

The main drawback with Unison is it’s slow, it takes many minutes to decide what files to sync. I also hate the interactive UI; it doesn’t work well when you have lots of files that changed in both places. I’m also a bit concerned that it’s no longer under active development but Unison is the rare software that’s a complete product, it’s not clear it needs many changes.

There are other tools solving similar file sync problems, none perfect. Dropbox is phenomenal but doesn’t have offline syncing of large files. Camlistore is promising but not quite ready for civilian use. git can be used to keep stuff in sync but is better suited for text files whose history you want to keepl. And CrashPlan is great for online backup but doesn’t really provide a second live copy.

  2014-04-07 16:32 Z
Gfycat (and CloudFlare) has a fantastic error page for when they have a server error.
Such a clear, simple statement of what the error is and what the user can do. One of my pet peeves is software that blames the user when it's not their fault, like the "your Internet is down" message Steam displays when their client can't connect to their server. This kind of message is much more honest and useful.

BTW, Gfycat is an awesome service. They host animated GIFs for sharing. And they transcode the bloated source GIF to much smaller HTML 5 video, then serve the smaller file to browsers who can handle it. The hosting is good, the 95% bandwidth savings is great.

  2014-04-01 15:56 Z

League of Legends has a serious security problem: denial of service attacks. Some of these attacks are against the game as a whole and bring down the whole system. Presumably Riot can eventually protect their servers from that. Worse are targeted DDoS attacks against single players; it’s not clear they can defend themselves

The motivation for attacking individual players is ugly. The game is very competitive at the highest levels of play, with prize money and pro careers on the line. Knocking an opponent offline or just lagging them is enough to get a win. Even if the game is thrown out as invalid a DDoS is still way for a losing game to be converted into a tie.

There are guides on how to avoid being a DDoS victim. They boil down to “don’t reveal your IP address”, which in practice means “don’t use Skype from your real IP”. VPNs, playing only from Internet cafes, etc are other options. But these are minimal solutions at best; hiding your location from the world is really difficult. There’s new rumors the whole IP identification part is entirely automated and foolproof.

I don’t really know how a normal Internet consumer can protect his ISP from being knocked offline with a DDoS. It’s an ugly situation.

  2014-03-11 21:35 Z

Apple totally screwed up SSL with a fundamental bug in their certificate checking implementation in both MacOS 10.9 and iOS 7. Every consumer iPhone, iPad, and Macintosh running recent versions of their OS is vulnerable. My understanding is SSL certificate checking basically does not work and any secure site can be spoofed with a man-in-the-middle attack. It’s about as deep a flaw as it goes. There’s a patch for iOS out but not yet for MacOS. You can test if a browser is vulnerable here.

The bug boils down to a simple typo in the code, the good ol’ C gotcha that indentation doesn’t match control flow. Bugs like that happen in C. What’s alarming is Apple didn’t catch the bug; not with a lint tool, not in code review, not in unit testing, not in integration testing. No aspect of Apple’s software development process caught this bug before releasing it to millions of users. That’s terrible engineering practice; in a critical security library it’s outright negligence.

At the moment MacOS users are entirely vulnerable and there’s no fix. In the past Apple has taken many weeks to fix critical bugs in things like Java, hopefully they’ll be faster here. Using Chrome instead of Safari will insulate you from malicious web servers, Chrome wisely has its own SSL implementation. But a whole lot of other Mac software is relying on the broken certificate library, presumably including Apple’s own software update system.

Nice of Apple to publish the exploit before the fix.

  2014-02-22 17:09 Z

I love the Clear Dark Sky Chart, a geeky little astronomer’s forecast. Also CSC Menu which puts it on a Mac OS menu bar. Here’s a sample image.

Above is the forecast for near Grass Valley, CA for the next two days. Time goes from left to right, each row is for a different sky condition: cloud cover, transparency, seeing, and darkness. Also the temperature, humidity, and wind for your backyard comfort. See the legend for details, but basically dark blue is good. Once you learn to read this presentation you can quickly tell if it’s likely to be a good night to look at stars in thousands of locations. Looks like it'll be clear but relatively poor tonight.

These charts are derived from a more traditional map forecast prepared by the Canadian Meteorological Center. Their site shows you maps of things like cloud cover by the hour. The Clear Dark Sky site basically samples the pixels at a specific location and displays the time series as a strip chart. Simple and useful. The mysterious seeing forecast is particularly idiosyncratic to astronomy, an experimental forecast of how bad atmospheric distortion is likely to be.

  2014-02-21 23:17 Z

The “angry rainbow” palette is the colors you get when you set saturation and value to 100% and then spin the hue wheel. From bright red #ff0000, briefly through yellow, a long linger in #00ff00 green, longer still around dark blue #0000ff, and finally back to red via an eye-searing trip through purple. The term “angry rainbow” isn’t in common usage but I’m doing my best to spread it. I got the term from someone else, maybe another student at the MIT Media Lab? (See also: angry fruit salad).

The angry rainbow is always the wrong palette for data visualization. It’s too bright, too colorful, and too reliant on non-uniform hue discrimination. But it pops up all the time, from random weather maps to heatmap examples to NYTimes work sketches. It seems to be the default palette for various visualization tools, no doubt because it’s easy to generate in software. I’ve certainly been guilty of using it myself, somehow it’s always at my fingertips.

So what’s a better choice? Honestly, almost anything. Even knocking saturation and value down to about 80% gives a more pleasing result. If you have continuous data try plotting it with varying brightness instead of hue, or narrowing down to a red/blue color ramp (properly interpolated) instead of the full rainbow. if you want to do it right consider a ColorBrewer scale; the D3js implementation is a fine place to start. If you roll your own palette, work with colors that are not fully saturated and not fully bright. Think carefully about whether hue is really the thing you want to vary.

Angry Rainbow Dash by Uxyd
  2014-02-13 18:02 Z

RSA Security (part of EMC) was one of America’s most respected security companies. Thanks to Edward Snowden, we now know the price of their reputation: $10 million. For that tiny sum RSA sold out their customers, deliberately installing a compromised random number generator in their core security library BSafe at NSA’s request. For $10M, a company’s reputation destroyed.

The nature of NSA’s sabotage is worth looking at in detail. We knew back in 2006 that Dual_EC_DRBG, a NIST standard crypto random number generator, was fishy. That algorithm has baked into it an arbitrary constant; two Microsoft researchers figured out that if an adversary had chosen that constant, then the numbers were predictable and any system built on it was insecure. Snowden’s leaks confirmed in Sep 2013 that this backdoor had been placed. And now in Dec 2013 we know the price: $10M. (Interestingly, one old-school cypherpunk knew the price back in September).

It’s worth noting that RSA’s complicity with NSA is not their only enormous security black eye. Back in 2010 their flagship SecurID two factor login system was also widely compromised, it’s assumed by the Chinese government trying to get military and commercial access to US and European interests.

Open source ends up looking good in all this mess. NSA has probably attacked other random number implementations. There was a weird push from Intel to get Linux to completely trust their undocumented hardware generator, something resisted by the Linux team (thankfully). And OpenSSL, the open alternative to RSA’s library, doesn’t use the compromised algorithm (although their code has had its problems).

I remain indignant that NSA is willfully going around deliberately sabotaging the security of core Internet components. Even if you believe it’s good for NSA themselves to be able to break all encryption, it is so dangerous to have back doors like this hiding in systems. NSA is actively undermining everyone’s security.

  2013-12-26 11:25 Z

YouTube Center is good software. It’s an unofficial browser extension to make YouTube work better. Works in most browsers; for Chrome you download the Opera .crx file and drag it into the Tools/Extensions page.

What does it fix? #1 thing is it lets you disable DASH playback, the nonsense YouTube implemented a couple of years ago. In theory DASH makes videos play faster and more efficiently; in practice it’s the crap that makes it impossible to pre-buffer a video or seek backwards while playing. YouTube Center also does a good job at resizing the video window to use more of the screen, so that a 720p video actually has a 720 row high window to play in. I also use it to prevent auto-play and to select the video resolution I prefer.

The main drawback is that there are too many configuration options, many of which you don’t need. Classic hackerware; the author lets you configure everything, so it’s up to the user to tune the few things they really need to set.

I’ve used a few “fix YouTube” extensions in the past that were flaky or broke when YouTube changed something. This one seems to be working for me. I don’t understand why Google’s let their video product get so crummy that it’s necessary to hack it like this.

  2013-11-30 15:42 Z

Here’s something ugly, the whois response for pirate book site Below is an extract of the interesting parts that both MacOS and Debian’s whois display.

$ whois

   Whois Server:

Billing Contact:
  Name           : li xiaoing
  Email          :
<script src=

Huh? What’s an HTML tag doing in this whois response? And under what circumstances might that script tag be executed? I can imagine a na├»ve Web interface just injecting that script wholesale into my browser. Every way I load the referenced script it seems benign (right now), but that’s an attack vector waiting to happen.

  2013-11-15 16:44 Z