Some Bits: Nelson's weblog

Debian breakage with BitTorrent

I love BitTorrent. Not because it lets me steal the w4rz0rz, but because it’s a really great new transport for large downloads. I’ve been impressed with Bram’s stewardship of the protocol and am glad to see he’s pursuing commercial options.

But open source protocols in hackerland are fragile, there’s a danger they fragment. And that seems to be happening with BitTorrent. I’m seeing torrents now with three kinds of tracking: the standard TCP tracker, UDP tracking, and trackerless. Which client works for all?

I’d been wondering why my torrent downloads recently weren’t working. I think the problem is that Debian testing installs bittorrent 3.4.2-4, which doesn’t seem to work very well. The 4.0 .deb from bittorrent.com works better.

tech • bittorrent
 

Downloading Return of the King

After surveying BitTorrent download traffic for Matrix: Revolutions last month I thought I’d check out download traffic for a movie that didn’t suck.
Return of the King traffic is about the same as Matrix Revolutions: lots of traffic early, then the tracker crashes. People were successful though: several thousand probably got the whole file before the tracker died, patient ones could finish when the tracker was restarted, and there were at least 10 other torrents out there.

10,000 people grabbing a 2 gig file is a lot of video on demand. Still, crappy quality. The one sample I saw had decent sound and the picture didn’t shake but the colour was all washed out. The cinematography in ROTK is so beautiful $9 is a bargain.

tech • bittorrent
 

Downloading The Matrix: Revolutions

The Matrix: Revolutions had a complex worldwide simultaneous release, ostensibly to fight piracy. Piracy happened anyway; 24 hours after the release, copies showed up on the net for download. Badly compressed movies of shaky camcorder copies. Pay the $9, folks.

Using the same /scrape URL that torrentspy uses, I tracked the BitTorrent activity of a 1.2G copy of the movie over the last week (from the day after movie release).

The main story this data tells is that a lot of folks tried to download the movie: 8500 at the peak, roughly the most people I’ve ever seen hit a single BitTorrent. The gaps are when the tracker was down; I’m surprised that so many people stuck around when it came back up. I’m also surprised at how many people stuck around to serve the file after downloading it; a steady 20% or more after things settle down.

For what it’s worth, this graph pattern is common for all the popular torrent streams I’ve monitored. The big variance seems to be how long the tracker survives.

tech • bittorrent
 

Good BitTorrent windows client #2

TorrentStorm is the latest good Windows BitTorrent client. It’s based on the experimental client that I liked, but with lots of extra features. Screenshots.

Good things it does:

  • Displays seed, leech data like TorrentSpy
  • Saves .torrent files
  • Resumes downloads on startup
  • Gives unsorted view of downloaded pieces
  • Fast resume algorithm based on piece files
  • Simple bandwidth configuration
  • Excellent documentation
I’m disappointed there’s no source available. I think it’s funny there’s Python under there and no one can tell.
tech • bittorrent
 

TorrentSpy

TorrentSpy is good software to see what’s going on in BitTorrent. It shows the .torrent data in a nice GUI and asks the tracker how many complete copies and downloaders there are. Nicer than my BitTorrent dumper.

BitTorrent has a flaw; it’s easy to get 99.9% of a file and never complete because no one has the last few bits. The main use of TorrentSpy is to see if any client has a full copy. Unfortunately it can’t detect if the whole file is out there, but no one client has it all. With the average torrent having only 20-30 clients I guess that’s unlikely.

tech • bittorrent
 

Business model for BitTorrent

Watching BitTorrent penetrate the game demo market has given me an idea for a business model for BitTorrent: charge companies for help hosting files via BitTorrent. It could be a consulting business, teaching sites how to set up and run trackers and seeds. Or it could be a service business, running a BitTorrent hosting service for others. You could offer client support, maybe custom-branded clients. There may even be room for proprietary software here: special trackers and monitoring tools.

I don’t think any of this would be a huge business, but it’d be enough to fund BitTorrent development. Customers: anyone hosting downloads of more than 10 megs.

tech • bittorrent
 

Good BitTorrent windows client

The BitTorrent experimental download client is good. Installs with little fuss, and when I click on a .torrent link it just works. Nice UI to show you how the transfer is going with simple settings to throttle upload bandwidth. If you’re a Windows user and haven’t used BitTorrent, this is the place to start.

Game demos are starting to be distributed via BitTorrent. Perfect use: lots of enthusiasts, giant files (200 megs is common). Places to go for legitimate BitTorrent game files: GameTab, 3dgamers. Try out the cool Tron demo!

tech • bittorrent
 

BitTorrent survey, thoughts

I used my BitTorrent dumper to survey what was out there. What I learned is that BitTorrent isn’t a file sharing network, it’s a transport. Discovery and download initiation are still highly centralized. Every shared file is managed by a single tracker host that plays an active role helping find peers to download from. I fear most .torrent files are short-lived.

I did a survey from three sites of dubious legality: torrentfiles.com, animetorrents.com, and torrents.co.uk. In 870 .torrent files I found 32 different trackers. The top 4 trackers accounted for 75% of all the files. Not a lot of diversity. Of course, my sample is biased.

I was also interested in how folks use the BitTorrent metainfo. About 80% of all files use a piece length of 256k, followed in popularity by 512k and 1024k. I also found a bunch of unofficial tags: path.utf-8, creation date, comment, and md5sum.

I’d like to do a more formal survey with a wider sample; this torrent search engine claims 8200+ files.

tech • bittorrent
 

BitTorrent dumper

I was experimenting with BitTorrent, so I wrote a little file dumper to see what was in the mysterious .torrent files. The code doesn’t just parse the protocol; it’ll dump whatever the decoder can find. Sample output: 
info                
  length          41470132
  piece length    262144
  name            Halo2_E3.wmv
  pieces          [159 SHA-1 values]
announce        http://news.gametab.com:6969/announce
The format seems too simple in one way - fixed-size chunks. I’d expected some sort of recursive file definition. It’s also too complex in one way - I don’t think multi-files should be in a file transport mechanism. And really the .torrent file should be XML.

But it’s easy to criticize; BitTorrent is still awfully cool.

tech • bittorrent
 

BitTorrent - P2P without the search

I finally tried out BitTorrent, Bram’s P2P file sharing app. The clever innovation in BitTorrent is that it separates the search for files from the download of files. Unlike KaZaA or Gnutella or whatever, BitTorrent only supports file download, not search. You just use the web to find the .torrent locator file you need for download, for instance for the Half Life 2 trailer.

BitTorrent is pregnant hackerware - it works great but is still poorly documented. And the apps aren’t well integrated. Someone could build a killer tech company out of it. Some extra info is available in Brian’s BitTorrent FAQ and the BitTorrent Wiki.

Me, I started downloading by doing

apt-get install bittorrent
btdownloadcurses hl2_trailer.torrent
tech • bittorrent