Way back in 1994 I wrote an undergraduate thesis for my math degree at Reed College. It was a fun project, studying a discrete dynamic system that was an extension of the Ising model. Sort of cellular automata meets statistical mechanics.
A few years ago my colleagues from the Santa Fe Institute wrote a preprint from that work, Vortex Dynamics and Entropic Coulomb Forces in Ising and Potts Antiferromagnets and Ice Models. They were kind enough to list me as an author even though I barely understand half the paper! I do have the pretty pictures, though, plus a healthy appreciation of the complexity of discrete systems.
I've never met one of the co-authors, Cosma Shalizi. But thanks to his having a weblog I now know more about him than the other guys.
Starlink is oversold in North America. I've had the service since March 2021 and it's mostly great. But every evening it slows down. On bad nights I can't watch a single 1080p video stream reliably. Over half of Starlink customers report problems. Starlink's speed test app now admits "the network may be affected by slower speeds during busy hours". As if that were OK.
Overselling capacity is a common problem with American ISPs. More customers means more revenue and if customers get a crappy experience? Too bad, there's no regulation to stop them. Starlink has a serious financial challenge so of course they have an incentive to oversell. And service quality is likely to keep getting worse. Their user growth is accelerating and the new RV service means literally anyone can buy a dish now without waiting (albeit at a lower service tier.) They are adding capacity but their growth plan hinges on the troubled Starship launch vehicle.
Customers were promised better. Starlink was advertised as offering 100-200Mbps and 20ms latency; their legalese description promises 50-250Mbps / 20-40ms. My reality is speeds drop to 10-20Mbps every evening. Upload speeds are tiny, often well below 5Mbps. 20ms latency is a fantasy; 50ms is typical. And capacity is highly variable minute by minute, a technical challenge for rate limiting protocols.
The US government is giving Starlink $900M to sell rural Americans 100Mbps download / 20Mbps upload. But Starlink is delivering just a tenth of that download speed during peak hours and nowhere near that upload speed ever. I hope the FCC RDOF contract includes measured performance targets.
I am still grateful for Starlink, it's significantly better than anything else I can get in Grass Valley, CA. But they're making a business decision that's bad for customers. It's a reminder of how important it is to have Internet competition. Investing in wired infrastructure is as important as ever.
I’m back to using Goodreads but I lost my friends list, if you use Goodreads please add me as a friend.
After Goodreads lost my account I was furious and of course intended to never use it again. But to their credit, Goodreads did some extra work and managed to get me a copy of all my lost review data that I could import. And in the meantime I’d come to really miss Goodreads and decided I’d prefer to go back to using it.
After my experience I got interested in the IndieWeb idea of POSSE, Publish on your Own Site. You publish content like short updates or book reviews on your own infrastructure, then syndicate it to Twitter or Goodreads as appropriate. I like the spirit of owning your own data. But it requires you have your own infrastructure to publish to. I set up a basic Hugo blog for book reviews but it just doesn’t cut it. Goodreads offers so much more. Reviews are published in the context of a database of books, quickly crosslinked by author and genre and with publishing data and covers. Also Goodreads is social, reviews are shared with friends. That second part is why I didn’t switch to The StoryGraph; it’s a promising product but the community is small.
Still I’ve learned a hard lesson; your data isn’t safe on any cloud service. Making backups of your data is a good idea, I now have data dumps from 15 services. Although you still need a product to make that data useful. I’m still curious how Goodreads lost my data, they haven’t told me. One thing this whole debacle got me thinking is how dangerous account deletion is.
Consumer websites need to be very careful about data deletion. There's a risk of an account being hacked and deleted without the owner's consent.
The GDPR includes a right to erasure, California's CCPA has a right to delete. These are good laws, they allow an individual to require a company delete all personal data they have on someone. However this right also contains a risk. What if someone unauthorized requests the deletion? Proper deletion cannot be undone, in theory even backups should be deleted.
One solution is to delay the deletion and make every effort to contact the user before it's done. Some users might interpret the delay as the company acting poorly but I think it's an important protection against accidental or malicious deletion. Facebook has had a reasonable system for this for many years now; when you delete an account you have 30 days to change your mind. As a side effect some Facebook users keep their accounts in a perpetual state of almost-deletion, the super-logoff. Even better if the user's data is hidden while in the delete-pending state.
I don't know the legal niceties of whether a company can inject a delay. The GDPR language talks about "without undue delay", which seems to leave room for a safety net. CCPA is explicit about businesses having 45 or 90 days to "respond to a request to delete".
This whole post is motivated by my Goodreads disaster. One explanation for what happened is someone could have hijacked my Goodreads account and then deleted it to hide their tracks. At first I was outraged my data could ever be deleted. But Goodreads would be correct to do that in response to a valid request for deletion. And it looks like Goodreads will delete irrevocably immediately. (I'm not certain.) If they'd put in a 30 day delay I would have noticed in time. Speculating about this scenario made me realize that instantaneous deletion is a dangerous feature for any product.
The best of the data exports comes from Google Takeout. They were a pioneer in making a proper product out of data export and the Google Data Liberation Front did a lot of activism both within Google and externally to sell the idea. It's not an obvious thing for a company to do; letting customers download all their data opens the door to competitors. But it's the decent and right thing to do and it allows your power users to do complex things without much support.
Data export is also increasingly the legally required thing to do. The GDPR enshrines a right to data portability in the law governing businesses in the EU. California's CCPA also has a data access right. It's a little weaker than GDPR's but a lot of sites seem to just provide GDPR to everyone, or at least to Californians. These are excellent regulations; they protect consumers and enable competition. They do put a regulatory burden on the companies implementing them but it's not too huge and the technical infrastructure has other uses too. (Imagine, Goodreads could have backups of user data!)
One thing I hadn't appreciated is how hard it is to build something to use the data. Recreating a product like Goodreads or Gmail is a lot of work! In practice the exports seem most useful when some other commercial service is designed to import them. There's not a big ecosystem of open source tools to work with export data. Some of the data exports I got are pretty rough, low level dumps in CSV or JSON format. Then again Twitter has a whole working live webapp, you can browse and search nicely formatted tweets right from the files.
My Google data is the most valuable to me; I wrote up notes on what I found in my 67GB export. It's impressive; Takeout covers some 50+ Google products, many of which have done a thoughtful job making their exports another designed product feature. Not only did the Data Liberation Front get the company to export the data but they created an infrastructure and culture of supporting and improving those exports. It's a good thing.
Goodreads lost my entire account last week. Nine years as a user, some 600 books and 250 carefully written reviews all deleted and unrecoverable. Their support has not been helpful. In 35 years of being online I've never encountered a company with such callous disregard for their users' data. Update: Goodreads gave me a recovered copy of my data
Do you use Goodreads? Don't trust them with your data. Protect yourself with a backup; use their data export service right now. Consider quitting Goodreads entirely. LibraryThing and The StoryGraph are promising competitors. This blog post also has some ideas on DIY indieweb alternatives.
Don't trust any cloud service with the only copy of your data. Most companies are not quite so reckless but consider what you'd miss if an uncaring company lost your data. Many of the better services have data export products; Google Takeout is fantastic, Twitter has good export, as does Facebook and Instagram and Letterboxd and others.
I've enjoyed using a product like Goodreads. My plan now is to
host my own blog-like collection of all my reading notes like Tom does. It will be a lot
of work to set up. Fortunately not all is lost, I happened to take a
data export last July and I can recover some of the more recent data
from emails they sent to my friends.
For anyone wondering how Goodreads could have simply lost all my data, I'm wondering too! It bespeaks contempt for users. And terrible system design, services should not be able to lose data irrecoverably. The specific bug is related to my removing Twitter API access to Goodreads last week (they stopped supporting Twitter login months before). Somehow that triggered their system to delete everything. Goodreads promises me it was a true delete, the data is wiped from their database. I don't believe this: sites generally flag data as deleted, they don't actually remove it. Goodreads also ignored my request to restore my data from backup. Either they don't have backups or they can't be bothered.
I've learned a hard lesson in trusting cloud services. Unfortunately just having a copy of your data isn't enough; it's a lot of work to build a useful product. In the meantime I will be more careful about which companies I trust. Goodreads has been in decline ever since Amazon bought them in 2013. Apparently an anti-competitive purchase, not a strategic acquisition.
The best thing an individual American can do right now to fight global warming personally is to switch to electricity for all your energy needs. Your next car should be electric and your next home heater and water heater should be heat pumps. Rooftop solar is an excellent idea too.
That’s the central argument in Electrify, Saul Griffith’s new book about global warming (Amazon). He’s been talking about this for awhile; Electrify covers a lot of the same ground as his free self published Rewiring America and his Make articles.
The most interesting part of the new book isn’t the personal advice but rather his global plan for how the whole world mitigates global warming. He starts by pointing out how urgent the problem is: we have to start doing more right now, this very year, and there’s no time to wait on new technologies. Electricity is the best form of energy for transportation and storage. The basic idea is to shift from fossil fuel to electric consumption while in parallel adding more carbon-neutral electricity sources to the grid. He argues we’ll need 4x as much electricity in the US to achieve full electrification but this is a huge net gain (50%ish) in both total energy consumption and actual costs. He advocates for an effort akin to World War II mobilization to get it done, financed with low interest debt.
What I like best about his argument is it breaks the Gordian knot about “what can we do”? Electrify now and work on adding clean energy sources. I also like his holistic clarity, he really looks at whole-world energy consumption and economics. The optimism is great too. I find the argument convincing.
That being said, I also think global warming is a terrible problem that world politics won’t solve in time. We’re at 1.2C warming now. The Paris goal of 1.5–2C seems impossible. 3C seems inevitable and I suspect we’ll get to 4C at least before warming starts to reverse. That’s an enormous global catastrophe so big I think we have to consider geoengineering in parallel to decarbonizing to stave off the worst effects. As the book points out there's a risk people will see geoengineering as an alternative to decarbonization. It's not, but it might be a band-aid to give us time to get CO2 under control.
I'm a published cartographer! I contributed a couple of static maps of Enga Province in Papua New Guinea for the academic press book The Absent Presence of the State in Large-Scale Resource Extraction Projects. They're in chapter 3, Restraint Without Control, which talks about the interplay of the mining industry and local tribal fighting. You can read more about it on author Alex Golub's blog. There are high resolution versions of the maps on my site.
Alex is a friend from college, he knew I was interested in mapping and asked me to contribute some custom maps to help locate his narrative. PNG is a famously remote country and poorly mapped. Even in the modern era PNG map quality is very uneven. SRTM makes the topography map easy. But maps of human features are harder to come by. Google Maps and Bing Maps have very limited data. OpenStreetMap is much better and there are some specialist maps that are excellent.
I wrote up a bunch of technical notes on making the map back when I did the work. QGIS and OSM were both essential.
I had an idea it'd be nice to make maps for Wikipedia, one per PNG province. But the thought of dealing with Wikipedia politics was overwhelming. If there's a map advocate / mentor who would work with me I could probably get it done.
I felt physically ill reading this story about Facebook’s facilitation of murder and slavery.
... reports from employees who are studying the use of Facebook around the world, including human exploitation and other abuses of the platform. They write about their embarrassment and frustration, citing decisions that allow users to post videos of murders, incitements to violence, government threats against pro-democracy campaigners and advertisements for human trafficking.
There’s been plenty of stories about Facebook’s malfeasance over the years. Two things make this story particularly awful. One is the human scale of the harm they cause. It’s not some abstract discussion about political influence, it’s personal examples like a woman named Patricia being recruited and sold into slavery. The other is that Facebook knows about the problems and is choosing not to act. Or at least not act enough to be meaningful.
Facebook treats harm in developing countries as “simply the cost of doing business” in those places.
I had enough. I rage quit Facebook Thursday. Or at least tried to.
The problem is I’m a captive of Facebook. Because despite all the horrors it’s still a good semi-private way to keep in touch with people. It’s my primary social connection to the small gay community in Nevada County, for instance, including a group that organizes weekly meetups. Also it’s helped me reconnect with old high school friends. I’m well aware of the hundreds of other social media tools I could ask them to use, I helped design some of them. But the reality is that a lot of community happens on Facebook and if you don’t participate there, you miss out.
I don’t know what I’m going to do with Facebook. I was going to delete my account but feel I can’t. I’m trying to stay logged out but I already feel the need to connect occasionally, even if only to arrange social connections outside of Facebook. I’m one of those extremely online people, I can’t just disconnect entirely. But I’m being forced to visit the house of a psychopath.
This story this week is one in a series of investigative articles by the WSJ. Some Facebook employees have been trying to lessen the harm their company is doing and they’re tired of being ignored. So they’re talking to reporters, particularly Jeff Horwitz. As a collection the reporting is incredibly damning. Lying to their oversight board, ignoring mental harm to young women, amplifying anger and lies, sabotaging American vaccination efforts, and helping the business of murderers and slavers.
Facebook is in some ways just reflecting the larger evils of society. I’ve worked on social media policy. I understand the difficulty of moderating conversations. But as a medium Facebook is a very efficient amplifier of evil; its existence uniquely enables things like the genocide of the Rohingya. That creates an obligation on Facebook to mitigate the harmful uses of their product. They have failed to that. Maybe the only remedy is to stop them from operating.
Starlink is good technology. I’m posting this blog entry from space. By which I mean Starlink, SpaceX’s new LEO satellite Internet service. I’ve been beta testing it for six months and using it exclusively for two. It is terrific. Some notes from a United States perspective.
Does it work well? Hell yeah! It’s more like having cable Internet than satellite. Over the last month my average bandwidth has been 100Mbits/s down, 12Mb/s up. Average ping to 220.127.116.11 of 41ms with 0.5% packet loss. Bandwidth is highly variable (50-200Mbps) but latency is pretty solidly 30-50ms. The main failure is occasional outages of ~10 seconds. That got a lot better mid-July with a Starlink change and will only get better as they launch more satellites. The service is still technically beta and there are some rough patches but it’s totally usable.
Is Starlink for you? Maybe. If you can get wired service (cable, fiber, faster DSL) that is probably a better choice. If you’re in a poorly served rural area in Ajit Pai’s America and you’re struggling with ViaSat or Hughes or using cellular, definitely. For me it’s an upgrade to my 12/1 Mbps fixed wireless service.
Will Starlink work at your house? Probably! It requires a clear view to the north. The free Starlink mobile app has an augmented reality tool to show you whether you have a good view. A few small obstructions are OK but if you live in the middle of a bunch of trees you need to go higher on your roof or get a mast.
Can you get it? Probably not soon. They are enormously back-ordered; Starlink has a limited amount of bandwidth per satellite and they are slow to add new users. They just passed 100,000 installs globally and are rumored to have 500,000+ customers on the waiting list. The best thing you can do is pre-order and put down a $99 deposit. It may be a year. (I got super lucky.)
Is it nerd friendly? Totally. You can use your own router; Starlink provides one but does not require it. Dishy has an open gRPC interface for getting detailed stats. The Internet service is quite solid and not messed with in any way I can tell. They sorta support IPv6 already and promise more. The main drawback is that (at least in IPv4) the service is cgNAT, you really can’t run a server behind Starlink in any reasonable way. The cgNAT is for good reason: they’re doing some very sophisticated routing, your packets may be relayed through several base stations hundreds of miles apart and it’s remarkable you have a stable external IP address at all.
Will Starlink succeed as a business? That’s hard to say. The program is still beta and currently has no bandwidth caps or significant throttles. And it’s $99/month: a lot by US urban ISP standards but competitive for rural areas. The problem is the satellites can only handle so many users and it seems too early to tell whether it works out to be profitable. Launching thousands of satellites is expensive but SpaceX are experts at that. Also crappy companies like ViaSat keep suing to stop Starlink because they can’t compete. Amazon is trying to make up for being years behind by trying to get the FCC to harass Starlink. The legal attacks seem to be failing so far in the US but you never know.
What’s next for Starlink? They launched their first full shell of satellites earlier this year, then took a pause. They seem to be fine-tuning algorithms and transceiver power settings right now. The next major change is a plan to use laser links so satellites can route packets directly (currently everything is relayed to the ground). It seems like a very hard problem but they are serious about doing it; ultimately Starlink might be better than wired service.
I’m not an Elon Musk fan but I have to say Starlink is amazing. And audacious; I never would have believed it would work (remember Iridium?). But it does work and it’s been a significant upgrade for me, so thank you SpaceX. The hilarious thing is the whole idea of Starlink apparently is about Mars; the project started as a way to design networking infrastructure for a colony on a new planet. Oh and then completely upended Earth’s ISP market as a sort of proof of concept.