(This was originally a comment on the preceding thread. I decided to promote it to blog entry, just because.)

If you've read Season of Skulls you probably guessed I did some research on travel in England in 1816 (plus before and after). So, some notes follow ...

The 18th century saw a marked improvement in British roads, culminating in the spread of macadam surfaces (albeit not tar-sealed) replacing earlier dirt/gravel tracks and roads paved with unmortared stones. Stage coach services also matured, and turnpikes and well-maintained highways were largely walled off from woods to either side following enclosures -- as a result, the prospects for highwaymen became perilous (it was a capital offense, and they couldn't leg it into the nearest woods, and the coaches were rolling faster). Upshot, highway robbery fell off drastically after 1800 (although there was a spike after Waterloo, as lots of penniless discharged soldiers found themselves at a loose end).

In 1810, a stagecoach or mail coach ticket from London to Edinburgh would cost roughly a month's wages for a servant, and the journey would take 48 hours or so (depending on the weather). Seats on the outside of the coach were of course cheaper than the first class seats with a roof over them, and you'd be shaken down repeatedly for tips by the guards -- this could easily add another 50% to the ticket price if you were going the distance.

Railways were barely a thing before 1830, but by 1860 they'd changed everything. Again, same sort of pricing (a month's wages to cross England from end to end), but the journey would be far faster (with express trains running at up to 60mph) and everyone got to sit inside, even if it was on a wooden bench in an unheated third class carriage.

So your go-to metaphor is: England in 1816 was about the size of the globe in 2016, in terms of cost and duration of travel (we went from stage coaches to jumbo jets but the time taken en route, and the chunk of your income it consumed, remained roughly the same).

(Isochrone maps are cool!)

Now, recall that nowhere in Great Britain is more than about 40 miles from the sea. But sea/river/canal travel was powered by wind, oar, or a horse walking along a tow path. Good for bulk transport and freight (you can cram a lot more stuff into a ship's hold than you can put on a horse-drawn coach) but probably slower than the coach, and subject to weather delays due to storms or still air.

By 1840 steam engines had begun to change this, and again, by 1860 there were coastal steamers.

Finally, poor folks would walk long-ish distances that today we wouldn't dream of unless we were doing a recreational hike. My father (who would be 100 next year if he was still alive) used to walk between Leeds and Dewsbury in the 1930s -- that's about 12 miles -- to go to market. And my great-grandfather, per family legend, met my great-grandma in Poland (in what was then part of Russia) on the return leg of a trading trip (he was a wool merchant, buying and selling stock from town to town) that took him overland from western Poland to Tehran and back in about two years in the 1870s. Mostly on foot! (Or riding with the stock in trade.)

TLDR: some folks walked very long distances even before rail or steam arrived. Many people walked distances we'd consider driving/commuting range today for stuff like markets or fairs. But the world was, on balance, far larger than it is today -- travelling from England to the Americas before steam would be as daunting a prospect as visiting Dilbert Stark's Mars base in the 2050s (if his Starship reusable interplanetary launch system works out as planned).

I continue to work through my worries about where whe are. I’m still thinking about where I stand in an industry that no longer seems to care about what it makes. It’s become obvious that the software industry doesn’t care about software.

It’s only getting worse

Of course, I’m biased since I both wrote a long essay on this a couple of years ago and an entire book last year.

My theory, which I outlined in the book, is that software quality and business success, especially for individual managers, have been completely decoupled. Insisting on software development methods that are known to reduce defects and improve User Experience is more likely to harm your career than help. High-level managers and executives get rewarded for project size, not long term success. The software industry has never been particularly good at what it does, but it’s been getting worse.

Just look at all the lay-offs.

Any executive that has led their company to a situation where they have to lay off 10–20% of their workforce is by definition incompetent.

They’ve led their company into a horrifying financial and organisational disaster with detrimental consequences that will echo on for an entire decade. Their management put the company in dire straights. In a rational market, they would have been forced to resign, if not outright fired.

Instead, they became a role model. Just as with all the other incredibly poor software development decisions these companies have made, the rest of the industry decided to follow and copy them.

And, their executives get rewarded as well.

Basically, that our software is deteriorating should not be a surprise to anyone. It’s an irrational market, governed by people who don’t know what they’re doing.

The AI frustration

Nowhere is that more evident than with AI. A few weeks ago Google’s CEO, Sundar Pichai, went on “60 Minutes” and made a number of glaringly false statements that either belies a complete lack of understanding of the technology he was promoting, or complete dishonesty.

I prefer to think he was just being ignorant, but that just emphasises my point that these people should all have been fired.

I’m getting a bit off track here, but the software industry is going down a path that is just so very disappointing.

Watching executives making promises based on a fundamental misunderstanding of the tech they’re hyping, while every single one of their actions seems tailor-made to increase defects, bugs, and degrade user experience is…

Frustrating is what it is. Frustrating.

The one consistent aspect about this deterioration of the industry is their inability to think about the larger context, specifically about the various social contracts that govern tech’s position in society.

With AI, tech has broken the web’s social contract

The problem is that quite a few people in tech don’t believe in any social contract. Their conceptualisation of society is that it’s an equilibrium of dominant wills motivated by mimetic desire. That the rich are on top; that the rest of us aspire to be like them; and that any and all criticism towards them is born from jealousy. The world can only be improved by those with power over others. Any form of pro-social reasoning, consensus-building, or genuine negotiations seems to be alien to them.

These people are reactionary libertarian assholes, and they are tech’s ruling class. They might see themselves as benevolent shepherds of humanity’s future, esp. the creepy longtermist types, but by and large, they are power-hungry libertarian assholes.

This is why they leave scorched earth behind.

It’s not a secret that much of what the tech industry has done has what economists would euphemistically call “negative externalities”.

Microsoft’s inability to manage software defects meant that, for close to two decades, society had to bear the cost of dealing the fundamentally broken security of most versions of Windows. Their incompetence created a $4 billion USD market for antivirus software at the “peak” of Windows’s insecurity in 2004 and nobody knows how many billions of actual costs to society from software virus infections and hacks.

(In a former life I worked for an antivirus software vendor. If anything, we’re vastly underestimating the destructive cost Microsoft inflicted on our economies with their security incompetence during these years.)

It just keeps going on.

Android is a mess of malware and unpatched phones.

AirBnB seems specifically designed to increase property prices beyond what locals can afford.

The ‘gig’ economy is designed to completely undermine general pay and job security.

Their advertising-oriented universal surveillance is the authoritarian government’s wet dream. There are too many examples to fully list out.

But AI is the latest.

The old social contract

Now, the deal with the web has generally been rather simple:

• We and media companies put stuff up on the web for free. Some of us do it for business reasons. Some of it is personal. Much of it is just culture. People being people.
• Tech companies use this stuff, but mosty in an economically complementary way. Search engines use it to help you find what you’re looking for. Social media sites show you ads. Some of it is processed in order to improve the services they provide, such as spellcheck or autocomplete.

This has been breaking down for a while. Translators got hit hard by translation software, and that was only possible by processing texts made by human translators.

Google has steadily been manoeuvring their search engine results to more and more replace the pages in the results. Some of this has resulted in lawsuits or outright legislation. Many of Google’s legal issues in the EU stem from this shift.

Tech’s universal surveillance has also pushed the boundaries of what many found acceptable. Even my parents use ad blockers now.

But, language and diffusion models go further. The deal tech is offering us there is also simple:

• We and media companies put stuff up on the web for free. Some of us do it for business reasons. Some of it is personal.
• Tech companies use this stuff to create systems that can make shoddy, degraded versions of our work, deepfake us, and make convincing fake only personas for astroturfing, destroying our work, businesses, and social interactions.

This is a bad deal.

This is not a remotely fair deal for those of us on the “putting stuff up on the web for free” side of the equation. It doesn’t matter whether it’s illegal or not—though legislation is probably the only way to get the tech industry to stop it—because the social contract is broken.

Our incentive recedes in lockstep with the increasing dominance of generative AI content. As it recedes, fewer and fewer people and organisations will contribute to the digital commons. More and more stuff will be locked behind a paywall.

This has already affected my relationship with the web.

In terms of percentages, I posted fewer extracts from The Intelligence Illusion online than I did Out of the Software Crisis. I’ve also all but stopped posting photographs like I used to. I may be blogging more, but I’m also worrying more whether I should.

I’m at the point in time when, under normal circumstances, the strategic thing to do would be to push on with writing more online. It would normally be the most effective way to improve my career under the circumstances.

But now there’s a question mark against all public writing.

Do I gamble that the flood of language model texts will put a premium on thoughtful writing? That I’m not just improving the models by putting more writing out in the world?

Do I figure out ways of putting more of my writing behind some sort of pay- or login wall, even though that would be counterproductive for my career? Wouldn’t that also just disconnect me from my friends and the online community in general?

Do I just ignore the fact that I’m helping train the generative pap that tech hopes will replace us all?

I don’t know.

I know I’ll keep writing about it. Just writing this post helped clarify my thoughts and put order to my emotions.

Will I continue to post my writing online?

I hope so. I hope online writing survives this.

But I’m no longer certain it will.

The best way to support this blog or my newsletter is to buy one of my books, The Intelligence Illusion: a practical guide to the business risks of Generative AI or Out of the Software Crisis.

For many people the first word that comes to mind when they think about statistical charts is “lie.”

– Edward R. Tufte

When I moved from engineering to management, people expected me to make charts.

After spending some time learning about data visualization, I’ve come to two important conclusions:

1. Good data visualization is powerful
2. Powerful data visualization is rare

But creating a compelling chart is an underrated superpower for engineers. Here are some ideas that helped me learn how to do that.

Tufte’s principles: a case study

Take a look at this chart from the Wikimedia Foundation’s 2023–2024 budget projections:

This is a default Google Sheets chart for this data.

What I’m able to glean from this chart:

• “Building analytics & ML Services”: the biggest, > 25%
• “Features and functionality” and “Supporting volunteers”: smaller, ~20% each
• “Fundraising,” “Protecting access,” and “General & Admin”: smaller still, ~10% each
• The colors seem meaningless

Now consider what we glean from the table used to generate it:

Program	Budget (millions)	Percent
Building analytics & ML services	$46.4	26.2%
Features and functionality	$39.7	22.4%
Supporting volunteers	$35.1	19.8%
General & Admin	$21.3	12.0%
Fundraising	$17.9	10.1%
Protecting access	$16.6	9.4%
Total	$177.0	100.0%

This table gives us more information than the chart in a similar amount of space.

From the table, we learn:

• Exact percentages—no need to guess
• Exact dollar amounts
• The total budget as a dollar figure

And we get rid of the meaningless colors.

OK—confession time: there were percentages on the original pie chart. I edited them out.

But I did that to prove a point. The pie chart is doing less work than the numbers. The slices of the pie add almost nothing to the numbers.

Tufte would prefer the table above to the pie chart above because the table:

• Shows the data
• Maximizes the data-ink ratio
• Avoids chartjunk (like the meaningless colors)

And in this case, I agree: I like the table.

But choosing data visualizations is difficult.

How to choose a chart: visual perception accuracy ranked

If you intend to communicate, then people must be able to read your chart.

In 1985, Cleveland & McGill conducted what is still the most cited experiment on what charts are easiest to read. Their purpose was simple: rank standard charts by the number of errors people make while reading them.

Here’s their ranking of charts—from easiest to hardest to read:

Icon	Name	Example
	Position along a common scale	scatterplots, bar charts, sparklines
	Positions along nonaligned scales	stacked bar charts
	Length, direction, angle	pie charts, donut charts
	Area	bubble charts, treemaps
	Volume, curvature	3d charts
	Shading, color saturation	Heatmaps

But I love heatmaps! You can use heatmaps. Just understand that only a sophisticated audience can interpret heatmaps correctly.

Know your audience, then use something like the data viz catalog to choose your chart.

Click here to go see the bonus panel!

Hovertext:
Look the fact that there's a misspelling in the votey panel just makes the point more strong.