In today’s Stratechery Weekly Article — which is insightful as always from a purely business analysis point of view — Ben Thompson makes a mistake he and many other business analysts often make. In discussing the history of Instagram, he notes (as he often has in the past) that there is a significant difference between what users say they want when asked and their actual behavior in practice:

Facebook [which owns Instagram] knows better than anyone that, when it comes to their services, revealed preference  —  what users actually do  —  is a far more powerful indicator than stated preference  —  what they say they want.

No doubt this is true, so far as it goes. Facebook and other social media networks have acquired immense amounts of data backing up this thesis. Users say they want a non-algorithmic feed, but they scroll more, look at more items, use the app more, when presented an algorithmic feed. From this data the social media giants (and commentators like Thompson) conclude that people in fact actually want what the algorithmic feed gives them.

Now, it is common in just the sort of economic analysis that Thompson mostly does to treat human behavior as a reflection of rational actors carrying out their actual desires. No doubt there is some degree to which this economic analysis is accurate. Our conscious desires and what we actually do often differ. One has only to think about our eating and exercise habits to see this! This is not a new problem, and it certainly extends into every aspect of our lives.

Unfortunately, though, many people operating from that economic frame go beyond simply observing the difference in stated preference and actual behavior. They make our “revealed preferences” normative. This is dangerous nonsense. It leads companies to ignore what users say they want in favor of what they “reveal” they want even if the latter is simply bad. And why? Because our revealed preferences are often maximally consumptive, maximally corruptible — and therefore maximally easy to abuse in the pursuit of power, wealth, or both.

We all know this to be the case — even setting aside my Christian presuppositions about human nature. Every parent knows that the revealed preferences of children are terribly self-destructive. Much of the work we describe as “growing up” is learning to exercise self-control, that is: to reject what our revealed preferences show our natural inclinations to be. To that I would add the aforementioned Christian presuppositions: we are fallen, which means that we should expect our desires to be distorted and warped. We spend a great deal of our lives trying to reshape our desires: to come to want what is good, to habituate ourselves to virtue.¹

When social networks choose — and make no mistake, it is a choice — to ignore our stated preferences in favor of what our behavior reveals to be our nature, they are being foolish at best and wicked at worst. It is long past time we stop treating what economic analysis of human behavior reveals as normative. Often as not it simply shows how broken we are and how much care we ought to take in the design of tools and systems lest they simply lead us further astray.

---

A parting note: Thompson knows this. By and large, he intentionally sticks to economic analysis, but he sometimes takes a step back and considers the kinds of things I have outlined here. When he does, he usually calls out just the sorts of problems I have! Unfortunately, though, his stated pattern of sticking to the business analysis means that he often at least implicitly reinforces the norm of “revealed preferences are right” that is so pernicious. We cannot do business or economic analysis in a vacuum. That social networks have successfully weaponized our fallen human nature over and against our stated preferences is to be lamented and condemned, not admired or even simply explained.

---

Notes

1. This is not something we can do by sheer force of will, but this post is not the place for an excursus on the doctrine of sanctification. ↩︎