Weakness of Will & the Engineered Self · Episode 6 · 25-min read

The room was built to beat you

Transcript · audio coming

It is two in the morning, and the phone is in your hand. For a moment, set aside the argument you are having with yourself about whether to put it down, and look instead at the object itself. Not at yourself — at the device.

Consider what it is. A screen. A feed with no end. A small bar at the top that you draw down with a thumb and release, and it turns, and something new appears. Ask who decided that this object should be the most interesting thing in the room at two in the morning — brighter, more insistent, harder to set aside than sleep. Because it was decided. None of it is natural, and none of it is accidental. Every feature of the device was designed — the colour of the notification dot, the exact delay before the next item loads, the slight resistance as you pull to refresh and the release as you let go. It was designed by people, and it was designed to make money.

And here is the thought this episode will develop, which many listeners will find a great relief. Suppose the thing that failed at two in the morning was never inside you at all. Suppose it was in the situation — built into it, deliberately, by people who are paid when you lose. Last week, inside the self, we found a division — a place where the self comes apart. This week we ask who has studied that division, measured it, and built a device to locate it in millions of people at once and widen it, a little, every night. The contest was arranged in advance. And once we see how thoroughly it was arranged, the word we began with — weakness — begins to look misplaced.

That relief is earned, and much of it is simply true. But we should be cautious for a moment before we accept it, because there is a difficulty underneath it that we do not reach tonight. Tonight we only describe the situation.

This is Philosophy for Us — philosophy for everyone, no degree required. This is the sixth of eight episodes on the oldest and most ordinary failure there is: knowing better, and doing otherwise.

Last week, Harry Frankfurt put a structure where the word "willpower" used to stand — a self with levels, and a division between them, at which the wanting a person endorses loses to the wanting they disown. Gary Watson declined to say which level is the real person. We were left with a harder question and no verdict. This week the question turns outward, from the self to the situation: if two in the morning was built to defeat us, how much of what failed there was ever ours? And a word that has been present the whole time — "willpower" — turns out to have a history and a date. Before the hour is out, we will find it.

One further point before we begin. I am not setting myself above anyone here; I lose this contest too, and the device I am about to describe has measured me exactly as it has measured you. So this is the two of us examining, for once, the outside of a shared difficulty — the thing that was built to produce it — and asking the question no one who is selling us anything wants asked: how much of this was the situation, and how much of it was still us.

Let me describe the situation, beginning with the single most important fact about the device — the one that explains why the advice to "just put it down" has never worked. Not because the person is weak, but because the device is exploiting a law of behaviour that was understood long before there were phones.

Return to the middle of the twentieth century, and to the psychologist B. F. Skinner, who worked with a box containing a lever and a hungry animal. Skinner was studying something simple and fundamental: how reward shapes behaviour. Place an animal in the box, and each time it presses the lever a pellet of food is delivered. The animal learns quickly. Press, and eat. Press, and eat. The reward is predictable. Now here is Skinner's finding that bears on two in the morning. If you want to establish a behaviour that is almost impossible to extinguish — one the animal will repeat long after the food has stopped, repeat to exhaustion, repeat against its own interest — you do not deliver a pellet every time. You deliver it sometimes, unpredictably. Press, and nothing. Press, and nothing. Press, and a pellet. Reward on a schedule the animal cannot predict.

That single change — from every time to sometimes, unpredictably — is the difference between a habit one can break and a habit that does not release its hold. Skinner called it a variable-ratio schedule. The name is not important. What matters is that the most profitable object on a casino floor was built on this principle: the slot machine. The slot machine is profitable not because it pays a great deal, but because it pays unpredictably. You pull the lever; most of the time nothing follows; occasionally something does; and you can never tell which pull will be which. That uncertainty is the whole of the attraction. It is the most reliable method anyone has found for keeping a creature pulling a lever long past the point of sense.

Now consider what the thumb is doing at two in the morning. You draw the feed downward — the same small motion — and it turns, and something loads. Most of the time it is nothing: a post of no interest, an advertisement, a person you dislike expressing an opinion you did not need. And occasionally, unpredictably, it is something that engages you: a message, a photograph, something funny, something that makes you feel, for a moment, less alone. Nothing, nothing, and then something. You are not refreshing the feed because you want the next post; if that were so, you would have stopped many posts ago, since the next post is almost always nothing. You are refreshing for the possibility. That is the same act as pulling the lever. The pull-to-refresh gesture was designed by people who understood precisely what Skinner's box had demonstrated. It is a slot machine that fits in the hand, that is carried to bed, and that never closes.

Be careful here, because it would be easy to take this as a loose comparison — the phone is a bit like a slot machine — and pass over it. It is not a loose comparison. It is the same mechanism: the same law of behaviour, applied deliberately, because the people who build the feed studied the same science on which the slot machine was built. The variable reward is not an incidental feature of the feed. It is the feed. A feed that delivered something good every time would be less compelling, not more, because you would get what you came for and leave. A feed that delivered nothing good would also be less compelling, because you would give up. The feed is tuned to the same unpredictable schedule Skinner found most powerful: enough possibility to keep you pulling, never enough to let you stop.

Run two in the morning through this, and the word "willpower" no longer fits. A person comes to this blaming a faculty — a muscle, a reserve, something within that ran low. But what they were contending with was not a temptation sitting passively, in the way a slice of cake sits on a counter. It was an active device, engineered on a law of behaviour, and designed to defeat the very faculty they were told to oppose it with. "Just put it down" is advice suited to the slice of cake. It is useless against the slot machine, because the slot machine was built to defeat "just put it down"; that is its entire purpose. To tell someone to out-discipline a variable-ratio schedule is to ask them to succeed by effort against a device engineered so that effort will fail.

And here the relief begins, and I will not resist it, because the first part of it is honest. For years — perhaps for a whole life — whenever a person lost this contest, every voice around them said the same thing. The gym, the diet, the productivity application, their own reflection: you have no willpower, try harder. In fact they were being told to win, by trying harder, a contest engineered by people who had studied precisely how they would fail, and engineered so that trying harder would not work. That is not a fair contest repeatedly lost. It is an arranged one, with the loser charged the entry fee. And "try harder" was close to the cruellest thing that could be said to someone inside it, because it charges them, personally, for a defeat that was designed in a meeting they were never invited to.

The weight of that is real, and much of it lifts here — because for once one is not being deceived. The contest was arranged. "Willpower" was a cruel name for an arranged defeat.

So far this is a device and a law of behaviour, and one might still tell a milder version of the story: the feed simply developed this way, no one really intended it, it is an accident of the technology. That version is false, and the way it is false matters. There are people on the other side of the screen — people who studied our weak point deliberately, and who are paid, very well, to widen it. It was never an accident.

It is something most people never consider: someone designs the slot machine. It does not fall from the sky. There is a person, with a job, who decides exactly how fast the reels turn, what sound the machine makes when a player almost wins, how the seat is angled, where the button sits so that the hand need never leave it. And there is an entire industry of such people, who are very good at the work.

The anthropologist Natasha Dow Schüll spent some fifteen years studying them — both the people who design gambling machines in Las Vegas and the people who play them. She set out her findings in a book published in 2012, Addiction by Design, and the title states the argument. What she found in talking to the designers is the part that changes everything. One would assume the machine is built to sell the player a win — the jackpot, the lights, the moment of triumph. It is not. The designers told her, more or less openly, that the win is almost beside the point. What the machine is built to maximise is not the player's winning but the player's time at the machine. The industry has a term for it — "time on device" — and that is the figure the designers are paid to increase. Not the player's pleasure, and not the jackpot, but the minutes. They are engineering, with great care and real science, the longest possible stretch of a person sitting there and pulling.

Schüll also found that the players furthest into it are no longer pursuing money at all. They describe entering a state she calls the "machine zone," in which winning and losing cease to matter, and the object is not to win but to remain — to keep the rhythm going, to dissolve into it. Time recedes; the surroundings recede; the sense of self recedes. And the machine is tuned, to the millisecond, to produce that state and to sustain it. That is what the designers are for. That is the product: not a game one might win, but a state one cannot easily leave.

Now, everything Schüll documented on a casino floor in the 1990s was subsequently taken up, refined, and placed in our pockets. The people who build the feed do the same kind of work, and they have their own version of "time on device" — they call it engagement, or watch time, or daily active use — and it is the figure on which their careers depend. They run experiments on millions of people at once: this shade of red on the notification dot against that one, this delay before the next video against that one, automatic play on against off — and they retain whatever keeps people using longer. They are not guessing. They are measuring us, in our millions, in real time, and adjusting according to the measurements. The two in the morning that a person blamed on themselves is the output of that system operating correctly. When one cannot put the phone down, the system is not failing. It is succeeding. One's staying awake is the outcome it was built to produce — its success, not ours.

And it is not only the phone, because the same method was applied to the oldest appetite of all. Consider why it is so much easier to eat a second helping of a certain kind of food — the engineered kind, from the brightly coloured packet — than to eat too much of a plain boiled potato. It is not that a person has less self-control around the packet. It is that the food was designed, by food scientists, to strike the exact combination of sugar, fat, salt and texture that the human reward system finds hardest to refuse — a point the industry deliberately identifies, tunes for, and sells on. No potato in a field is doing this. It was done in a laboratory, on purpose, because a person who cannot stop eating the product buys more of it. The logic is the same as the feed's: find the weak point, and build a product that fits it.

So set the whole matter out where it can be seen. For five episodes we searched for the thing that failed at two in the morning inside the person — a faculty, a piece of knowledge, a divided will. Here is a candidate that was never inside the person at all. What failed was a contest between one tired individual, running on a brain that evolved to want sugar, novelty and connection, and an industry of highly capable people, with budgets in the billions, whose entire work is to locate that brain's weak points and build devices that fit them — devices that grow more profitable the more thoroughly they defeat us. That is not a problem of a faculty. It is a problem of a system. What failed is not one's willpower. It is the situation, the people who built it, and the money that flows to them when we lose.

And the relief described a moment ago reaches its honest height here. When a person lost, they were not weak; they were outmatched. They were one individual, alone, at night, against the coordinated, well-funded, scientifically optimised effort of some very capable people, all directed at the single aim of making them do the thing they had resolved not to do. And then those same people, or the culture that profits alongside them, told the person that the loss was a flaw in their character. That is not merely false. It is among the oldest manoeuvres there is: defeat someone by an arranged contest, and then persuade them that losing reveals a fault in themselves. Many people have carried that idea for years. Much of it was never theirs to carry.

That is the situation, then, fully described. And it points directly at the one word I keep using and keep promising to examine. Before we rest in the thought "it was never me, it was the word I was sold," we should find out where that word came from — who produced it, and when, and, above all, whom it was built to blame. The word has a history and a date. Let us go and find it.

I have been using the word "willpower" throughout, as though it were a word everyone has always had. It is not. It is far more recent than it feels, and once one sees how recent it is, one sees what it was built to do.

Consider the word "will-power" — the specific idea of a single inner faculty, a kind of fuel or muscle of resolve that a person possesses in some measurable quantity, that can be built up, run down, and found wanting. Used in that sense, it becomes common in the eighteen-seventies, with a few scattered earlier uses. I am not a historian of the dictionary, and I am not offering a precise date fixed beyond dispute. The point is that the word is recent — that "willpower," in the sense of a quantity one has a certain amount of, arose in a particular time and place, for particular reasons, and that human beings struggled with the difficulty of two in the morning for thousands of years before anyone reached for that word to name what had failed. So if it feels timeless — if "I have no willpower" feels like a fact about the human soul as old as the soul itself — that feeling is itself worth distrusting. Ideas that arrived as recently as this do not usually feel that way. It has been made to feel permanent.

Consider what came before it — figures we have already met. Take Augustine, from last week: sixteen hundred years ago, in acute distress, watching his own will divide in two. He did not say that he had insufficient willpower; the idea was not available to him. He said that there were two wills in him, both of them his, and that they were at war. The failure, for him, was not a low quantity of anything. It was a division — a whole self pulled in two directions, every part of it his own. Or go further back, to Aristotle, three episodes ago, and the man who has the knowledge but is not using it, in the way a drunk man can recite a proof without being guided by it. For Aristotle the failure was a matter of character — of a person not yet properly trained, whose hold on the better course loosens under pressure. Again: not a reserve, not a gauge, but a condition of the whole person.

Neither Augustine nor Aristotle has a meter. Neither imagines a single faculty, situated behind the forehead, that comes in amounts, that one might have more or less of than one's neighbour, and for a short supply of which one might be blamed. That meter — the reserve of willpower — is the new element. It is what was added in the nineteenth century. And what was the meter for?

Consider what happened across that century, the century in which the word rose. It was the century of self-help, of the temperance movement, of a strenuous ideal of self-mastery — improve yourself, master your appetites, the strong man is the man who conquers himself. In that atmosphere the older picture — a divided will, a character still being formed, a whole soul at odds with itself — was quietly reduced to something simpler and far more useful: a single personal faculty, the will, which one could strengthen by effort, as one strengthens a muscle, and for failing to strengthen which one could therefore be held personally responsible. The division Augustine wept over was reduced to a single measurement, recorded against one's own name. If you fail, the reading is simple: your reserve was empty. And an empty reserve is your own responsibility.

Set that beside the device described in the first half of the hour, and the two fit together — and here is something worth doing in one's own case. Trace the word. Take any word one reaches for in self-blame — "willpower," "discipline," "lazy," "no self-control" — and put three questions to it. When did it arise? What did it replace? And, the question that matters most, who benefits from my believing it?

Put "willpower" through them. It arose in the nineteenth century. It replaced a picture of a whole, divided self with a picture of a single, private reserve. And who benefits? Consider who is standing there when a person says "I have no willpower." The feed benefits, because if the failure is one's empty reserve, then it is not their engineered device, and one will return the next night to blame oneself rather than them. The diet industry benefits, because a person who believes their weight is a measure of willpower will buy the next programme, and the next, indefinitely, since no programme can repair a thing that was never the problem. The word "willpower" does a very specific job: it takes a failure that runs through a device someone built and a brain one was simply given, and files the whole of it under one's personal insufficiency. It directs attention at the single party in the arrangement who is not being paid. That is not a neutral description of what failed. It is a verdict — and a convenient one, for everyone except the person blamed.

And this is a habit of attention the series has taught before, in another connection. Earlier in this show, we learned to notice when a question that presents itself as a plain matter of fact is in truth a standpoint — a position from which someone is looking, presented as simply the way things are. On that occasion it was the question "but does the machine really understand?" — a question that seems like neutral curiosity and turns out to carry a whole theory of what minds are. This is the same move, heavier, and turned on oneself. "I have no willpower" feels like a plain report of a plain fact. It is not. It is a standpoint, with a date in the nineteenth century and a list of beneficiaries, and a person may have spoken from inside it for a whole life without once noticing they were standing anywhere at all.

So the relief is now complete. The device was built to defeat us. The very word we used to blame ourselves was built, in part, to keep us blaming ourselves rather than the device. From here it really does look as though the whole thing was done to us, and none of it by us. One could close the matter there and call it a fair night's work.

But before closing it, one quiet question is waiting — from a man who lets all of this stand without objection. He does not take the relief away. He grants every word of it. He refuses only to let it mean the one further thing we very much want it to mean.

His name is John Martin Fischer, and with a co-author, Mark Ravizza, he wrote a careful and demanding book, published in 1998, on exactly the question now before us: when does a situation that has been arranged against a person excuse them from responsibility for what they did? When does the fact of having been set up mean that one is not answerable? And the reason to bring him in is that he is not the moralist who waves the device away and tells us to try harder. He has taken in everything described so far, and he grants it. The contest was arranged — yes. The device is real — yes. The word was loaded against us — yes. He concedes all of it.

And then he makes one observation. He points out that two claims have been run together as though they were one. The first is: the situation was arranged against you. The second is: therefore you had no part in what happened. And these, he says, are not the same claim, and the second does not follow automatically from the first.

Here is the distinction, which is the whole of it tonight, so let me set it out slowly. Fischer offers an illustration: a pair of weighted dice, loaded so that they fall badly for you more often than fair dice would. The weighting stands for the arranged situation. His point is simply this: that the dice are loaded is one thing, and who throws them is another. Loaded dice still have to be picked up and thrown. The weighting alters the odds; it does not reach out and throw the dice on one's behalf. And so the question of responsibility, for Fischer, is never whether the situation was fair — because no situation is ever wholly fair; every choice any of us makes is weighted by inheritance, upbringing, and the circumstances we happen to be in. If an arranged situation cancelled responsibility, then no one would ever be responsible for anything, because no one acts on fair terms. So that cannot be the test.

His test is a different one, and it is worth holding onto, because it is a serious answer to the question. What matters, he says, is whether the process on which one acted — the mechanism running at two in the morning — was still responsive to reasons. Could a reason have reached you? Suppose that, while you are scrolling, a friend knocks at the door and says quietly, "you have that important thing in the morning; you told me you wanted to be sharp for it." Could that have moved you? Could it have altered what your hand did? If the honest answer is yes — that could have reached me, I could have put it down — then you were not a stone rolling downhill, moved only by outside forces. You were a person who could still be reached by a reason, and who, in some real if exhausted and disadvantaged sense, was not. The dice were loaded against you. You threw them even so.

And this is easily misheard, so let me be exact. Fischer is not swinging back to "then it is your fault after all, and the device is no excuse." He is not the same moralist in different dress. He is doing something narrower and more honest. He is refusing to allow one particular step — the step from "the situation was arranged" to "so none of it was me." He grants the arrangement in full. He says only that the arrangement, by itself, does not deliver "none of it was me." That conclusion requires its own argument, and the device does not supply it. The relief is real. The relief is simply not a verdict.

Consider, then, where that leaves us — the situation fully described, and one quiet man declining to let us treat it as the last word. We came into this hour and turned the question outward for the first time, from the self to the situation, and the situation proved to be everything promised and worse: a real device, built by real people, on a real science of how we break, tuned to profit when we lose, and sold to us under a word invented to make the losing feel like a private failing. All of that is true, and none of it is a trick about to be withdrawn. A great deal of the weight you walked in with was never your own.

But here are the two questions we have held apart since the first night: what failed, and how much of it was ours. And the situation has done something to those two questions. It has moved the first a great distance — "what failed" is no longer "a muscle in me ran low," but "a device outside me, built to defeat me, did its work." That is a real answer, and a better one. But the second — how much of it was ours — the situation has not touched. It seems to have answered it. It seems that "the device did it" answers "how much was mine" with a clean "none." But Fischer has just shown that this is a different question, mistaken for the first. "The situation was arranged" is the answer to what failed. It is not the answer to how much was ours. And that second answer we do not have. You came in tonight expecting to be released, and were not — not because you are weak; we have set that word aside for good, and it does not return — but because "the device did it" and "none of it was me" turned out to be two different statements, and only the first has been established.

And next week the difficulty deepens, in a way one can already anticipate by recalling the first night — the friend, the injection, and the wanting that simply stopped. Everything built tonight concerns the situation. The feed, the device, the engineered food — all of it works on a person from the outside, through the circumstances, through the dice in the hand. Next week something enters that does not rearrange the circumstances at all. It goes straight past the situation, past the circumstances, past the dice, and reaches the wanting itself, and reduces it directly. A drug. And here is what to expect: it will make tonight's relief seem small. It will offer a relief so complete — the wanting was not a flaw in my character, it was a matter of chemistry, and it can simply be switched off — that you will want to take it with both hands. And that is precisely the moment, with what looks like the best news of the whole series in front of you, at which I will put the most uncomfortable question I have. Not whether the relief is real, but whether you can tell the difference between being set free and being let off.

Thanks for listening. I'll see you next time.

← Back to Weakness of Will & the Engineered Self Back to top ↑