Part Two · The Shared World & Technology · Series 31

AI Governance

The computer said no, and there's no one to ask why.

Forthcoming Political & SocialEthics

The question

You applied for the loan online, and the answer came back in about four seconds. Declined. Not "let's talk about it," not "here's what's missing" — just declined, with a reference number and a link to a page that says your application did not meet the criteria. So you call. And the person on the phone is kind, and completely powerless, and what she tells you, more or less, is that she can see the decision but she can't see into it. The model scored you. She doesn't know which things counted, or how much, or whether the late payment from four years ago is the whole story or none of it. She can't override it. She can re-run it, but it'll say the same thing, because nothing's changed. There's a box you can tick to "request a review," and you tick it, and you already know — the way you know things at 2 a.m. — that the review is the same machine, looking at the same numbers, reaching the same place.

Now look hard at the exact shape of what's bothering you, because it's easy to name it wrong. It is not, mainly, that you think the machine got it wrong — maybe it didn't; maybe you are, by some cold measure, a worse bet than you feel like. It's something else, and it's harder to say. It's that a decision got made about you — about your house, your year, your sense of whether the system is for people like you — and there was no one standing behind it. No reason you could follow. No one you could make answer. The thing decided, and then it closed, like a door with no handle on your side. And here's the part the next half hour is going to be about: suppose I could prove to you that the machine is more accurate than the loan officer it replaced — fairer, less moody, less swayed by your accent or your name. Would that fix the thing that's wrong? Or would the door still have no handle?

The episodes

Each episode builds on the one before, and the finale hands the question back to you sharper, not settled.

8 of 8 transcripts are readable now — audio is coming.

1

The computer says no

A loan declined in four seconds, with no reason you can follow and no one who has to answer. The first episode fixes the exact shape of the complaint — not that the machine erred, but that a decision fell on a life and no one stood behind it — and asks whether proving the machine accurate would settle it.
Read transcript →
2

Make it fairer

The instinct to audit the bias out is taken completely seriously, and so is the builders' reply — the machine can be less biased than the humans it replaced, and insisting on arguing your case with a person may cost the worst-off more than it spares them. The episode ends with a suspicion that fairness was never the whole of what was wanted.
Read transcript →
3

What makes any authority legitimate?

No machines this week. The episode returns to the oldest question about power — what makes anyone's say-so binding? — with Joseph Raz on authority, and separates three things usually blurred together — getting it right, being entitled to decide, and being a standing one could accept. Correct and binding turn out to be different questions.
Read transcript →
4

Suppose it's right

Everything is granted — a system more accurate than any human, audited until no trace of bias remains, governed and overseen. The episode then puts the listener back in front of the screen and asks whether the complaint is gone, and finds it still standing — now detached from accuracy, and not yet named.
Read transcript →
5

The right to contest

Two accounts of the missing thing. Danielle Citron and Frank Pasquale argue that automating the decision quietly abolished due process — what is owed is notice, a hearing, a channel that can actually move the result. Reuben Binns presses the deeper reading — a scoring model never meets you; it locates you as an instance of a class. Both are built seriously, and neither is crowned.
Read transcript →
6

Would a human pass?

The relief of finally reaching a human is examined and does not survive. The standard built for the machine — a real way to contest, a someone who must answer — is turned on the humans we trust, and they often fail it too. What remains is the refusal to want the machine instead, which the episode declines to explain away.
Read transcript →
7

When it goes wrong, who answers?

A frozen account, and a chain of parties — vendor, deployer, regulator — each of whom acted correctly by the time it left their hands, leaving a harm that fell on one person and a responsibility that falls on no one. The word accountability is traced to its root, a reckoning owed to someone who could demand it, and its algorithmic use is found to have quietly deleted that someone.
Read transcript →
8

Could you accept it?

The final episode adds nothing new. It assembles the seven considerations and deliberately declines to settle them; both easy answers — ban the algorithms, and if it is accurate, use it — are closed. The question handed back is what would make a decision that goes against you one you could accept, and whether the demand to be answered is a real wrong or a want.
Read transcript →

The question

The episodes

Where it connects