Language, LLMs & Machine Understanding · Episode 4 · 18-min read

What's the missing extra?

Transcript · audio coming

Last week ended with two theories that cannot be reconciled. One holds that a word's meaning is fixed by the pattern of the other words it appears alongside — and the machine has all of that. The other holds that meaning requires contact with the world itself, something a system of words alone can never supply — and the machine only ever has more words. Neither view prevailed, and the choice between them could not be made.

But beneath that unresolved choice was a further assumption, one that went unexamined — and it is the one we take up directly tonight, because it has been relied on from the very first minute. It is the conviction that you, the listener, have the real thing: that beneath your words, behind your use of them, there is an understanding — an inner grasp of what they mean — and that whatever the machine is doing, it has not got that. Both of last week's theories left the assumption untouched. You left with it intact: at least I understand, from the inside, and the machine does not.

Tonight that conviction is put a single question. Point to it. Identify the grasp. Show what lies beneath your use of a word and is your understanding of it — the part that is not simply everything you can already do with the word.

This is Philosophy for Us — philosophy for everyone, no degree required.

This is the fourth of nine episodes. Last time, the model's defender argued that a word's meaning is the pattern of the other words it appears alongside, and that this is exactly what the machine has — and neither that claim nor the reply that a web of words can never put one in contact with the world itself could defeat the other. Tonight a third voice goes further than either. It says meaning was never a separate thing lying behind the words at all. And it addresses directly the one thing that has been kept in reserve this whole time: the understanding underneath, the grasp you are sure you have.

Let me introduce that voice by the ordinary observation it begins from — something we do all the time and rarely question.

How do we tell whether someone understands a word? Not in theory, but in practice, on an ordinary day. A child uses the word "ironic" in a sentence. A colleague says "leverage." Someone claims to understand how a mortgage works. How do we check? We do not open the person's head and inspect the contents; we cannot. What we do — the only thing we ever do — is observe what they can do with the word. Can they use it in a fresh sentence, and not merely repeat the one they heard? Can they notice when it is used wrongly? Can they explain it to someone else, answer a question about it, recognise when it does not apply? That is the whole of the test. Understanding, where we can actually observe it, is a set of things a person is able to do: to continue, to apply the word correctly to a case they have never met before.

The voice for tonight belongs to the philosopher Ludwig Wittgenstein, in a book published in 1953, the Philosophical Investigations. I am going to report what he argued rather than reproduce his manner; he writes in short numbered remarks, mostly questions, circling a point. Here is the thread. We speak as though understanding a word were one thing — a state one is in, an inner act of grasping — and using the word correctly were a second thing, the outward sign of the first. Wittgenstein spends the book testing the suspicion that this has it backwards: that there is no inner grasp lying behind the use, causing it, which is the meaning. He puts the claim carefully, and the care matters, because he is often quoted loosely. For a large class of cases, he writes — not all, and he is explicit about that — for a large class of cases, the meaning of a word is its use in the language. The use is not a symptom of some understanding behind it. As far as anyone can ever point, the use is the whole of it.

If that is right, the direction of the argument is already clear. We have been confident that there is something further. Behind our competent, fluent use of "gravity" or "justice" or "irony," we picture a further inner grasp — the real understanding — and we are certain it is there, and certain that the machine's version of it is empty. Wittgenstein's question, which is what everything tonight turns on, is a simple one. Try to find it. Look behind your own fluent use of your own favourite word, and say what is there that is not simply more of the use.

That is the challenge. And a warning before we take it up: it does not threaten only the machine. It applies to your own case just as much. Let me set the argument out at full strength, and the reason will be clear.

Start with the moment understanding actually arrives, because Wittgenstein builds everything on it, and it is a moment everyone has lived through many times.

Suppose you are being taught something — a rule, a pattern, a word. The teacher sets down a few examples. And there is a moment — a familiar one — at which something falls into place and you say: now I have it. Now continue. Carry on the pattern, use the word, do the next case yourself. What happened at that moment? One wants to say: something appeared in my mind, the understanding, and now I am reading off it. But consider what actually shows anyone — including you — that the moment was genuine. The only thing that shows it is that you then continue correctly: you do the next case right, and the one after, including cases the teacher never showed. If you could not continue — if you could reproduce only the three examples and stopped at the fourth — no inner feeling of now I have it would convince anyone, and should not convince you, that you had understood. The understanding is not the moment of grasping. It is the going-on that the moment makes possible. Wittgenstein's phrase for it is a plain one: understanding is the mastery of a technique.

Now look harder at the thing still being protected — the inner grasp, the private getting-of-it supposed to sit behind the technique and drive it. Wittgenstein's sharpest question is what work that inner thing could actually be doing. Suppose there were a private inner state behind your use of "justice," available only to you, that was the real meaning. The difficulty is that it makes no difference to anything. Your use of the word — how you continue, what you apply it to, the errors you catch — would be exactly the same whether the inner state were present or absent. And something that could be present or absent while changing nothing anyone could ever point to, in yourself or in another, is not doing the job it was brought in for. It is not the meaning. The meaning is out in the open, in the technique, where it was all along. So look behind your own fluent use of your own surest word for the further thing, the grasp itself, and Wittgenstein's expectation is that you come away with nothing every time — not because you did not look hard enough, but because there is nothing there but more of the use you already had.

Consider what this does to tonight's question, because the consequence is direct. You came in certain the machine is missing something — a real understanding beneath its words. The plan was to keep it out by pointing behind its use and observing that there is nothing there. But if Wittgenstein is right, there is nothing behind your use either — there was never supposed to be — and "nothing further behind the use" stops being the mark of a fake. It is the mark of how meaning always worked, for everyone. The machine uses words. It continues — applies them to new cases, corrects, extends. On this account, to demand that it also possess an inner grasp beneath the use before one will grant that it understands is to demand of it something that does not exist in your case either and never did. The "real understanding underneath" that has been held up as the thing the machine lacks may be, in Wittgenstein's term, a picture — a way the grammar of the word "understand" leads us to imagine an inner object — rather than a fact about anyone.

Be precise about what this voice does and does not claim, because it is easily over-read. It is not saying the machine understands. It does not crown the parrot or the model. It is doing something quieter and harder to dislodge: it removes the move one was about to make. The move was to settle the matter cleanly — it merely shuffles words; I grasp them — and the reply is: the grasping you mean, the thing over and above the skilled handling of the words, produce it, in your own case; and you cannot, because it was never a separate thing. This is not a fringe position. Serious philosophers have held every part of it for seventy years. And it does not hand you a verdict on the machine. It takes away the verdict you thought you already had.

So one might think the machine is cleared — or at least that the easy way of ruling it out is gone. And then comes the voice that will not let the matter rest. There is an objection here, and it is not a quibble. It is the thing you have been wanting to say for the last few minutes and I have been holding off. Say it now.

Here is the objection. A parrot continues. Teach a parrot "Polly wants a cracker" and it will produce the phrase in roughly the right places all day. Drill a child on the times tables with no idea what multiplication is, and the child will recite "seven eights are fifty-six" without a fault. They continue; they use the words. And no one thinks for a moment that the parrot understands "cracker" or that the child understands multiplication. So competent use cannot be the whole of understanding — because here is competent use with the understanding plainly absent. There is the gap in the account. Whatever Wittgenstein has said about there being nothing behind the use, something separates the parrot from a person who understands; it has not yet been named, but it is plainly there.

Now the reply, because the use-theory has one, and it rests on the very word it was relying on all along: rightly. Consider what the parrot cannot do. Move the cracker exchange to a new setting, one the drill never covered, and the parrot is lost; it has the sound, not the technique. Give the drilled child a number the table never reached, or a word problem in place of a bare sum, and the child is stuck, because there was never any multiplication there, only a list. The one who understands is not the one who reproduces the old cases. It is the one who continues correctly into new ones — who applies the word to a situation never met before and gets it right, who catches a misuse, who can be wrong and knows it. That is what "mastery of a technique" was guarding all along: not mere output, but output answerable to a standard, output that extends and that can fail. The parrot was never a counterexample. It is the easy case of continuing without understanding, and "correctly, into new cases" disposes of it.

But once the word "rightly" has been used, it unsettles the whole account. For what makes a use right? Suppose you have added numbers correctly your whole life. What fact about you makes it true that you have been performing addition, rather than some other rule that agrees with addition on every sum you have ever actually done and then diverges on some enormous pair you will never reach? This is a well-known difficulty, sharpened in a reading that the philosopher Saul Kripke drew out of Wittgenstein, in his book Wittgenstein on Rules and Private Language, published in 1982. The unsettling part is that you cannot answer it by pointing inward. One wants to say: because I grasp the rule, addition, in my mind. But that is the inner state again, which we have just watched do no work. One wants to say: because I am disposed to give the right answers. But dispositions are finite and faulty — one makes mistakes, one tires, one would die before reaching the largest cases — and a rule is not "whatever I happen to be disposed to produce," or every mistake would count as a new and correct rule. So what fixes the "rightly"? Wittgenstein's own direction is outward — to what he calls a form of life. You count as adding because you are inside a shared practice, carried on with other people, in which there is such a thing as doing it wrongly and being corrected — a public standard you are held to. Meaning lives in that practice, not inside the head.

And the moment that is said, the machine question reopens from the other direction — and I am going to leave it open, because Wittgenstein leaves it open and so does everyone honest about it. If being right is a matter of being inside a practice with others in which one can be corrected, then is the machine inside a practice of that kind? Or did it only take in the records of ours — the fixed transcripts of a game it never actually played with anyone, was never corrected in, was never answerable to? I am not going to settle that tonight. Notice only that the question has changed. It used to be: does the thing have an inner grasp? It is now: is the thing inside the practice in which meaning is made? That is a different question, and we do not reach it properly until later in the series.

So consider carefully where the two voices leave us, because this is the whole of tonight. The first says: stop searching for an understanding beneath the use — look, and there is nothing there, in your case no more than the machine's; the use was always the whole of it. The second says: but continuing correctly cannot be the whole of it either, or the parrot understands — and the "rightly" needed to keep the parrot out turns out to be something that cannot be fully specified, neither for the machine nor for yourself. The worry cannot be put down. The extra cannot be found. Reach for the thing the machine is missing and there is nothing to take hold of; decide that use must be enough and the parrot objection returns. The machine question, once again, is not really about the machine. It is the oldest question in philosophy in a current form — what is it to understand anything at all — and one has been answering it a whole life long without noticing one had taken a position.

Something has changed in the last hour. Let me state it plainly — and no, nothing has been settled; I keep my promises.

You came in carrying one thing you had never had to defend: that there is a real understanding beneath your words, an inner grasp, and that the machine has not got it. You held it the way one holds an obvious fact. You leave with that sentence turned into something you can actually hear — not a plain fact about the machine, but a wager on a theory of what understanding is, and a wager that was answered tonight, hard, by a serious voice that asked you to find the grasp and showed you that you could not. That is the gain, and it is a usable one: from now on you can catch the moment you reach for "the real understanding underneath," and you can notice the odd position it leaves you in. You cannot point to the grasp. And you cannot give it up. An hour ago you did not know you were holding it. Now you can neither justify it nor abandon it.

This is the fourth of nine. Here is the shape of the series so far. The first episode opened the question. The second put at full strength the case that the machine is only a kind of parrot. The third saw the model's defender answer that case's central argument. And tonight a voice went underneath all of it and asked whether the thing being kept in reserve — one's own inner understanding — was ever there to keep. Every test so far has had a way of turning back on the person applying it.

Next week, the tests that do not. For not every difference one might draw between oneself and the machine also turns back on oneself — and it would be a cheat to let tonight close as though it might. There are serious voices that say: no; there is something the machine was never built to do, something a person has and it plainly lacks, and no amount of fluent continuing will ever supply it. They are not the credulous reflex, and they are not tonight's deflation; they are the arguments that keep a real difference in place. They have the floor next, as each of these has had it in turn. For now it is enough to know that the difference does not only run inward.

Before then, run tonight's idea once yourself, on a case I have not touched; watching it done is not the same as doing it, and the second time is where it takes hold.

Picture two children who can both recite the two-times table perfectly — two, four, six, eight, all the way to twenty-four, without a fault. From the outside they are identical. Now ask each of them for something the drill never reached: continue past a hundred, or say what two of the things that make six would be. One child carries on — a hundred and two, a hundred and four — and can tell you it is pairs, groups, the thing the counting is for. The other stops, because there were sounds in order and nothing underneath to go on with. There it is, with no machine anywhere near it: the difference between understanding and drilling, and it shows itself in exactly one place — who can continue correctly into a case they were never shown. A test out in the open, with no inner state required. So run it yourself, on the thing on your screen. Give it something genuinely new, well past anything you would expect it to have encountered, and see what happens. A striking amount of the time, it continues. It does the case it was never shown. And there is the difficulty I will not resolve for you tonight: the test you would reach for to catch a drill — can you continue into the new? — the machine does not fail it. Which leaves two ways to read the result, and no clean way to choose. Either continuing correctly really is the whole of understanding, in which case a thing that continues like that has a genuine claim to it, as much as you do. Or its continuing is a drill so vast that nothing is ever truly new to it, and the test you trusted to catch a pretender was never fine enough to catch this one. There is no third thing to break the tie, because the third thing would be the inner extra — and you spent tonight watching that come up empty every time you reached for it. Hold the two children in mind on one side and the thing on your screen on the other, and you cannot cleanly say which child the machine is.

That is where the episode leaves you. You came in sure there was an understanding lying behind your words, and sure the machine had none. Tonight you went looking, in your own case, behind your own surest word — and found the position you were defending to contain nothing but more of the same use, and still could not shake the sense that something is there. You are no less certain than last week that you understand and the machine does not. You simply can no longer find the thing your certainty was about.

Thank you for listening. I will see you next time.

← Back to Language, LLMs & Machine Understanding Back to top ↑