Hi — I’m writing this book in the open and I would love for you to follow along, share your thoughts, etc. Its purpose is to help people understand how to live with AI, and have the necessary understanding of where it makes us better, and where it is not so useful.
Part One, consisting of something like eight chapters, is aimed at providing an understanding of some simple mechanics of how a GPT works… and how it does not. A machine that speaks so well, yet understands nothing. It is powerful, but also a bit behaviorally deranged, hence its nickname: The Mostly Helpful Psychopath.
Part Two, which is another six chapters or so, is where we talk about how to live with this new companion, or visitor, or even intruder (depending upon your opinion), and how its behavior can shape our own, whether individually, within our closest relationships, and in the broader circles of our lives.
Chapter 2: The Wordsmiths
The Magician and the Octopus
I’m not sure what your favorite magic trick is, but one of mine is when a magician cuts a piece of fruit open (usually a lemon) and behold, within it is a rolled up playing card, the Seven of Hearts that someone had only moments before “randomly picked” out of a deck of cards.
I KNOW that what the magician is doing is not truly magical. But I want it to be!
I know that the reality underneath is 1,000% cleverness and craft combined with my own inability to see that cleverness. They do things I can’t see, and in that way seem, frankly, impossible.
Have you ever thought about that? I mean, why do you like being fooled, even though you know you are being fooled?
It’s got to be that we enjoy being fooled, right?
But it is strangely more complex. People who research this stuff say that it is because our brain prefers the most ridiculously improbable explanation (that a card magically transported itself to the inside of a lemon)…even writing that makes me think I must be crazy. But my brain, our brains, prefer something that we know is not true to that very scary alternative of not knowing at all.
We’re wired that way.
So, here’s where I think you’ll go down one path or the other, and either way we’ll meet at the end of this trail: Do you really think the machine is “smart” and that it “understands you” and “knows a lot”? It’s okay if you do…it was built to be convincing. Or maybe you are only a 5 out of 10 in believing that…like it kinda seems true, but maybe not?
But here’s something worth thinking about: how much of that is because of this magician effect? How much does your brain want to believe in playing cards can teleport inside of citrus fruits, because you don’t know how this all works?
Here’s the thing about all things magical: once you see how the trick works, you can’t unsee it.
Spoiler alert.
The magician puts a duplicate seven of hearts card in the lemon in advance. They always use citrus because you can cut it open or punch holes in it without it showing bruises. It’s never a banana or an apple. That our volunteer “chose” the seven of hearts was not random, but instead the card was “forced” (a card technique) to the person…no matter what the volunteer said or how they said “now”, the card that they would hold up and show the crowd was always going to be the seven of hearts. This was the only remotely difficult thing, and the magician dealing the cards had been practicing this technique since they were nine years old. Watch it on YouTube, and you will never again be fooled by the “pick a card, any card” technique.
So, we’re going to pull back the curtain on the machine, see how it was stuffing lemons with playing cards while you and I were still sleeping, and then also watch in slo-mo how it takes what we say and cleverly reshapes it and echoes back to us a different set of words that are calculated in ways that are mystifying…until you know the trick.
We’re going to unlock the secret of the machine’s ability to act like it understands.
I’ll start by telling you the story of Olivier, the Statistical Octopus.
Olivier is a deep-sea octopus that was born with a savant like ability to do statistical analysis of patterns. He lives a good life, knowing when the best times to hunt certain prey, and when the best times are to hide and not become prey. Now that he has done this for several years, and he knows the patterns, his skills are somewhat underused.
Olivier as it happens, lives in the deep sea trench beneath the shark-infested waters between two deserted islands. He has never seen the islands or anything else except that which has drifted or swam down into his deep-sea trench.
One day, an unfortunate boating accident strands two people, Abby and Ben, one on each of the two islands. There is enough water and food on each island for them to survive, but that’s about it. Eventually, they both discover that there is a telegraph line that runs underwater between the two islands, nowhere else. They learn to use Morse code to communicate, and they talk multiple times every day.
Olivier, being so observant (and bored) eventually notices the telegraph cable that goes through his trench. He touches it and can feel the pulses. He notices patterns. With some practice, he learns to generate his own pulses.
After a while, Olivier can see the patterns of the pulses, even though he does not know what they mean. In the morning there is a pattern where Abby sends Ben a “Good morning, Sunshine!” Olivier doesn’t know what the pattern means. But when Ben routinely replies, “Yes, another day of sunshine.” Olivier notices that pattern. Sometimes, Ben replies, “You still here?”, as a joke. Olivier doesn’t know what that pattern means, but he does know that the pattern of exchanges that follow Ben’s two different responses are different. And often almost predictable.
One day, Olivier decides that he will send pulses as well and pretends to be Ben (who hasn’t woken yet, and whom he doesn’t know anything of) when Abby sends her morning prompt. And then he sees what signals come back. Sometimes he just listens…but he is always learning what the patterns of the signals. A certain 37-dot-dash sequence is almost always responded to by a 42-dash-dot sequence. And so on.
Some days this causes confusion between Abby and Ben, when Ben seems to not remember that they already said good morning. But no matter, says Abby, while she secretly thinks that Ben has become forgetful.
This story is my rendition of a fable created by the linguists Emily Bender and Alexander Koller in a 2020 research paper. It won awards.
You’ve probably figured out that Olivier is intended to represent the machine, the AI. And the fable suggests that Olivier can converse in a limited way, but does not understand what he is saying.
You probably get that Olivier can pretend to be Ben because he has trained himself by listening and noticing patterns, right? In a sense, he is building his own Chinese room rulebook…but a book just based on what he heard from how Ben responded to Abby, though he did not know either of them.
Olivier’s PhD in Statistics
But Olivier doesn’t just have a good memory, his nine-part brain hosts a powerful statistically-calculating memory. And after a while, having heard thousands of morning conversations between Ben and Abby, he can look at most things that they say, and not be surprised with how they reply. He notices that when a certain sequence of 45 dot-dashes come through, then Ben replies with a much shorter 12 dot-dash sequence, and sometimes it has another 23 or so that follow it.
I’m out on the trail…walking now because I just ran up a fairly steep hill and need to catch my breath. I come around a turn and then all of a sudden someone’s there in front of me coming the other way, a woman in a stylish, purple yoga outfit with hiking boots and her cell phone in one hand.
As she looks up at me, I say, “hey, good morning!” and smile slightly.
I’m expecting she might just say something simple, like “Hi” or even a “Yes, it is a wonderful morning!”
If she said either of those things I would know there was nothing more to say. We completed the transaction.
Psychiatrist Eric Berne, in his famous book Games People Play, showed how much of what we say to each other is scripted. Berne used simple interactions, like the (awkward) “talking in the elevator” script, that includes the discussion of how nice of a day it is (or how hot it will be, etc.), to illustrate that we often say something highly scripted, but are really doing something different: we are confirming the other person’s interior.
You might call that script the woman and I were doing the “running into someone on the trail” script.
Our language is like that… more than you or I ever realize. For any string of words that comes from one person, there are a fairly small set of strings of words that you would normally reply with. Some seem more normal than others, but as Eric Berne would point out, any one we choose says something about us as well.
But the woman stops walking, and keeps looking at me, and as I get closer, says, “Hi. Is there an exit from this trail coming up soon?” and pointing in the direction I had just come from.
In that moment, my brain realizes that we are now in a different script, the “help I am lost, can you give me directions?” script.
Now, her question poses a dilemma for me, because these moments in the morning are one of my favorite “alone times” where I find peace and introspection, connect to myself and a sense of place here in the beautiful mountains of Southern California.
I would rather not reply to her at all, but I’m not that kind of person, and also I don’t think I’d want her to think me so rude. She doesn’t look like she’s been hiking for so long that she’s about to die from thirst or exhaustion… Rather, when I take in the whole moment, I realize she’s probably never been on this trail before, and has gotten as much as she wants and can she get off this ride now?
OK, maybe this is the, “How do I get off this fucking trail back to civilization again?” script.
And the introvert/curmudgeon in me, knowing that she will find the trailhead just a mere five minutes away or so, feels like I should just say “No.” but even the curmudgeon admits that there’s no way the answer is no… And if I said no, and then she kept on walking and saw that I lied to her…well, I wouldn’t want her to think badly of me. Maybe I’ll run into her again.
I did all of that thinking during this time it took to take one step, maybe less, really fast…you know what I mean? I mean we humans are really kind of weird that way, right? We are always calculating probabilities and alternatives… I think one of the most apt descriptions of human beings is that we choose our words carefully. More carefully than we even admit to ourselves.
And during all that brief moment, another part of my brain was trying to outshout the curmudgeon, and say that we needed to help this woman who obviously was out for a casual stroll the close of which she had hoped would come a bit sooner.
I stopped and nodded, and quickly looked back at the trail that I had just come down, estimating in my mind how far it is to the little branch that leads back to the street, and said, “Yeah, pretty close… Maybe 100 or 200 yards from here. Pretty easy to see when you get there”
She let her arm holding the phone drop to her side, Google Maps no longer needed, smiled, and said thanks as I started moving again, now jogging so that I could put distance between us…and get back to my quiet time.
Twelve seconds for that whole thing, that whole moment. All that thinking going on about what might be said, what has been said, and what to say…and I even did a little debate about it.
We are all statisticians who choose our words carefully. We can find multiple meanings of anything very quickly, and sort through them to choose how to reply.
But we have some tools, a complex shortcut that makes this so much easier, and it is the same shortcut that the machine uses: we know what words go together, and which don’t.
After a month or two of watching the dots and dashes go back and forth, Olivier notices that he doesn’t need exact matches in order to get it right: if Abby sends a slightly different, “Good Morning, Mister Sunshine” then Ben still replies with his standard answer or the joke answer. The same reply works for different messages, as long as they are not that different, Olivier realizes.
One day, unbeknownst to Olivier, a grizzly bear washes up on Abby’s island. Abby immediately sends a message to Ben asking for help, “Help! A grizzly bear washed up on my island!! All I have is sticks to protect myself, what do I do?”
Olivier listens to see what the response is to this new set of dots and dashes, but Ben is still asleep. After waiting for a few moments another set of dot-dashes come from Abby, again Olivier waits to see what the response is and there is no response. So he sends the most common dash-dots that Ben sends, which (unbeknownst to him) mean, “Yes, another day of sunshine.”
More dot-dashes come from Abby, different patterns than he has ever seen, but the difference means he has no idea what the right dash-dots are to respond with. He tries the other message that Ben often sends, but eventually the line goes silent.
Olivier never understood the English words that the dots and dashes represented. He didn’t know what an island or sunshine were. He didn’t know Abby, Ben, or what a human being is. And he didn’t know what a bear was.
But he knew how to reply to Abby with Ben’s favorite joke, “You still here?” Even though he did not know what that meant.
You and I, and even the machine, have our own internal versions of Olivier. We notice when we are hearing familiar words and we have familiar replies that we use. Some things that sound a little or even a lot different actually mean the same thing, so we can use the same reply.
Someone in the elevator says, “Gonna be a hot one today, right?” or “Can you believe this heat?” and they’ll probably get the same response from you, because although the two phrases do not look or sound the same, or even carry the same meaning, you understand (sort of) why they are being said, and also (sort of) what you should respond with.
You were born a natural wordsmith
I don’t know you, but I do know that you have great expertise at using our language.
It wasn’t a class you attended or any book you read. It was how you learned to combine groups of words, starting from when you were somewhere around 12 to 18 months old, and by the time you were five, you had the adult-like ability to pour any new verb into a general pattern. As a child, you might hear a nonsense verb like “to gorp” (or make one up yourself) and correctly put it with other words “he gorped the ball to me.”
We learned it so early and so automatically, that we don’t notice it for what it is, but it is frankly amazing. All human languages share this sort of shape…it is a shape that we are already pre-wired for. It is unavoidable.
And when I say pre-wired, I mean it: children will build a language even when nobody gives them one. In Nicaragua in the 1980s, deaf children were brought together in schools for the first time. No teacher taught them to sign; each child arrived with only the homemade gestures they used with their families. Within a few years, the children had pooled those gestures and built them into a full language, Nicaraguan Sign Language, with its own grammar, invented entirely by children. The adults around them couldn’t speak it.
It happens in smaller doses, too. Adults thrown together without a shared language cobble together a rough trading tongue, and then their children do something the adults never could: they take that broken material and regularize it into a complete language, with all the shape and machinery of any other. Linguists call the children’s version a creole, which is the same word we use for the blended language and culture of Louisiana, which formed exactly this way.
You may have even watched a tiny version of this in your own house. Twins sometimes develop a private language between themselves (of course it has an utterly horrible scientific name, cryptophasia) where the two children are building their word-machinery at the same time.
Why am I telling you all of this? I think this is the hidden version of the magician having practiced dealing cards for forty years, starting when they were old enough to (barely) shuffle a deck. We have a way of working with language that is extraordinarily clever and we started practicing it somewhere between 50,000 and 300,000 years ago. The earlier versions may have been some combination of noises and gestures, a Homo Sapiens Sign Language you might call it.
The machine uses many of the things that we can’t see that are part of the magic of language. Like we did in childhood, it works with phrases, not words. And like Olivier, it compares patterns to choose a response.
The jaguar and the tree
And patterns are the key to the magic. Once I got my head around this one…then the rest felt a lot easier.
So let’s start with one single pattern:
“A jaguar ran into a tree.”
Six words.
You probably had an image before you finished reading them, maybe multiple. Was it a big cat in a rainforest, maybe, crashing into a trunk? Or a sleek car wrapped around one?
Maybe the jaguar was a car, and the tree was a Christmas tree and the whole thing happened in a parking lot in December. You didn’t agonize over any of this. You just… picked.
But look at what your brain did in that half-second. “Jaguar” is two completely different things, and you chose one without slowing down. “Ran into” is three completely different things — collided with, like hitting a deer; bumped into, like spotting your college roommate at the airport; or ran toward, like sprinting into the kitchen when you smell smoke. Same two words, three different physics. And “tree” could be oak, could be ornamental, could be genealogical. You sorted all of that, instantly, without noticing there was anything to sort.
It is a phrase that feels like a phrase, right? And I bet you can make a different version of it without changing the meaning much. Like if I am using the luxury car version, I could say any of these to you and they would pretty much be the same:
A jaguar drove into a tree.
A jaguar hit a tree.
The tree was hit by a jaguar.
For all of these you would probably say something back to me like, “Oh, wow, was the driver hurt?”
That’s what Oliver was learning, right? He was learning that the same response can work for different variations of the same phrase.
The machine does not need to know what something means. It only needs to be able to see what the most likely response to the phrase is.
If you can do that then you don’t need meaning, right? I mean you don’t even need to know what a “driver” is if you know that “Oh, wow was the driver hurt?” is a correct-enough thing to reply with.
The machine replies to the Jaguar phrase by giving five different variations that are very likely assuming that Jaguar means an automobile.
For right now, I want you to just focus on the (amazing) fact that the machine knows what to say (the five replies it gave us). It also puts a “wrapper” around its answer, and I’ll explain that part later. But here it just gave us a list of the most-likely jaguar-as-a-car responses.
Amazing, right?
I’ll explain in the next chapter how it has been training for this very moment, like a young magician in the making shuffling and dealing millions of hands of cards. It took humans hundreds of thousands of years, most magicians took twenty-plus years, and it took the machine about six years to learn this trick.
Tricks work, except when they don’t. What if there is no lemon available? And what if there is no way to reply? Like if the phrase doesn’t make sense. For Olivier, it was Abby sending a message about a grizzly bear. Here’s ours:
Bicycle the if of through which.
Six words, same as the jaguar phrase. But your brain has nothing to work with here. There’s no scene to picture, nobody speaking, no situation to figure out. You can read it, you can see the words, but it doesn’t add up to anything. And you know it.
Turns out what we’re good at is knowing which words go together. And which ones don’t.
Now here’s the thing. The machine does the same work, using phrases.
It does it silently and fast, like you did. But the way it does it is completely, structurally different from the way we do it.
The machine receives the nonsense phrase and responds with the statement that this is not a recognizable sentence in English.
How did it do that? The simple version is that it looked and could not find a phrase to respond with. If I use a phrase that seems to make sense, like our jaguars and trees, it can respond because it has seen similar phrases and the words that replied to them.
But it has never seen the words together, so it also could not find anything that looked like the words of a reply.
And that’s partly how Olivier must have felt, when Abby sent the bear-message. Certainly, there is a part of me that wished for an Olivier that had heard more conversations, so he might have replied better to Abby.
A few weeks after the conversations ended, Olivier found himself increasingly bored. He wondered if there might be other cables on the bottom of the ocean that he could listen to. It was the first time he had ever left the deep underwater canyon between the islands.
Within a few days, he came upon two very thick cables that were filled with these signals, literally thousands of them at any moment. He wrapped his tentacles around them and started listening.
Looking for the other chapters?
Jack Skeels
Jack Skeels
Thanks for reading The Mostly Helpful Psychopath! Subscribe for free to receive new posts and support my work.
