ChatGPT, Language and Symbol Grounding

Emily Bender (as well as Timnit Gebru and “Shmargaret Shmitchell”) are warning that AI is not really “thinking” by a “thinker,” that its development and use is contaminated by commercial and malign interests, and that it poses dangers. They are right. But they do not seem to have any new insights into why and how AI is not really thinking (nor do they, or anyone, have any solutions for the dangers it poses).

Bender’s point about the difference between form and meaning in language is actually about the symbol grounding problem (which she does cite, but as far as I can see, she does not have a proposal for solving it).

There is way too much chatter going on about chatGPT right now, so it’s very hard to see whether there is any signal in all the noise about it. It seems to be the same thing over and over about “take-overs” and “singularities”.

About chatGPT, my own tentative understanding is that its performance capacity shows how much latent structure there is in words and propositions, across all languages, even though the shape of words is arbitrary, and so is the shape of the syntactic conventions we adopt in our languages (with the possible exception of Chomsky’s Universal Grammar).

The reason is simple: Despite the obvious differences between images and words (or objects and their verbal descriptions) some of the structure of things in the world is systematically shared by the structure of the sentences describing them. And that’s what gives the ungrounded universe of words that chatGPT swallows, together with the algorithms it applies to them, both the real capabilities it has, and the illusion that it gives us, of talking to a real “thinker.”

A trivial example will illustrate this. Although a cat lying on a mat can be described in countless different ways (“a picture is worth more than 1000 words…”), within and across countless languages, even the simple arbitrary english proposition “The cat is on the mat” shares, systematically, some structural properties of the scene and object it is describing. That structure, encoded in the verbal description, is systematically present in all verbal descriptions, and it is extended systematically in bags of googols and googolplexes of words and propositions.

That structure is, in a way, a formal shadow of the thing described, in the description. It’s also what makes google’s database of “commented” images so searchable and navigable. (It does it doubly well for software code, if the code itself is also commented in English [or any language]).

This shadowed structure is an epiphenomenon; it is not meaning, or thinking. But it can still do a lot of things that look intelligent, because it is parasitic on the grounded meanings of the words in the heads of all the human speakers that spoke or wrote all the googols of words, mined by the algorithms underlying chatGPT and the like.

Apart from all the money and mischief to be made by mining and manipulating these shadows, they are also reflections of the revolutionary nature and power of language itself, the cognitive capacity it provides to real, grounded brains and bodies to encode the structure of their sensorimotor experience and learning into communicable and storable words and propositions.

All the rest is in the sensorimotor (robotic) grounding in the brains of the real thinkers who ground and use the words.

None of this is deep or rocket-science stuff. But it’s what makes us perceive that we are communicating with a thinker when we communicate with it in words. That in turn is driven by our “mirror neurons,” which trade on the neurological similarity of shape between what we do, and what we see others doing. That is what enables us to mimic and imitate, not just with mutual gestures, but also with words and propositions and their shared sensorimotor grounding. That is what underlies both our everyday mutual mind-reading and (robotic) Turing-Testing. It’s also what is completely missing in chatGPTs, which are just ungrounded, well-mined wordbags that Bender calls stochastic parrots,” parasitic on the shared structure between our words and the world they are about.

Word-Cloud Golem (Dall-E) 12/3/2023

A Whimper

I have of late 
lost all my faith 
in “taste” of either savor: 
gustate 
or aesthete. 
Darwin’s “proximal 
stimulus” 
is  just 
the Siren’s Song 
that 
from the start 
inspired 
the genes and memes 
of our superior 
race 
to pummel this promontory 
into 
for all but the insensate 
a land of waste.

Dementia 101

One of the “interesting” surprises of aging is that most new ailments don’t get better, you just adapt to them better (especially the cognitive ones). Probably another instance of necessity mothering invention…

Egon

Egon was born in Princeton NJ in June 1970. His parents, first cousins, had been together briefly as tiny children in a refugee camp in Austria in 1945, the only remaining avatars of their respective 25% of what had once been a large Hungarian-Jewish family.

Shipped off to be reared in gentile homes on opposite sides of the American continent, with no encouragement to correspond, they always knew vaguely about one another’s existence but never took up the thread in earnest until, in 1969, on their first day of graduate school, they met by chance in Princeton’s Foreign Students Center, drawn there, not by the technicality of their overseas birth, but by a subterranean yearning they had always felt, and that they now fulfilled by marrying after only a few weeks of ceaseless and haunting deja vu.

Egon’s birth was in Bob Dylan’s “Year of the Locust,” with cicadas whirring all around. Everyone said he was a hauntingly beautiful baby, but as he got to be one, two, three, four, he didn’t speak, and human contact seemed somehow painful for him. His parents, who had by now made Princeton their permanent home, had another child, a cheerful, talkative girl called Anni; all hope was gradually lost that Egon, who was now seven and had shown exceptional drawing ability, would ever speak or go to school. His drawings were remarkably detailed and empathetic depictions of little creatures — birds, mice, insects.

Since his birth, Egon had had severe allergic reactions to foods other than fruit, nuts, potatos and milk. His meagre diet and even more meagre appetite kept him very thin and pale, but people still kept remarking how beautiful he was, even from those few head-on glimpses they ever got of him, for he seemed to find it very uncomfortable to be looked at; direct eye contact was almost nonexistent.

Egon was not sent to an institution, although his toilet-training was not secure and he had gone through a period when he had repeatedly tried to injure himself. He was cared for at home, where everyone loved him, even though he did not seem to feel or like personal contact. The only way he seemed able to express himself was his animal drawings, which were getting smaller and smaller, until now they were only close-up details of insects. Anni made up for Egon’s silence by being a very gay, chatty, sociable, affectionate girl with a huge appetite who did very well in school and even became something of a local celebrity for her expressive and imaginative performances in a children’s theatre.

Then Egon reached twelve, puberty, and a sudden change occurred. He was standing in his usual way, with his back to the window, occasionally glancing sideways into the front yard. These were the glances with which he had proved to be able to take in an enormous amount of detail, for this was how he glimpsed the little creatures he would draw, never gazing head-on. Egon looked up abruptly and cried in a clear and penetrating voice:  “Mama, wait, don’t back out!”

His mother heard the first five words he had ever spoken just as she was pulling her keys from her pocketbook to lock the back door before going out to the garage to get into her car. Anni heard them just as she was starting down the stairs to take over her mother’s vigil over Egon.

What they both saw when they rushed to him was Egon facing the window instead of with his back to it, and peering out directly and intently instead of just swaying his head languidly to and fro. Ninety silent seconds went by; then he turned toward them, and back to the window, intoning softly, with a slight pubertal hoarseness in his voice, six more words: “Look, you would have hit him,” pointing toward an old dog, dragging a leash, who had been running dazedly up the street for several minutes and had only now reached their driveway, at the same instant the car would have emerged from it if everything had gone as planned. “Can you call his owner, Mama?”

Egon went to school. It turned out he could already read and write, though no one could remember having seen him with books or magazines for any length of time, and even then all he had ever done was turn them round and round passively, never holding them right side up as if to read them.

Not everything about Egon reverted suddenly to normal as of that day. His personal contact was still very vague. He would sometimes smile with some embarassment in response to a glance, but he still rarely looked at anyone directly. And though he could now talk, he certainly was anything but talkative. Days would still go by in which he would not say a word. His family had the feeling that communication was still somehow painful for him.

And he stopped drawing altogether. No one could get him to do it. He had no interest in his sketching materials whatsoever. And of course he had never given any of his finished drawings — collected across the years, displayed all over the house, and filling boxes and boxes — a second glance after doing them. Instead, he now began to collect and take care of real animals. Well, not animals, actually, but insects. His room was full of terraria, where he raised and bred all kinds of beetles, spiders, mealworms, roaches.

In school Egon did well in mathematics and history. He had difficulties with English because he did not seem to have a clear sense of fiction. He was extremely slow and hopelessly uncoordinated in gym. And he had almost no social life, although his fellow-students did not dislike him. He would perhaps have been perceived as aloof, if it were not for the endearing fact that he was always to be found crouching intently around bushes or tree trunks, or the terraria in the biology lab, obviously preoccupied with his invertebrate friends rather than snubbing his fellow-vertebrates. And what saved him from ridicule was that he still retained that haunting beauty people had noticed since his birth.

One night, in May of 1987, Egon did not come home after school. Since it was not rare for him to linger over things he saw on the way home, it wasn’t until supper time that the family became worried in earnest.

His parents drove back and forth along the streets between their home and the high school. Anni phoned all her friends, and had them call their friends, searching for a trace of who had seen him last.  The police were alerted. 

At 11 pm an officer patrolling Marquand Park found him squatting by a tree, monitoring the slow march of the legions of cicadas who had been straining upward from the depths of the earth to surface simultaneously at dusk of that very day and march horizontally overland to the nearest tree, then vertically to a safe height, where they would fasten their feet firmly and begin laboriously extricating themselves from the rugged armour in which they had been dwelling underground for 17 years, awaiting this night’s summons to the surface by an unseen, unheard biological call that bade them to abandon forever their dark roach-like former forms, still clinging faithfully to the trees, and emerge at last as ghostly white nymphs, awaiting daybreak when their tiny twin backpacks of crumpled yellow would unfurl and dry into enormous transparent wings, their bodies would darken, their eyes would turn ruby red, and their abdomens would begin to whir in the tireless crescendos and decrescendos of their urgent collective lovesongs.

There was no question of scolding Egon. They were just grateful that he was alright. More nights would follow in which he came home late at night or not at all as he maintained his vigil over the closely timed emergence of the cicadas that had burrowed into the earth as little newborn specks 17 years ago. Many of them now found concrete where there had been soil 17 years earlier. Egon planted his fingers before them vertically, treelike, and they dutfully began to climb. Then he airlifted them in squadrons of six or eight over the perilous sidewalks where they were being squashed in great numbers by passersby, who hardly even saw the slow-moving legions in those last gray moments of dusk in which they were erupting daily. He placed the hand with the clinging cicadas horizontally, touching a treetrunk with his fingertips, and the cicadas would resume their march, along his fingers, till they reached the vertical tree bark, to which they transferred, leaving Egon to secure another handful of passengers.

It was to the site of these airlifts that Egon returned most often in the succeeding weeks to watch the cicadas singing in the trees as they mated and lived out this last, brief supraterranean portion of their life cycles. These were his cicadas.

Egon was visibly distressed in the last days of his cicadas. They had sung and mated and laid their eggs. Now, taking no more food since they had emerged from the earth, they were waiting to die, falling out of the trees as they weakened, flying chaotically into auto windshields and store-fronts, unable to find their way back into the trees.

Egon frantically revived his airlift, taking one errant cicada after another back to the trees and safety. He would stoop down among the legs of bemused passersby, trying to rescue fallen cicadas even as they were being squashed left and right by the insouciant multitudes.

“But Egon, they’ve finished their life cycle, they’re going to die anyway!” everyone kept telling him, but he was bent only on his mission, to rescue his red-eyed friends.

When the car struck him, he had an unusually large flotilla of passengers — four on one hand, six on the other. Evening was approaching, the congestion of rush hour was over, so the cars were moving quickly on Mercer Street. He was also frail, having done no sports at all during his entire short life. He must have lost consciousness right away, though he only died a few hours later, in the emergency room of Princeton Medical Center. They had to pry the ten cicadas, dead too but still clinging, from his rigid fingers.

Istvan Hesslein Princeton NJ June, 1989

12 Points on Confusing Virtual Reality with Reality

Comments on: Bibeau-Delisle, A., & Brassard FRS, G. (2021). Probability and consequences of living inside a computer simulationProceedings of the Royal Society A477(2247), 20200658.

  1. What is Computation? it is the manipulation of arbitrarily shaped formal symbols in accordance with symbol-manipulation rules, algorithms, that operate only on the (arbitrary) shape of the symbols, not their meaning.
  2. Interpretatabililty. The only computations of interest, though, are the ones that can be given a coherent interpretation.
  3. Hardware-Independence. The hardware that executes the computation is irrelevant. The symbol manipulations have to be executed physically, so there does have to be hardware that executes it, but the physics of the hardware is irrelevant to the interpretability of the software it is executing. It’s just symbol-manipulations. It could have been done with pencil and paper.
  4. What is the Weak Church/Turing Thesis? That what mathematicians are doing is computation: formal symbol manipulation, executable by a Turing machine – finite-state hardware that can read, write, advance tape, change state or halt.
  5. What is Simulation? It is computation that is interpretable as modelling properties of the real world: size, shape, movement, temperature, dynamics, etc. But it’s still only computation: coherently interpretable manipulation of symbols
  6. What is the Strong Church/Turing Thesis? That computation can simulate (i.e., model) just about anything in the world to as close an approximation as desired (if you can find the right algorithm). It is possible to simulate a real rocket as well as the physical environment of a real rocket. If the simulation is a close enough approximation to the properties of a real rocket and its environment, it can be manipulated computationally to design and test new, improved rocket designs. If the improved design works in the simulation, then it can be used as the blueprint for designing a real rocket that applies the new design in the real world, with real material, and it works.
  7. What is Reality? It is the real world of objects we can see and measure.
  8. What is Virtual Reality (VR)? Devices that can stimulate (fool) the human senses by transmitting the output of simulations of real objects to virtual-reality gloves and goggles. For example, VR can transmit the output of the simulation of an ice cube, melting, to gloves and goggles that make you feel you are seeing and feeling an ice cube. melting. But there is no ice-cube and no melting; just symbol manipulations interpretable as an ice-cube, melting.
  9. What is Certainly Truee (rather than just highly probably true on all available evidence)? only what is provably true in formal mathematics. Provable means necessarily true, on pain of contradiction with formal premises (axioms). Everything else that is true is not provably true (hence not necessarily true), just probably true.
  10.  What is illusion? Whatever fools the senses. There is no way to be certain that what our senses and measuring instruments tell us is true (because it cannot be proved formally to be necessarily true, on pain of contradiction). But almost-certain on all the evidence is good enough, for both ordinary life and science.
  11. Being a Figment? To understand the difference between a sensory illusion and reality is perhaps the most basic insight that anyone can have: the difference between what I see and what is really there. “What I am seeing could be a figment of my imagination.” But to imagine that what is really there could be a computer simulation of which I myself am a part  (i.e., symbols manipulated by computer hardware, symbols that are interpretable as the reality I am seeing, as if I were in a VR) is to imagine that the figment could be the reality – which is simply incoherent, circular, self-referential nonsense.
  12.  Hermeneutics. Those who think this way have become lost in the “hermeneutic hall of mirrors,” mistaking symbols that are interpretable (by their real minds and real senses) as reflections of themselves — as being their real selves; mistaking the simulated ice-cube, for a “real” ice-cube.

Anna Netrebko: па-де-де

Аннюшка, дорогая, just figure out a way to say that you are against war (who’s for war?) without giving the slightest indication of who started this war, or why. 

I have taken some time to reflect because I think the situation is too serious to comment on without really giving it thought.

Yes, приятелька. no call for snap judgments when unprovoked bombs are again raining on Ukraine from a nuclear superpower, thrice its size, and people are dying. Just stress that you are a patriot, and apolitical.

“First of all: I am opposed to this war…

And that is of course what you say to твоему приятельнику, Володке, each time you meet…

 “I am Russian and I love my country… 

Very reassuring, and very à propos – when your country is bombing Ukraine…

“…but I have many friends in Ukraine and the pain and suffering right now breaks my heart.

To hear what you are going through breaks my heart…

“ I want this war to end and for people to be able to live in peace. This is what I hope and pray for. 

And what I wanted when I presented 1M rubles to the Russian nationalist leader, draped in the Russian secessionist flag, when my country invaded and annexed Crimea… 


“I want to add one thing, however: forcing artists, or any public figure, to voice their political opinions in public and to denounce their homeland is not right. This should be a free choice. Like many of my colleagues, I am not a political person. I am not an expert in politics. I am an artist and my purpose is to unite people across political divides.’

Which was also what I had in mind when I presented 1M rubles to the Russian nationalist leader, draped in the Russian secessionist flag, when my country invaded and annexed Crimea

To hear what you are having to go through to save your career breaks the whole world’s heart…

Age Quod Agendum (Est): Sentience and Causality

I had known about Sapolsky as a neuroendocrinologist and primatologist but had not (and have not) read his popular works. So I just looked at part of his latest podcast interview about the book he’s writing now about free will. It’s a self-help kind of book, as I suspect many of his books are. He writes about how all the genetic and experiential factors that influence what we do leave no room for free will, but that there’s still some “hope for change” because of the way that thinking, even though it is “determined,” can change brain states in ways that are not possible in other animals. I suspect this is wrong (about other animals) but it might well be another way of trying to counter depression about the feeling of helplessness. This is not the aspect of the question of free will that I (personally) find interesting. It’s the usual self-helpy, me-me obsession that not only such pop books are full of, and cater to, but I think it misses the point about what really matters, and that is not about me. 

But that’s just about me. As to free will, I agree with Sapolsky that there is no “independent” causal force – in the brain, or anywhere else – that influences the causal pattern of events. It’s all unfolding mechanically by cause and effect since the Big Bang. That it seems otherwise is probably just due to two things: 

(1) Uncertainty; there are many causal factors we don’t know and that cannot be known and predicted, so there are many “surprises” that can be interpreted as interlopers, including me and my “decisions”. The physicists say that uncertainty is not just that of statistical uncertainty (we can’t predict the weather or who will win the lottery, but not because it is not all causally determined, but just because we don’t know all the causal details); there’s supposedly also “quantum uncertainty” which is not just that we don’t know all the causal details but that some of the causal details are indeterminate: they somehow come out of nothing. (This could be true — or our understanding of quantum mechanics today may be incomplete. But in any case it has nothing to do with free will. It’s the same in all of the inanimate universe, and would have been the same even if there weren’t living, seemingly autonomous organisms — and especially one species that thinks it’s an exception to the causal picture).

(2) More important and relevant (at least in my understanding of the FW question) is the undeniable fact that FW is a feeling: Just as seeing red, hearing a loud sound, or feeling tired feels like something – and feels like something different from seeing green, hearing a faint sound or feeling peppy —  so stumbling because you lost your balance or because someone pushed you feels like something, and something different from doing it deliberately. And that same feeling (of “volition”) applies to everything you do deliberately, rather than inadvertently. That’s why I think the full-scale FW puzzle is already there in just a lowly Libet-style button press: deciding whether and when to do it, and, when you do, feeling as if “I” am the one who made it happen. It’s not a cosmic question, but a very local question, and, under a microscope, either a trivial one or, more likely, a special case of a much bigger unsolved puzzle, which is why do sentient organisms feel anything at all, whether redness, loudness, fatigue or volition? (In fact volition is the biggest puzzle, because the puzzle is a causal one, and sensations just happen to you, whereas voluntary action feels like something you are yourself causing.

The fact that there exist states that it feels like something to be in, is true, and sentient organisms all know what it feels like to feel. (That’s the only substantive part of Descartes’ “Cogito”.)

It’s also true that what has been lately dubbed the “hard problem” (but used to be called the “mind/body problem) is really just the problem of explaining, causally, why and how organisms feel. Darwinian evolution only requires that they be able to do, and be able to learn to do, whatever is needed to survive and reproduce. What is the causal contribution of feeling to the Darwinian capacities to do? What is the causal value-added of feeling? No one knows (though there are lots of silly hypotheses, most of them simply circular).

Well the FW problem (I think) is just a particular case of the hard problem of the causal role of feeling, probably the most salient case.

And it’s not the metaphysical problem of the causal power of sentient organisms’ “will” or “agency” (a misnomer) in the universe.  Organisms are clearly just causal components of the causal unfolding of the universe, not special ringers in the scheme of things.

But the puzzle remains of why they think (or rather feel) that they are – or, more generally, why they feel at all.

And that question is a causal one.

Intelligence and Empathy

“A family of wild boars organized a cage breakout of 2 piglets, demonstrating high levels of intelligence and empathy”

The capture as well as the breeding of other sentient beings for human uses are imprisonment and slavery – involuntary – and contrary to the biological imperatives of the victims. It is anthropocentric arrogance and aggression to presume that humans have a natural (or divine) right to inflict this on other sentient beings (except in cases of vital [not commercial or hedonic] conflict of biological imperatives, such as between biologically obligate carnivores and their prey).

La capture ainsi que l’élevage des autres êtres sentients pour les usages humains sont de l’emprisonnement et de l’esclavage — involontaires — à l’encontre des impératifs biologiques des victimes. C’est une arrogance et une agression anthropocentriques de présumer que les humains ont un droit naturel (ou divin) d’infliger cela à d’autres êtres sensibles (sauf en cas de conflit d’impératifs biologiques [pas les intérêts commerciaux ou hédoniques], comme entre les carnivores biologiquement obligés et leurs proies).

*IF* plants HAD feelings, how WOULD this affect our advocacy for animals?

That plants do feel is about as improbable as it is that animals (including humans) do not feel. (The only real uncertainty is about the very lowest invertebrates and microbes, at the juncture with plants, and evidence suggests that the capacity to feel depends on having a nervous system, and the behavioral capacities the nervous system produces.)

Because animals feel, it is unethical (in fact, monstrous) to harm them, when we have a choice. We don’t need to eat animals to survive and be healthy, so there we have a choice.

Plants almost certainly do not feel, but even if they did feel, we would have no choice but to eat them (until we can synthesize them) because otherwise we die.