{"id":1939,"date":"2023-06-13T12:11:47","date_gmt":"2023-06-13T11:11:47","guid":{"rendered":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/?p=1939"},"modified":"2023-06-13T15:50:27","modified_gmt":"2023-06-13T14:50:27","slug":"mirror-of-the-mind","status":"publish","type":"post","link":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/2023\/06\/13\/mirror-of-the-mind\/","title":{"rendered":"Understanding Understanding"},"content":{"rendered":"\n

HARNAD:<\/strong> Chatting with GPT is really turning out to be an exhilarating experience \u2013 especially for a skywriting addict<\/a> like me! <\/p>\n\n\n\n

From the very beginning I had noticed that skywriting can be fruitful even when you are \u201cjousting with pygmies<\/a>\u201d (as in the mid-1980\u2019s on comp.ai<\/a>, where it first gave birth to the idea of  \u201csymbol grounding\u201d). <\/p>\n\n\n\n

Who would have thought that chatting with software that has swallowed a huge chunk of 2021 vintage text and that has the capacity to process and digest it coherently and interactively without being able to understand a word of it<\/em>, could nevertheless, for users who do<\/em> understand the words — because they are grounded in their heads on the basis of learned sensorimotor features and language \u2013 provide infinitely richer (vegan) food for thought than anything ever before could, including other people \u2013 both pygmies and giants — and books.<\/p>\n\n\n\n

Have a look at this. (In the next installment I will move on to Noam Chomsky\u2019s hunch about UG and the constraints on thinkable thought.)  Language Evolution and Direct vs Indirect Symbol Grounding<\/a><\/p>\n\n\n\n

Anon: But doesn\u2019t it worry you that GPT-4 can solve reasoning puzzles of a type it has never seen before without understanding a word?<\/em><\/strong><\/p>\n\n\n\n

HARNAD:  <\/strong> I agree with all the security, social and political worries. But the purely intellectual capacities of GPT-4 (if they are separable from these other risks), and especially the kind of capacity you mention here) inspire not worry but wonder. <\/p>\n\n\n\n

GPT is a smart super-book that contains (potentially) the entire scholarly and scientific literature, with which real human thinkers can now interact dynamically — to learn, and build upon. <\/p>\n\n\n\n

I hope the real risks don\u2019t overpower the riches. (I\u2019ll be exploiting those while they last\u2026 Have a look at the chat<\/a> I linked if you want to see what I mean.)<\/p>\n\n\n\n

Anon:  <\/em><\/strong>No. It is not like a book. It converts symbolic information into interactions between features that it invents. From those interactions it can reconstruct what it has read as well as generating new stuff. That is also what people do.  It understands in just the same way that you or I do. <\/em><\/strong><\/p>\n\n\n\n

HARNAD:\u00a0<\/strong> That\u2019s all true (although \u201cunderstands\u201d is not quite the right word for what GPT is actually doing!).\u00a0<\/p>\n\n\n\n

I\u2019m talking only about what a real understander\/thinker like you and me can\u00a0use<\/strong>\u00a0GPT for\u00a0if that user\u2019s sole interest and motivation is in developing scientific and scholarly (and maybe even literary and artistic) ideas<\/em>.\u00a0<\/em><\/strong><\/p>\n\n\n\n

For such users GPT is just a richly (but imperfectly) informed talking book to bounce ideas off –with full knowledge that it does not understand a thing \u2013 but has access to a lot of current knowledge (as well as current misunderstandings, disinformation, and nonsense) at its fingertips. <\/p>\n\n\n\n

Within a single session, GPT is informing me, and I am \u201cinforming\u201d GPT, the way its database is informing it. <\/p>\n\n\n\n

Anon: You are doing a lot of hallucinating. You do not understand how it works so you have made up a story. <\/em><\/strong><\/p>\n\n\n\n

HARNAD:<\/strong> I\u2019m actually not quite sure why you would say I\u2019m hallucinating. On the one hand I\u2019m describing what I actually use GPT for, and how. That doesn\u2019t require any hypotheses from me as to how it\u2019s doing what it\u2019s doing. <\/p>\n\n\n\n

I do know it\u2019s based on unsupervised and supervised training on an enormous text base (plus some additional direct tweaking with reinforcement training from human feedback). I know it creates a huge \u201cparameter\u201d space derived from word frequencies and co-occurrence frequencies. In particular, for every consecutive pair of words within a con-text of words (or a sample) it weights the probability of the next word somehow, changing the parameters in its parameter space \u2013 which, I gather includes updating the text it is primed to generate. I may have it garbled, but it boils down to what Emily Bender called a \u201cstatistical parrot\u201d except that parrots are just echolalic, saying back, by rote, what they heard, whereas GPT generates and tests new texts under the constraint of not only \u201creductive paraphrasing and summary\u201d but also inferencing.<\/p>\n\n\n\n

No matter how much of this I have got technically wrong, nothing I said about how I\u2019m using GPT-4 depends on either the technical details that produce its performance, nor what I\u2019m doing with and getting out of it. And it certainly doesn\u2019t depend in any way on my assuming, or guessing, or inferring that GPT understands or thinks.<\/p>\n\n\n\n

What I\u2019m interested in is what is present and derivable from huge bodies of human generated texts that makes it possible not only to perform as well as GPT does in extracting correct, usable information, but also in continuing to interact with me, on a topic I know much better than GPT does, to provide GPT with more data (from me) that allows it to come back (to me) with supplementary information that allows me to continue developing my ideas in a way that books and web searches would not only have taken far too long for me to do, but could not have been done by anyone without their fingers on everything that GPT has its (imperfect) fingers on.<\/p>\n\n\n\n

And I don\u2019t think the answer is just GPT\u2019s data and analytic powers (which are definitely not thinking or understanding: I\u2019m the only one doing all the understanding and thinking in our chats). It has something to do with what the structure of language itself preserves in these huge texts<\/em>, as ungrounded as it all is on the static page as well as in GPT<\/em>.<\/p>\n\n\n\n

Maybe my hunch is wrong, but I\u2019m being guided by it so far with eyes wide open, no illusions whatsoever about GPT, and no need to know better the tech details of how GPT does what it does.<\/p>\n\n\n\n

(Alas, in the current version, GPT-4 forgets what it has learned during that session after it\u2019s closed, so it returns to its prior informational state in a new session, with its huge chunk of preprocessed text data, vintage 2021. But I expect that will soon be improved, allowing it to store user-specific data (with user permission) and to make all interactions with a user in one endless session; it will also have a continuously updating scholarly\/scientific text base, parametrized, and open web access.)<\/p>\n\n\n\n

But that\u2019s just the perspective from the disinterested intellectual inquirer. I am sure you are right to worry about the potential (perhaps inevitable) potential for malevolent use for commercial, political, martial, criminal, cult and just plain idiosyncratic or sadistic purposes.<\/p>\n\n\n\n

Anon: <\/em><\/strong>What I find so irritating is your confidence that it is not understanding despite your lack of understanding of how it works. Why are you so sure it doesn\u2019t understand?<\/em><\/strong><\/p>\n\n\n\n

HARNAD: <\/strong> Because (I have reasons to believe) understanding is a sentient<\/em> state: \u201cnonsentient\u201d understanding is an empty descriptor. And I certainly don\u2019t believe LLMs are sentient. <\/p>\n\n\n\n

Understanding, being a sentient state<\/a>, is unobservable, but it does have observable correlates. With GPT (as in Searle\u2019s Chinese Room<\/a>) the only correlate is interpretability<\/em> — interpretability by real, thinking, understanding, sentient people. But that\u2019s not enough. It\u2019s just a projection onto GPT by sentient people (biologically designed to make that projection with one another: \u201cmind-reading\u201d).<\/p>\n\n\n\n

There is an important point about this in Turing 1950<\/a> and the \u201cTuring Test<\/a>.\u201d Turing proposed the test with the following criterion. (Bear in mind that I am an experimental and computational psychobiologist, not a philosopher, and I have no ambition to be one. Moreover, Turing was not a philosopher either.)<\/p>\n\n\n\n

(1) Observation<\/strong>. The only thing we have, with one another, by way of evidence that we are sentient, is what we can observe. <\/p>\n\n\n\n

(2) Indistinguishability.<\/strong> If ever we build (or reverse-engineer) a device that is totally indistinguishable in anything and everything it can do<\/strong>, observably, from any other real, thinking, understanding, sentient human being, then we have no empirical (or rational) basis for affirming or denying of the device what we cannot confirm or deny of one another<\/em>.<\/p>\n\n\n\n

The example Turing used was the purely verbal Turing Test. But that cannot produce a device that is totally indistinguishable from us in all the things we can do<\/strong>. In fact, the most elementary and fundamental thing we can all do is completely absent from the purely verbal Turing Test (which I call \u201cT2\u201d): T2 does not test whether the words and sentences spoken are connected to the referents in the world that they are allegedly about (e.g., \u201capple\u201d and apples). To test that, indistinguishable verbal capacity (T2) is not enough. It requires \u201cT3,\u201d verbal and sensorimotor<\/strong> (i.e., robotic) capacity to recognize and interact with the referents<\/strong> of the words and sentences of T2, indistinguishably from the way any of us can do it.<\/p>\n\n\n\n

So, for me (and, I think, Turing), without that T3 robotic capacity, any understanding in the device is just projection on our part.<\/p>\n\n\n\n

On the other hand, if there were a GPT with robotic capacities that could pass T3 autonomously in the world (without cheating), I would fully accept (worry-free, with full \u201cconfidence\u201d) Turing\u2019s dictum that I have no grounds for denying of the GPT-T3, anything I that I have no grounds for denying of any other thinking, understanding, sentient person. <\/p>\n\n\n\n

You seem to have the confidence to believe a lot more on the basis of a lot less evidence. That doesn\u2019t irritate me! It\u2019s perfectly understandable, because our Darwinian heritage never prepared us for encountering thinking, talking, disembodied heads. Evolution is lazy, and not prescient, so it endowed us with mirror-neurons<\/em> for mind-reading comprehensible exchanges of speech; and those mirror-neurons are quite gullible. (I feel the tug too, in my daily chats with GPT.)<\/p>\n\n\n\n

[An example of cheating, by the way, would be to use telemetry, transducers and effectors from remote sensors which transform all sensory input from real external \u201creferents\u201d into verbal descriptions for the LLM (where is it located, by the way?), and transform the verbal responses from the LLM into motor actions on its remote \u201creferent.\u201d]<\/p>\n\n\n\n

\"\"<\/figure>\n","protected":false},"excerpt":{"rendered":"

HARNAD: Chatting with GPT is really turning out to be an exhilarating experience \u2013 especially for a skywriting addict like me!  From the very beginning I had noticed that skywriting can be fruitful even when you are \u201cjousting with pygmies\u201d (as in the mid-1980\u2019s on comp.ai, where it first gave birth to the idea of  \u201csymbol grounding\u201d).  … <\/p>\n

Continue reading “Understanding Understanding”<\/span><\/a><\/p>\n","protected":false},"author":3074,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146,106,3],"tags":[],"class_list":["post-1939","post","type-post","status-publish","format-standard","hentry","category-chatgpt","category-language","category-other-minds-problem"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts\/1939","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/users\/3074"}],"replies":[{"embeddable":true,"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/comments?post=1939"}],"version-history":[{"count":4,"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts\/1939\/revisions"}],"predecessor-version":[{"id":1945,"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/posts\/1939\/revisions\/1945"}],"wp:attachment":[{"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/media?parent=1939"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/categories?post=1939"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/generic.wordpress.soton.ac.uk\/skywritings\/wp-json\/wp\/v2\/tags?post=1939"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}