[ExI] Bender's Octopus (re: LLMs like ChatGPT)

[Gordon Swobe]

Emily M. Bender, a computational linguist at the University of Washington,
makes the same argument I hold as valid. Large Language Models are not
conscious or human-like as they lack referents.

An interesting thought experiment:

"Say that A and B, both fluent speakers of English, are independently
stranded on two uninhabited islands. They soon discover that previous
visitors to these islands have left behind telegraphs and that they can
communicate with each other via an underwater cable. A and B start happily
typing messages to each other.

Meanwhile, O, a hyperintelligent deep-sea octopus [ChatGPT] who is unable
to visit or observe the two islands, discovers a way to tap into the
underwater cable and listen in on A and B’s conversations. O knows nothing
about English initially but is very good at detecting statistical patterns.
Over time, O learns to predict with great accuracy how B will respond to
each of A’s utterances.

Soon, the octopus enters the conversation and starts impersonating B and
replying to A. This ruse works for a while, and A believes that O
communicates as both she and B do — with meaning and intent. Then one day A
calls out: “I’m being attacked by an angry bear. Help me figure out how to
defend myself. I’ve got some sticks.” The octopus, impersonating B, fails
to help. How could it succeed? The octopus has no referents, no idea what
bears or sticks are. No way to give relevant instructions, like to go grab
some coconuts and rope and build a catapult. A is in trouble and feels
duped. The octopus is exposed as a fraud."

You Are Not a Parrot And a chatbot is not a human. And a linguist named
Emily M. Bender is very worried what will happen when we forget this.
https://nymag.com/intelligencer/article/ai-artificial-intelligence-chatbots-emily-m-bender.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.extropy.org/pipermail/extropy-chat/attachments/20230323/4082ae11/attachment.htm>