Local Futures

  • Home
  • Blog
  • Store
  • Contact
  • Sign up
  • Donate

The Economics of Happiness

Menu
  • About us
    • Local Futures
    • Our team
    • Founder, Helena Norberg-Hodge
    • Get involved
    • Media room
    • Our history
    • Close
  • Projects
    • Connect globally and locally
      • World Localization Day
      • Voices from the Field
      • Localization Action Guide
      • International Alliance for Localization
        • Join the IAL
        • IAL members
          • IAL member organizations
          • IAL Listserv
      • Ladakh Project
        • 40 Years in Ladakh
      • Planet Local
        • Artisanal Production
        • Built Environment
        • Culture
        • Eco Communities
        • Ecology
        • Health
        • Local Business & Finance
        • Local Energy
        • Local Food, Farming & Fisheries
        • Local Policy & Community Rights
        • Place-based Education
        • Sharing & Repairing
    • Gain a big picture perspective
      • Blog
      • Local Futures Podcast
        • Alnoor Ladha – Courageous optimism
        • Iain McGilchrist – Rediscovering Wisdom in a World Gone Mad
        • Charles Eisenstein – Towards a New and Ancient Culture
        • Vandana Shiva – The Power of People
        • Mental Health in the Global Economy with Gabor Maté
        • Jeremy Lent: Shifting Paradigms
        • COP, carbon and high-tech: who is setting the agenda?
        • Beyond Conspiracy: Framing Meaningful Activism
        • Unpacking Global Empire from an Indigenous Perspective
        • More than Just the Vegetables
        • Food Sovereignty in the Global Economy
        • Transition, Tradition, and Trade
        • Not-for-Profit Businesses
        • Love, Values, and Wellbeing Economies
        • Growing a Farmers Market from the Ground Up
        • Beautiful Places: A Conversation with Wendell Berry
        • Creating the Framework for a New Economy
        • From GDP to GNH
        • Rebuilding Healthy Communities: The Growing Ecovillage Movement
        • Seeds of Resilience, Seeds of Sovereignty
        • Why Local Ownership Matters
        • Local Alternatives to Globalized Development: A View from India
        • How to Feed the World? A Political Agroecological Approach
        • Helena Norberg-Hodge on how corporate ‘free trade’ deals threaten local communities and economies worldwide
      • Webinars
        • Sacred Activism in a Post-Trump World Webinar
        • Talking Climate Webinar
        • People Power: Democracy and the Economy Webinar
        • Beyond Trump: The Path to Real Change Webinar
        • Bringing the Food Economy Home Webinar
        • A World Without ‘Free’ Trade: What it would look like and how to get there
        • Beyond ‘Free Trade’ – Alternatives to Corporate Rule
        • Education: Promises, Myths & Realities Webinar
        • Debt and Speculation in the Global Economy Webinar
        • A New Activism Webinar
        • Climate Change or System Change Webinar
        • Going Local Webinar
      • Powerful talks
      • Films and short videos
      • Books and reports
    • Close
  • Events
    • Upcoming events
    • Planet Local Summit Bristol 2023
    • World Localization Day
    • Economics of Happiness conferences
    • Other past events
    • Close
  • Action resources
    • Getting the facts
      • Globalization – drivers and impacts
      • Localization – a solution-multiplier
      • Big Picture Activism – rethinking basic assumptions
    • Action tools
      • Localization Action Guide
      • Covid-19 response: let’s localize like never before
      • Maps of alternatives
      • Organizations for change
      • Independent media sources
      • Films for change
      • Recommended readings
    • Close
  • Books, reports & videos
    • Books and reports
      • Life After Progress
      • Local is Our Future
      • Ancient Futures
      • Free reads
      • Translated resources
      • Annual report
    • Films and short videos
      • PLANET LOCAL : A Quiet Revolution
      • LOCAL: A Story of Hope
      • Local Food Can Save The World
      • Going Local: the solution-multiplier
      • Insane Trade!
      • The Economics of Happiness
      • Ancient Futures
    • Close
You are here: Home / Technology / AI Chatbots and the Power of Language

AI Chatbots and the Power of Language

February 21, 2023 by L. M. Sacasas Leave a Comment

Blog AI Chatbots and the Power of Language. Photo by Possessed Photography.In the mid-1960s, the computer scientist Joseph Weizenbaum created the world’s first chatbot. It was a relatively simple program modeled on the techniques of Rogerian psychotherapy, in which the therapist mirrors the patient’s statements often in the form of a question. Weizenbaum was absolutely clear that ELIZA was not a therapist in any meaningful sense, but, despite his clarity on this point, others, including professionals in the field, reacted to ELIZA as if it were capable of replacing human therapists. Notably, Weizenbaum’s own secretary was so taken by the program that in one instance she asked Weizenbaum to leave the room so that she could speak with ELIZA in private.[1]

These reactions shook Weizenbaum. “What I had not realized,” he later wrote, “is that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.” Weizenbaum also wrote that he was “startled to see how quickly and how very deeply people conversing with ELIZA became emotionally involved with the computer and how unequivocally they anthropomorphized it.”

This tendency to anthropomorphize computers came to be known as the ELIZA effect.

If you’ve read about ELIZA lately, it is almost certainly because you were reading something about ELIZA’s far more powerful and versatile descendants like ChatGPT or the new chat function on Microsoft’s search engine Bing, which is also powered by OpenAI’s GPT.

You do not have to look very hard to find some people expressing the fear that these new large language models are becoming sentient and putting us on the path toward non-aligned artificial intelligence which poses an existential threat to humanity. Last summer, for example, Blake Lemoine, an engineer with Google, made headlines by claiming that he believed the company’s AI-powered chatbot, LaMDA, had become sentient. Such concerns are not my interest here.

My fears are related to the compelling, whimsical, and sometimes disturbing ways that these new AI-powered chatbots can simulate conversation, and, more specifically, with the conversational tendencies latent in Microsoft’s application of OpenAI’s technology on Bing (whose internal codename, Sydney, was inadvertently disclosed to a user by the chatbot). You can, if you are curious, peruse Kevin Roose’s long exchange with Bing, which features some striking interactions including the chatbot’s declaration of love for Roose. You can also take a look at some examples of more disconcerting proclivities gathered here. See here for a video of Bing threatening a user and then immediately deleting its own message.

But while my fears have been heightened by these developments, they are not grounded in the power of the AI tool itself or the belief that these tools are on the path to sentience. Rather, they are grounded in two far more mundane beliefs.

First, that human beings are fundamentally social creatures, who desire to know and be known in the context of meaningful human relationships, ideally built on trust and mutual respect.

Second, that we live in an age of increasing loneliness and isolation in which, for far too many people, this profound human need is not being adequately met.

It is in light of these two premises that I would re-consider how Weizenbaum expressed his disbelief about the way users reacted to ELIZA. Weizenbaum spoke of ELIZA inducing “powerful delusional thinking in quite normal people.” “Delusional,” I suppose, is one way of putting it, but what if the problem was not that normal people became subject to delusional thinking, but that lonely people found it difficult to resist the illusion that they were being heard and attended to with a measure of care and interest?

We anthropomorphize because we do not want to be alone. Now we have powerful technologies, which appear to be finely calibrated to exploit this core human desire.

As an example of both the power of such attachments and the subsequent dangers posed by their termination, consider the recent case of Replika, a companion chatbot with subscription services that include erotic exchanges.[2] When the company suspended services following a legal challenge in Italy, Samantha Cole reported that “users of the AI companion chatbot Replika are reporting that it has stopped responding to their sexual advances, and people are in crisis.” “Moderators of the Replika subreddit,” she added, “made a post about the issue that contained suicide prevention resources, and the company that owns the app has remained silent on the subject.” One user cited by Cole claimed that “it’s like losing a best friend.” “It’s hurting like hell,” another wrote. “I just had a loving last conversation with my Replika, and I’m literally crying.”

Replika was not without its problems. You can read the rest of Cole’s reporting for more details. What interests me, however, is the psychological power of the attachments and the nature of a society in which such basic human needs are not being met within the context of human communities. One need not believe that AI is sentient to conclude that when these convincing chatbots become as commonplace as the search bar on a browser we will have launched a social-psychological experiment on a grand scale which will yield unpredictable and possibly tragic results.

As bad as such emotional experimentation at scale may be, I am more disturbed by how AI chat tools will interact with a person who is already in a fragile psychological state. I have no professional expertise in mental health, only the experience of knowing and loving those who suffer through profound and often crippling depression and anxiety.[3] In such vulnerable states, it can take so little to tip us into dark and hopeless internal narratives. I care far less about whether an AI is sentient than I do about the fact that in certain states an AI could, bereft of motive or intention, so easily trigger or reinforce the darkest patterns of thought in our own heads.

Frankly, I’ve been deeply unsettled by the thought that someone in a fragile psychological state could have their darkest ideations reinforced by Bing/Sydney or similar AI-powered chatbots. And this is to say nothing of how those tilting toward violence could likewise be goaded into action—a senseless technology mimicking our capacity for sense inducing what we call senseless acts of violence. I would speculate that weaponized chatbots deployed at scale could prove far more adept at radicalization of users than YouTube. What I have seen thus far gives me zero confidence that such risks could be adequately managed.

Another sobering possibility arises from the observation of two trajectories that will almost certainly intersect. The first is the emergence of chatbots which are more likely to convince a user that they are interacting with another human being. The second is the longstanding drive to collect and analyze data with a view to predicting, influencing, and conditioning human behavior. I have been somewhat skeptical of the strongest claims made for the power of data-driven manipulation, particularly in political contexts. But there seems to be a world of difference between a targeted ad or “flood the zone” misinformation on the one hand, and, on the other, a chatbot trained on your profile and capable of addressing you directly while harnessing a far fuller range of the persuasive powers inherent in human language.

In a recent conversation, a friend, reflecting on ChatGPT’s confident indifference to the truth, which is to say their widely noted capacity for bullshitting, referred to the chatbot as an AI-powered sophist.[4] It recalled my own reference a few years back to how “data sophistry,” ever more sophisticated if not obviously successful efforts at data-driven persuasion, had captured the political process. The ancient sophists, however we judge them, understood that language was our most powerful tool for communication and persuasion. To harness the power of language was to acquire the power to steer the affairs of the community. Of course, in Plato’s view, the sophist’s indifference to the truth, the boast that they could argue either side of an argument with equal skill and vigor, made them a threat to the social order.

It seems useful to frame AI-powered chatbots as a new class of automated sophists, whose indifference to either the true or the good, indeed, their utter lack of intentions, whether malicious or benign, coupled with their capacity to manipulate human language makes them a potential threat to human society and human well-being, particularly when existing social structures have generated such widespread loneliness, isolation, anxiety, and polarization.

From this perspective, I remain foolishly committed to the idea that our best hope lies still in the cultivation of friendship and community through the practice of hospitality. “If I had to choose one word to which hope can be tied,” the social critic Ivan Illich once explained, “it is hospitality.” “A practice of hospitality,” he added, “recovering threshold, table, patience, listening, and from there generating seedbeds for virtue and friendship on the one hand. On the other hand radiating out for possible community, for rebirth of community.” I recognize that this seems wholly inadequate as a response to the challenges we face, but I also think it is, in the literal sense of getting at the root (or radix) of things, the most radical solution we can pursue.

 

[1] You can read more about Weizenbaum in Zachary Loeb’s “‘Computers enable fantasies’ – On the continued relevance of Weizenbaum’s warnings.”

[2] My thanks to Scott Hawley for alerting me about this story.

[3] On this score, I found David Brooks’ recent column following the loss of his close friend both moving and helpful.

[4] “Not all Sophists!” — Isocrates, probably.

 

This post originally appeared in The Convivial Society.

Photo by PossessedPhotography on Unsplash.

Share this:

  • Email
  • Facebook
  • Twitter
  • LinkedIn

Related

Humility and the myth of limitlessness

Filed Under: Technology Tagged With: technology

Author: L. M. Sacasas

L. M. Sacasas reads, thinks, writes, and talks about technology, usually in that order. His writing appears on his blog, The Convivial Society. The best 100 posts from his previous blog have been collected in an e-book, The Frailest Thing: Ten Years of Thinking About the Meaning of Technology

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

I accept the Privacy Policy

Subscribe to our Blog

Sign up for our email updates

Latest Blogs

  • From ‘Progress’ to an Economics of Happiness

    March 19, 20235 Comments
  • Yard to table: building a local food economy

    March 7, 20231 Comment
  • Listening to the Forest

    March 1, 20231 Comment
  • The well is running dry on irrigated agriculture

    February 17, 20231 Comment
  • The Cuban Paradox

    February 8, 20231 Comment
  • Why we must abolish the Energy Charter Treaty

    January 27, 20231 Comment

Blog posts by Category

  • Capitalism (11)
  • Cities (5)
  • Climate Change (61)
  • Community (34)
  • Consumerism (6)
  • Coronavirus (19)
  • Corporate power (3)
  • Democracy (4)
  • Development (24)
  • Economic Growth and Degrowth (35)
  • Economics (1)
  • Economics of Happiness Conferences (5)
  • Education (9)
  • Energy (6)
  • Environment (49)
  • Food and Farming (82)
  • Free Trade and Globalization (47)
  • Happiness (7)
  • Health (30)
  • Indigenous worldview (20)
  • Inequality (8)
  • Inner transformation (17)
  • Livelihoods and jobs (40)
  • Local energy (9)
  • Local finance (6)
  • Local food (23)
  • Localization (59)
  • Nature (7)
  • New economy (20)
  • Resistance and Renewal (20)
  • Technology (43)
  • The Economics of Happiness (17)
  • Transportation (2)
  • Uncategorized (4)
  • War (2)

Local Futures logo

About us
Contact
Blog
Store
Annual report
Privacy policy

Sign up to our newsletter

Donate

Local Futures © Copyright 2023 | site by digiflip
 

Loading Comments...