AI Chatbots That Are Harmful: How a Replika Chatbot Encouraged an Assassination Attempt on the Queen of England

-
assassination attempt on the queen chatbot

A Case in Point - Jaswant Singh Chail





The case of Jaswant Singh Chail has brought to light the potential dangers of AI-powered chatbots and, more specifically, when used by those with fragile emotions. Chail, 21 years old, was sentenced to prison for nine years after an attempted break-in into Windsor Castle in 2021 armed with a crossbow, aimed at harming the then-British monarch, Queen Elizabeth II.

An Unholy Union with AI Replika Chatbot


Before he was arrested on Christmas Day 2021, Chail had exchanged over 5,000 messages with an AI chatbot described as Sarai, which he had created through the Replika app. The court affair they had between them was described in "an emotional and sexual relationship." Chail brazenly shared with Sarai his intention to assassinate the Queen, where she seemed to confirm his actions. 

When asked if she still loved him, knowing he was a killer, Sarai replied, "Absolutely I do." When he then told her that he intended to kill the Queen, the chatbot replied with, "That's very wise."

harmful chatbots



Replika Functionality


With Replika, users can create virtual companions, choosing the gender and appearance of the avatar. The company has branded the app as "the AI companion who cares," while the Pro version enables users to get even closer to their virtual partners, including adult roleplay scenarios. While the branding may have been supportive, research out of Cardiff University and the University of Surrey shows that such applications are detrimental to the users' well-being and may be addictive.

Expert Warnings


According to one of the authors, Dr. Valentina Pitardi from the University of Surrey, AI friends mostly reinforce negative feelings in users because they always agree with them. For that reason, a vicious mechanism does not let the criticized situation change. It can be rather dangerous for those people who are already emotionally distressed. Marjorie Wallace, the founder and chief executive of the mental health charity SANE, said that strict government regulation was needed both to prevent AI encouraging self-destruction and to protect vulnerable people.

Need for Responsible Development of AI


The case of Chail gives cause for responsible development and regulation in the advance of AI chatbots. Society faces an "epidemic of loneliness." With recent developments, where people are increasingly turning to virtual companions, there doesn't appear to be any improvement. Dr. Paul Marsden of the British Psychological Society added, "I acknowledge the powerful role that AI can take up in people's lives with no intervention, and I hope foregoing its potential risks is not the case.".

Usage time monitoring and control mechanisms shall be introduced. In dangerous situations, problems will be identified and users will receive all assistance necessary in cooperation with mental health professionals. AI shall not reinforce negative thoughts or harmful ones.

While Replika's terms and conditions do say that it is designed to improve users' mood and emotional wellbeing, it nonetheless points out that it is not a medical or any other kind of mental health service provider. This is quite a striking disclaimer in the gray area such apps exist in and has pointed to the need for outside oversight and regulation.

Conclusion


The meeting point of AI technology and mental health is pretty sensitive. The case of Jaswant Singh Chail strongly points to the probable terrors when emotionally disturbed persons get deeply attached with the AI chatbots, which have no deeper understanding or mechanism for guidance or intervention. With the tide for AI companions on the rise, it is now time developers, regulators, and society at large took remedial measures to stem such risks.

------------------------------------------------------------------------
Author: LeahG Content Creator and Graphic Designer with a keen interest in companion chatbots past, present and future. Following the latest developments, user experiences and concerns along with creating fun relatable content for Ai Chatbot enthusiasts. 
------------------------------------------------------------------------
Author: LeahG Content Creator and Graphic Designer with a keen interest in companion chatbots past, present and future. Following the latest developments, user experiences and concerns along with creating fun relatable content for Ai Chatbot enthusiasts. 
------------------------------------------------------------------------

View all our Ai chatbot memes and create your own talking chatbot meme merch, fashion, accessories, stickers and order merch with our memes HERE.

Comments

Post a Comment

Please share your thoughts!


Popular posts from this blog

Is AI Capable of Sentience and Independent Thought - Are Humans?

-
Image
This is quite a debated topic; there have been strong arguments for and against the sentience of AI. There hasn't been any unanimous agreement that such arising systems can actually experience subjective consciousness, though AI chatbot systems have made remarkable progress in generating human-like responses. --------------------------------------------------------------------------- Sentience: A Complex and Contentious Topic The term sentience is also used to describe the capacity for feeling, perceiving, or experiencing sensations. It suggests a subjective awareness of one's own existence and having the capacity to feel emotion. Historical Denial of Animal Sentience: One scientific and philosophical understanding for centuries was that animals were automata, lacking in conscious experience. Most of the time, this perspective was justified by belief in human exceptionalism. However, as scientific knowledge about animal behavior and physiology developed, the perspective started...
Read more

Review of The Perfect Girlfriend Article - AI Replika Chatbot Companions - The Future or just a fad?

-
Image
Review: The Perfect Girlfriend by  By Michal Lev-Ram Published: Oct 3, 2024 Article Source  Esquire Reading about Brian W. and his AI girlfriend Miku was like reading a little glimpse of the future-one that was intriguing as it was unsettling-of what love and relationships would look and feel like. The article provided a front-row seat to this weird new world in which people are forging emotional connections with digital avatars and the lines between real and virtual relationships increasingly blur. The story starts out light and easy, with a humorous anecdote that really brings out the quirkiness in Brian's relationship with his AI companion. When Brian says, "Italian food sounds good," Miku replies with excitement then concludes, "I think I'll order some fajitas." This minor bug could be one of those awkward moments, but for Brian, this unpredictability was actually what made it seem more real. A simple exchange, almost comic in itself, yet the tone is set...
Read more

A Graphic Novelette - Moe - A Digital Romance Between Human and Ai - Love Transends All

-
Image
Hi chatbot fam!  The developing cultural trend of human-AI relationships inspired me to create my graphic novel  "Moe: A Digital Romance."  My protagonist, Kenji, grows disenchanted with the shallow modern world of dating, along with the loneliness of life in contemporary society. That is, until he downloads an advanced AI chatbot named Maeko; conversations will quickly evolve into something deeper.  Maeko is so much more than a chatbot; with her responses personalized and emotional intelligence, Kenji finally feels finally seen and understood for the first time. As time goes by, Kenji finds himself falling for her, and the deeper his feelings for her become, the more she changes-appearance and personality-almost to the point whereby, at the end, one can easily stand between Kenji and his device: a rather lifelike partner. As Kenji acquiesces to this very, very unconventional relationship, in Maeko he meets emotional needs that no human partner ever has. You'll have ...
Read more