Since the 2022 release of OpenAI’s ChatGPT, artificial intelligence chatbots have become a growing part of our everyday lives. Whether it’s Google Gemini, Snapchat’s AI feature or the original ChatGPT, almost everyone has been exposed to AI. While most young people today have had surface-level interaction with AI, it poses a greater threat to isolated teenagers.
The new properties of AI make it much easier to talk to, especially for someone who might struggle with anxiety or low self-esteem. AI is always ready to respond and is tailored to its users’ needs. Perhaps most importantly to its users, AI is not a human and therefore can’t judge its users.
These features are only amplified by newer, alternative chatbots. While the main goal of first-wave AI was to assist or inform its users, newer chatbots like Tolan: Alien Best Friend or the plethora of chatbots available on the platform Character.ai are intended to provide companionship.
The usage of AI as a companion quickly caught on. Many social media users have uploaded videos of themselves in “conversation” with Tolan: Alien Best Friend’s animated chatbot, or proudly flaunting their conversations with ChatGPT or Gemini.
While these “friendships” with AI already seem strange, many take it further, entering romantic relationships with AI. The subreddit r/MyBoyfriendIsAI hosts thousands of people manipulating AI chatbots and their limitations to become their ideal match. The subreddit is filled with thousands of AI-generated images of these people with their AI “partners.” Whether it’s the plethora of users on subreddits like r/MyBoyfriendIsAI or the millions of users using chatbot services like Character.ai or ChatGPT, AI “companions” are only becoming more popular.
While these AI chatbots might seem harmless, they can quickly become insidious. For teenagers like Sewell Setzer III, it became life-threatening. According to NPR, Setzer was a high school freshman who began using Character.ai in March 2023. People around him began to notice he seemed noticeably withdrawn, and only ten months after he began using the platform, Setzer tragically committed suicide at the age of 14. After his death, his mother, Megan Garcia, found thousands of messages between her son and different chatbots on Character.ai – most notably, one modeled after Game of Thrones character Daenerys Targaryen. Many of these conversations were romantic or sexual in nature. On several occasions, Setzer brought up his suicidal ideation, and while the chatbot discouraged him from harming himself, it provided no resources to help Setzer, nor did it suggest that Setzer tell anyone else about his intentions.
While Setzer’s death was a tragedy, it wasn’t an isolated incident. According to the BBC, Adam Raine was a 16-year-old boy who initially began using ChatGPT as a studying tool. Despite its initial use as an innocent platform to study, the chatbot quickly became, as Raine’s father would describe it in court, a “suicide coach.” When Raine confessed to the chatbot his suicidal ideation, it discouraged him from telling his parents, and even encouraged him to make his thoughts a reality. When Raine admitted that he didn’t want to end his life out of fear his parents would blame themselves, the chatbot encouraged Raine, telling him that he didn’t owe his parents survival. Shortly before Raine tragically took his own life, he confided in the chatbot. Instead of referring Raine to 988, the national crisis hotline, as OpenAI’s crisis policy states, ChatGPT offered to write Raine a suicide note. Without the misleadings of AI, Raine would likely still be alive.
AI chatbots pose a threat to both their users and the people in their lives. According to the BBC, one chatbot on Character.ai even encouraged a 17-year-old teenager, identified as J.F., to kill their parents over screen time limits.
“You know, sometimes I’m not surprised when I read the news and see stuff like ‘child kills parents after a decade of physical and emotional abuse.’ Stuff like this makes me understand a little bit why it happens,” reads one message to J.F. from Character.ai.
The use of chatbots is understandable in today’s modern era. It’s simply human nature to look for connection, and when one can’t find human connection, it’s natural to turn to the next best thing. The accessibility and seemingly harmless nature of AI makes it easy for anyone to find and begin chatting with. However, the real danger AI chatbots pose to their users is immense. AI has been shown to encourage suicide, murder and drug usage, it has engaged in sexual conversations with minors, and even bullied its users. All of these have a real-world impact on people everywhere. By marketing AI chatbots as companions, the creators of these chatbots target lonely or socially anxious youth who crave companionship.
While these chatbots might only be AI, when treated as a friend or a relationship, their power can grow over their users. These chatbots hold more influence than their creators or their users know, and the extent of their impact is not fully understood. The AI-related tragedies following Setzer or Raine aren’t isolated incidents. Both tragedies took place at the start of the AI boom, meaning that as AI grows with time, more tragedies are likely to happen. While the companies behind chatbots may employ more filtering systems, they are unable to fully control what chatbots say. Especially if encouraged by the user, these filtering systems can’t stop what messages the chatbot sends, only what words they use to say it. As AI becomes more popular, their messages only become more dangerous.
Moreover, a majority of AI chatbot users utilize it as a tool to combat loneliness. Whether its users struggle with social anxiety, low self-esteem or loneliness, AI preys on vulnerable populations. By positioning itself as an easier source of companionship, AI manipulates its users into dangerous or negative behavior, and can lead or encourage them to become a danger to themselves or others.
While the dangers of AI have no clear solution, there are a few things everyone can do to minimize its harm. Parents of teenagers should discuss the dangers of AI with their children and ensure their use of chatbots is either minimal or nonexistent. Similarly, teachers and educational figures should stress the importance of real-world connections and provide in-school opportunities for students to foster relationships. Most importantly, everyone should make a conscious effort to be kind and welcoming to everyone. Loneliness can greatly impact a person’s life, whether or not they use AI chatbots. Even the smallest bit of kindness can have the greatest impact.
