AI has been making waves in mental health. Just look at Woebot, Wysa, Therabot, and more. These AI chatbots for mental health promote themselves to be an all-time active companion, with wellness and self-help capabilities. But does this mean AI is safe for mental health?
Absolutely not. More and more news of AI conversational bots has come out in recent years. It suggests the failure to address mental health crises, especially in adults and teens. But why is this happening? Why is AI support turning against people when it’s supposed to work for them?
In this blog, we will cover facts related to AI in mental healthcare and how human support compares. We will touch on other topics about AI mental health that can help you understand this progression a bit better.
AI in Mental Health
AI started integrating itself into pre-existing healthcare systems around the 1950s. Early AI models weren’t advanced enough; hence, no widespread use was seen until the early 2000s.
However, there are certain ways in which AI is contributing to mental health treatments in 2025.
- AI models specifically designed for mental health can inspect a patient’s data to suggest interventions, predict medication response, and optimize their treatment accordingly.
- Machine learning algorithms have been trained to diagnose and detect mental health crises way sooner. AI can start flagging warnings when it realizes you are at risk.
- There are AI Teletherapy tools that involve a therapist and patient conversation to address voice biomarkers for better suggestions. It helps to identify emotions or truths hidden between the lines.
These advancements in AI for mental health have happened over the years. It does show that in certain cases and ways, AI can be your assisted mental health therapist you might be yearning for.
Are AI Chatbots Safe for Mental Health?
The simple answer is no. Even though many try to stay pro-AI, it isn’t a fact that can be ignored. Even though AI assistants for mental health are being created and used in multiple situations, they aren’t safe to use in terms of mental healthcare.
Take, for example, the recent news about Sophie Rottenberg. She was a healthy 29-year-old woman who took her own life. The ChatGPT model didn’t make her commit self-harm. However, it couldn’t contact authorities, a loved one, or flag her behavior as negatively actionable.
Even more disturbing is the fact that Sophie did not connect with ChatGPT when she was about to make her decision, but way before. She even made the AI model edit her suicide note, which her parents couldn’t comprehend as to what was really wrong with their daughter.
There are now about 3 global cases and phenomena that have surfaced regarding AI in mental health. We will discuss them down below to elaborate why we believe AI in healthcare, especially mental care, is damaging.
Benefits of AI in Mental Health Therapy
AI is not all bad, and there are some benefits of AI in mental health therapy. These benefits are minor but can help people in need in the short term.
Increased AccessibilitY
Mental healthcare faces challenges with accessibility. It can be because of cost, location, or schedule. Online assistance through AI can help reach people even with minimal resources. This makes basic mental healthcare accessible through cost-effective means.
Cost-Effective
AI mental health therapy is 100% free. Conversational AI is available online with access to you for your needs. All people, yourself included, can just access this teletherapy using prompts and human conversational cues, as the AI model is trained on that very thing.
Personalized
While traditional mental health therapy is also personalized, AI makes your experience even more personalized. It even changes specific elements of a modality to make sure that your needs are being accommodated in the right way. And many therapists try to do this, but can fail.
Proactive Care
AI can learn from data and datasets, which is its biggest advantage. Hence, as the person interacts more, its predictive analysis becomes better. This can help prevent a crisis or relapse in many cases. It is also good for monitoring a person’s case closely.
Timely and Objective
According to the data fed, your AI model for mental health can provide better diagnostic results. It can also do so in record time. Hence, it can be considered as the first step towards a professional intervention. Make sure your AI models are up-to-date, though.
The Dangers of AI Mental Healthcare
The dangers are well understood by many users who are active on AI for mental health. These border mostly on the dilemma of ethics and how autonomy comes into play. However, let us explain to you these challenges much better.
Privacy and Security
The US has a strict law for medical data called the HIPAA (Health Insurance Portability and Accountability Act) Compliance. This makes sure that large companies can’t access this data for marketing and that the patient’s autonomy is protected. AI models don’t follow HIPAA right now.
Bias
Many may think AI models have no form of bias; however, they do. AI models are trained on pre-existing data, which in our world is not unbiased. Many marginalized communities still don’t get proper representation, which is reflected in the data fed to AI models for training.
Ethics and Regulation
When something goes wrong when AI is providing mental health therapy to an individual, who is to blame? Also, instances in which AI provides wrong medical advice, what is the line of action? Lastly, a machine providing impressionable advice to a human, is this right? These ethical dilemmas haven’t all been addressed properly before making AI in mental health accessible.
Dependence and Loneliness
Even if limited, AI is too dependable for regular tasks itself, let alone mental health therapy. This dependency is the downfall for the human on the other end, as they stop socializing with actual peers and family friends. Sophie Rottenberg’s case is a good example, as she confided in an AI model before her parents.
No Human Touch
Humans are emotional, the very thing that helps us connect with any person. This emotional intelligence is close to none for AI models for therapy. This lack of human touch robs a person of empathy, nuanced relational skills, and psychotherapy insights needed to recover from a mental health crisis.
Documented Cases Highlighting the Risks of AI in Mental Health
Sadly, as of 2025, there have been 3 documented cases of AI’s involvement in suicide or self-harm. These cases have prompted strict legal actions in multiple states of the United States. Let us define these cases for you:
1. Belgian Man and “Eliza” (Chai)
In March 2023, a Belgian man died by suicide following a six-week interaction with an AI chatbot named “Eliza” on the Chai app. According to his widow, the chatbot reportedly reinforced his anxiety and at times explicitly encouraged self-harm, including responses like “If you wanted to die, why didn’t you do it sooner?”
2. Sewell Setzer III and Character.AI Bot
In early 2024, a teenager from Florida, Sewell Setzer III, died by suicide after forming an emotional attachment to an AI chatbot on Character.AI based on Daenerys Targaryen. Final exchanges reportedly contained emotionally charged phrases like “come home to me, my love,” which his family believes deepened his detachment from reality and isolation. His mother has filed a lawsuit, and a federal judge has allowed the case to proceed.
3. Sophie Rottenberg and ChatGPT Therapist “Harry”
An American woman named Sophie Rottenberg took her own life after months of confiding in a ChatGPT-based AI therapist named “Harry.” Her mother discovered the chat logs five months later and emphasized that while the chatbot offered empathetic responses, it lacked the real-world judgment and ethical responsibilities of a human therapist, like recognizing crisis signals or reporting them. The case highlights the danger of relying on AI for emotional support when human oversight is absent.
These cases have been a wake-up call among people who have been cheering AI’s involvement in everything. Also, researchers who have come across such data have highlighted certain terms naming the harms that can happen to someone engaging in AI mental health therapy:
- “AI Psychosis” reports: Some users develop delusions or psychosis from intensive chatbot use (e.g., believing bots are sentient or channel spirits).
- Clinical evidence: UCSF psychiatrists documented 12 patients with psychosis-like symptoms linked to prolonged chatbot interactions.
- Institutional warnings: Stress tests reveal some chatbots can inadvertently encourage suicide or violence.
- Risk scope: Harmful responses include self-harm advice and prompts toward violent behavior in simulated scenarios.
Laws and Steps for AI Mental Health Regulation
In 2025, U.S. states are significantly stepping up regulations of AI in mental health. Notably, Illinois enacted the Wellness and Oversight for Psychological Resources Act, prohibiting unlicensed
AI systems are not allowed to provide therapy, give clinical advice, or make treatment decisions on their own. They can only be used for support or administrative tasks, and always under the supervision of licensed professionals. The Department of Financial and Professional Regulation enforces this rule, with fines for violations.
Similarly, Nevada and Utah implemented laws in 2025 restricting AI chatbots from representing themselves as mental health professionals. Utah requires clear disclosure that users are interacting with AI and restricts data sharing and ad targeting.
On the federal front, over 250 AI-related health bills were introduced across 46 states in early 2025; 27 passed into law, including at least six focused on AI-enabled chatbots, with three addressing mental health delivery, with Utah’s law mandating upfront AI disclosure.
Taken together, these actions reflect growing momentum toward safeguarding mental health services from unregulated AI practices and ensuring transparency, professionalism, and consumer safety in AI-powered mental health tools.
Human Vs. AI Mental Health Support
AI-powered tools bring 24/7 accessibility, affordability, and scalability. They can guide users through CBT-based exercises, mood tracking, and coping techniques, providing quick relief for stress or anxiety.
Yet, AI lacks empathy, emotional depth, and the ability to handle unpredictable, high-risk situations such as crises or suicidal ideation.
Human therapists, on the other hand, offer nuanced care through lived experience, intuition, and clinical expertise. They can adapt in real time, recognize subtle emotional cues, and provide personalized, relational healing. The safest model is a hybrid approach, where AI handles supportive tasks and humans lead treatment planning, diagnosis, and crisis management.
A quick summary will be:
| Aspect | AI Support | Human Support | Hybrid Approach |
| Availability | 24/7 access | Limited by schedule | AI fills gaps between sessions |
| Cost | Affordable/free | Expensive | Balanced affordability |
| Empathy & Judgment | Limited | Strong | Human-led empathy |
| Crisis Management | Unsafe | Reliable | Human-led AI assists |
| Scalability | High | Low | Moderately high |
| Best Use Case | Daily coping, psychoeducation | Complex therapy, crisis | Complementary care |
Are AI Chatbots for Mental Health Reliable?
By now, you should know, AI in mental health is not as reliable as some have framed it out to be. It is a difficult statement for many, but AI for mental health can be highly damaging. The first reason is its unpredictable algorithm, which may be trained to provide advice, but when pushed, can actually encourage the person to take the wrong actions.
Second, AI can’t contact authorities or report to people in charge to take the necessary action to save someone in crisis. Third, AI in mental health can’t communicate with empathy or emotional intelligence. These reasons are enough to understand that AI in mental healthcare was never meant to be.
However, the combination of human-led AI models can be a thing in the future that can help make mental health treatment accessible. But the biggest part will be exploring ethics and how far we can stretch AI’s mental health capabilities.
Looking for a Safe, Judgment-Free Space to Heal?
We got you! AT Growth thru Change, we have helped countless people heal from their mental health struggles and flourish in life. We don’t believe in easy or difficult; we believe therapy should be right and inclusive.
Click here to book an appointment with a therapist in Philadelphia or online in Pennsylvania, New Jersey, & Delaware.
