Ongoing Evaluation of AI-Use in Mental Health

Artificial Intelligence, or “AI”, tool usage related to mental health assistance has surged in since the introduction of formats such as ChatGPT (November 2022) and Claude (March 2023).  Experts are continuing to examine the role of artificial intelligence in everyday life and view the impact of AI on mental health support to be focused in a supplemental view as opposed to a replacement for professional interventions. AI was created as a tool, and professionals must consider the valuation of such tools used by clients. AI should not be viewed as a replacement for human therapists. Rather, AI and clinical professionals can coexist, with AI serving as a supportive tool while human expertise, judgment, and therapeutic relationships remain essential.

What Is A.I.?

• AI chatbots can be helpful for general wellness support, mood tracking, or self-help—not as a replacement for therapeutic interventions.
• Look for tools that clearly state they are not substitutes for professional care.

Considerations when using A.I.:

1. Always, Always, Always Review Privacy and Security Policies

• Read the privacy statement before inputting any personal details.
• Prioritize applications with end-to-end encryption and the option to delete your data.

2. Prioritize Applications by Evidence-Informed Professionals

• Select chatbots developed in collaboration with licensed mental health professionals or backed by credible institutions.
• Identify the applications use of scientific validation or peer-reviewed studies in the algorithm.

3. Exercise Critical Thought and Caution

• Verify well-being advice from chatbots with trusted mental health resources or professionals.
• AI CHATBOTS CANNOT DIAGNOSE a mental condition, medical condition or psychiatric disorder. Seek professional guidance and treatment from licensed professionals.

4. Have a Crisis Action Plan

• Create an offline crisis action plan in case of intense emotional distress (e.g., local emergency numbers, suicide hotlines, after-hours talk lines).
• If you believe you are in danger or distress, contact human support by calling 911 for life-threatening emergencies or 988 for mental health crisis support immediately rather than relying on Artificial Intelligence platforms or chatbots.

5. Touch Grass, meaning, Continue to Meet People in Real Life.

• Consider Using Both Types of Support: Balance human interaction with AI usage.
• Chatbots are to be used as a supplementary tool, not a replacement for real connection.

6. Hold These Platforms Accountable

• If the chatbot algorithm provides dangerous, misleading, or emotionally distressing responses, report it immediately to the developers or platform host.
• Avoid platforms that do not provide a way to give feedback or flag issues. Be wary of platforms that solely encourage and do not hold space for cautious action (see Sycophantic AI)

What Are The Risks of Using Mental Health Generative Chatbots?

1. Lack of Professional Oversight

• Chatbots are not licensed professionals in any capacity and cannot provide clinical diagnosis or treatment direction.
• Risk: Users relying on AI advice may delay assistance or support from licensed professionals.

2. Inaccurate or Harmful Advice

• AI models use everything found on the internet, correct or not. AI models may generate incorrect, oversimplified, dated, or culturally inappropriate suggestions.
• Risk: Following flawed advice could worsen mental health symptoms or delay effective treatment.

3. Privacy and Confidentiality: Do You Know Where Your Data Goes?

• Conversations and inquiries may be stored or analyzed to improve AI systems.
• Risk: Sensitive emotional or mental health information could be inappropriately accessed or shared if data policies are unclear or breached. (See Claude chat conversation breach from June 2026)

4. Overreliance and Emotional Dependence

• Users may form a false sense of companionship or overdependence on the chatbot resulting in negative impact to critical thinking skills.
• Risk: Reduced engagement with human social support networks or professionals.

5. Limited Crisis Support

• AI tools typically cannot recognize or respond adequately to emergencies such as suicidal ideation or self-harm. AI tools lack the flexibility and awareness to assess human emotional distress resulting in delayed, inauthentic and/or inappropriate responses to crises
• Risk: Delayed intervention in life-threatening situations.

6. Bias and Cultural Limitations

• AI Chatbots are trained to take broad datasets to provide generative responses resulting in the removal of nuance and specific cultural considerations.
• Risk: Use of Biased, insensitive, invalidating, or discriminatory responses.

7. Transparency and Accountability

• Some chatbots do not clearly disclose their limitations, data use, or overall, AI nature. Users must be reminded the AI chatbot is not a human person.
• Risk: Users may assume they are interacting with a human professional.

For further information about use of AI for mental health and well-being support, below you will find reference to the above mentioned information:

Artificial Intelligence in Counseling

Artificial intelligence, wellness apps alone cannot solve mental health crisis

Artificial intelligence in mental health care: a systematic review of diagnosis, monitoring, and intervention applications – PMC

Leveraging AI to Support Student Mental Health and Well-Being – Higher Education Today

The Rise of AI in Mental Health: Promise or Illusion? | Psychology Today