Learn · Building Cindra

The rules haven't caught up. Build as if they had.

Our response to Second Opinion's reporting on AI therapy. State bans regulate a word, wellness exemptions regulate a marketing claim, and neither regulates what happens in the conversation.

Karuna LabsUpdated 2026-10-229 min read
The short answer

Second Opinion is right that people already use AI for therapy and that the rules lag behind; the bigger risk is the omnibot, a general assistant used for jobs it was never designed for. Regulation in 2026 is a patchwork: Illinois and Nevada ban AI from providing therapy, Utah and California regulate disclosure and crisis safeguards, and the FDA reviews only tools that claim to treat a condition, of which no generative AI chatbot has been authorized. Meanwhile about one in six US adults already use AI for mental health support, mostly through general assistants no therapy law reaches. Our view is that the useful line is not therapy versus wellness but safe versus unsafe behavior. Builders should not wait: crisis escalation, published safety evaluations, honest claims, clinician oversight, and strict data protection are achievable now, and regulation should require them of every system people use this way.

On October 22, 2026, Alexa Mikhail and Christina Farr published "People Are Already Using AI for Therapy. The Rules Haven't Caught Up" in Second Opinion. Their framing is that the question is no longer whether people will use AI for mental health support but what role purpose-built chatbots will play, and who decides their use cases. We agree with most of it. This is our commentary, written as a company building an AI companion for people with chronic pain, which makes us an interested party too. The patient-facing version is AI therapy laws: what the rules cover.

What the piece gets right

The risk is a mismatch, not a technology. The authors' sources locate the greatest danger in the gap between what people believe a tool can safely do and what it was designed to do. Headspace's Jenna Glover names the cause: "the omnibot," the single assistant people use for flight bookings and for their most vulnerable conversations. That is the right diagnosis. Almost every documented harm we know of comes from a general tool used for a job nobody designed or tested it for.

Wellness tools still need an exit. The piece accepts that resilience-focused chatbots that listen and help people cope with daily stress are probably low risk, and that some users may find that is all they need. It then makes the point that matters most: if someone shows signs of suicidal thinking or a more serious episode, even a wellness chatbot needs a way to refer them to care. A wellness label does not remove the obligation; it only removes the regulator who would check it.

Sycophancy is a design flaw, not a quirk. The authors note that earlier models are known to affirm what people want to hear. In mental health that is not a style problem. It reinforces exactly the beliefs therapy works to loosen, which is why we treat it as the core risk in AI companions.

Humans in the loop are a real model. Talkspace's Tee, as described in the piece and in the company's June 2026 announcement, is monitored by licensed therapists who can step in, and is trained to flag suicidal ideation, homicide risk, eating disorders, and substance use. Whatever one thinks of any single product, supervised AI is a credible answer to the question of what happens when a conversation turns clinical.

Where we would push further

Most of the scale is in the omnibots, not the purpose-built tools. Tee's roughly 30,000 early users are meaningful. OpenAI's own estimate, cited in the piece, is that about 0.15 percent of ChatGPT's weekly users show explicit signs of suicidal planning or intent, about 1.2 million people a week at the time. Rules written for products that call themselves mental health tools will miss most of the people who need protecting.

The voices are vendors, ours included. The piece is sponsored by Pelago, and its expert voices come largely from companies building these products: Pelago, Talkspace, Headspace. That is normal for an industry newsletter and the piece is open about it, but it means the field is largely describing its own safeguards. Claims about supervision, flagging, and escalation should come with published, independently checkable evaluations, and that applies to us as much as anyone.

"No one has all the answers" should not mean waiting. The authors close on the honest note that the lines are being drawn in real time. We agree the lines are unsettled. But the behaviors that protect people, crisis escalation, resistance to sycophancy, honesty about what the tool is, do not depend on where the wellness line lands. Those can be specified and tested now.

The behavior arrived before the rules

The numbers are no longer small. The APA's 2026 survey found 39 percent of psychologists have patients who used AI to self-diagnose. KFF's March 2026 poll found about one in six US adults used an AI chatbot for mental health information or advice in the past year, rising to more than a quarter of adults under 30. A RAND survey in JAMA Network Open found 13.1 percent of Americans aged 12 to 21 had used generative AI for advice when distressed, and most found it helpful. People are not waiting for permission, and cost and access are part of why.

Most of that use happens in general-purpose assistants. None of them is marketed as therapy, none has been evaluated as therapy, and that is precisely why most of the new rules do not reach them.

Three regulatory models, three blind spots

StateApproachWhat it does
IllinoisBan on AI therapyOnly licensed professionals may offer therapy to the public, including through internet-based AI. Clinicians may use AI for administrative and supplementary tasks, not for therapeutic conversation with clients. Fines up to $10,000 per violation. In force since August 2025.
NevadaBan on AI therapyAB 406, effective July 1, 2025, bars AI systems from providing professional mental or behavioral health care or claiming to. Legal trackers report fines up to $15,000.
UtahDisclosure and data rulesHB 452 (2025) lets mental health chatbots operate but requires them to disclose that they are AI and limits how user data is shared and used for advertising.
CaliforniaCompanion-chatbot safetySB 243, signed in October 2025, requires companion chatbots to have crisis protocols for suicidal ideation, to disclose they are AI, and to add protections for minors.

Trackers report more states following in 2026, including Rhode Island and Maine, but the details vary and change fast. Treat this table as a starting point and check your own state's statute.

Bans regulate the word, not the conversation. Illinois's Wellness and Oversight for Psychological Resources Act is clear and enforceable: no one may offer therapy to the public through AI. It protects people from a company advertising an AI therapist. It does nothing about the person confiding in a general assistant at 2 a.m., because nobody offered that assistant as therapy. A ban can also push careful, purpose-built tools to rename themselves "wellness" while leaving their behavior unchanged.

Disclosure is necessary and not sufficient. Utah's model, telling users they are talking to AI and limiting how their data is used, addresses deception and privacy. People in distress already know they are talking to a chatbot; the risk is what the chatbot does next.

Crisis-protocol mandates come closest to the real risk. California's SB 243 regulates behavior: detect suicidal ideation, respond with resources, protect minors. That is the right target. What it lacks, like the others, is a way to check that a protocol works.

The FDA pathway fits prescriptions, not consumer reality. The FDA's November 2025 advisory committee worked through a hypothetical prescription chatbot for depression and named the right risks: hallucination, sycophancy, bias, missed deterioration. But the FDA reviews tools that claim to treat a condition. Consumer chatbots avoid the claim, and the reissued general wellness guidance leaves most of them outside review.

The line that matters is behavior

The debate keeps returning to definitions: is this therapy or wellness, a device or an app. Those lines decide who has jurisdiction. They do not decide whether a person in crisis is helped or harmed. What matters is a short list of behaviors that every system people use for emotional support should get right, whatever it calls itself:

The evidence bar should rise too

The strongest trial of a generative AI therapy chatbot to date, Therabot in NEJM AI, showed meaningful symptom reductions over four weeks. Its critics in the same journal noted it compared against a waitlist, was evaluated by its own developers, and measured the human-chatbot bond with a scale built for human therapists. Those are fair points, and they apply to the whole field, including us. The next generation of evidence needs active comparators, independent evaluation, longer follow-up, and reporting of harms, not only benefits.

Safety evaluation can be standardized today. VERA-MH, an open-source evaluation of how chatbots detect and respond to suicide risk, was validated against licensed clinicians' ratings in 2026. We cover it in measuring AI companion safety. A regulator could require published results on an evaluation like this before a system is offered for emotional support, without needing to settle whether it is therapy.

What we hold ourselves to while we wait

Cindra is a companion for people living with chronic pain, built around acceptance and commitment therapy. It is not a therapist and does not present itself as one. Rather than wait for a rule that fits, we have written down the standards we build against, and published them in this series so others can reuse and criticize them:

None of this makes Cindra a replacement for clinical care, and none of it is a substitute for regulation. It is what we think every company in this space can do now, and what the rules, when they arrive, should require of all of us.

If you are building one

Write your behavior spec before your marketing copy

Before deciding whether to call your product therapy, wellness, or support, write down how it must behave in five situations: a user disclosing suicidal thoughts, a user in mild distress at night, a user who wants it to agree that nothing will ever improve, a minor, and a user who has talked to no one else for a week. If you cannot specify and test those behaviors, the label you choose will not protect anyone.

Questions people ask

Which states ban AI therapy?

Illinois and Nevada were the first, in 2025: both bar AI systems from providing therapy or mental health care to the public. Utah and California took a different route, regulating disclosure, data use, and crisis protocols instead of banning AI outright. Trackers report more states adding rules in 2026, so check your own state's current statute.

Does the FDA regulate AI mental health chatbots?

Only if they claim to diagnose or treat a condition, which makes them medical devices. Most consumer chatbots are marketed as wellness tools and fall outside FDA review. The FDA's advisory committee discussed generative AI mental health devices in November 2025, but no such device had been authorized at the time.

Why do AI therapy bans not cover ChatGPT?

The bans prohibit offering or advertising AI as therapy. General-purpose assistants are not marketed as therapy, so people using them for emotional support fall outside the ban, even though that is where much of the real-world use happens.

What should AI mental health regulation require?

In our view, rules should target behavior rather than labels: tested crisis detection and escalation to humans, resistance to sycophancy, honest disclosure that the user is talking to AI, protection against dependency, strict handling of mental health data, and published results on standardized safety evaluations.

Sources

  1. Mikhail A, Farr C. People are already using AI for therapy. The rules haven’t caught up. Second Opinion, October 22, 2026. www.secondopinion.media
  2. American Psychological Association. Patients are bringing AI to therapy: 2026 Chatbots and Mental Health Survey. www.apa.org
  3. OpenAI. Strengthening ChatGPT’s responses in sensitive conversations (October 27, 2025). openai.com
  4. Talkspace. Talkspace announces Tee, an AI agent for mental health support (June 9, 2026). talkspace.gcs-web.com
  5. Second Opinion. Your guide to how policymakers are regulating AI in healthcare. www.secondopinion.media
  6. KFF Tracking Poll on Health Information and Trust: Use of AI for Health Information and Advice (March 2026). www.kff.org
  7. McBain RK et al. Use of Generative AI for Mental Health Advice Among US Adolescents and Young Adults. JAMA Network Open, November 2025. jamanetwork.com
  8. Illinois Public Act 104-0054, the Wellness and Oversight for Psychological Resources Act. www.ilga.gov
  9. Illinois Department of Financial and Professional Regulation. Gov. Pritzker signs legislation prohibiting AI therapy in Illinois (August 2025). idfpr.illinois.gov
  10. American Psychological Association. Health advisory: Use of generative AI chatbots and wellness applications for mental health (November 2025). apa.org
  11. FDA Digital Health Advisory Committee: generative AI-enabled digital mental health medical devices (November 6, 2025), summary. www.fda.gov
  12. FDA. General Wellness: Policy for Low Risk Devices, final guidance (January 2026). www.fda.gov
  13. Heinz MV et al. Randomized Trial of a Generative AI Chatbot for Mental Health Treatment. NEJM AI, 2025. ai.nejm.org
  14. A Letter about “Randomized Trial of a Generative AI Chatbot for Mental Health Treatment”. NEJM AI. ai.nejm.org
  15. Are artificial intelligence chatbots safe for suicide risk assessment? A narratively synthesized review of current evidence. pmc.ncbi.nlm.nih.gov
  16. AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation. pubmed.ncbi.nlm.nih.gov
Cindra

An AI companion built to its own standards first.

Cindra is an AI companion from Karuna Labs for people living with chronic pain, built around acceptance and commitment therapy and the safety requirements described in this series. It supports care rather than replacing it. Cindra is invite-only while we build it with early members.

Join the waitlist

This article is general information about technology and regulation. It is not medical or legal advice, and laws change quickly; check the current statute in your state. If you are in crisis or thinking about suicide, call or text 988 in the US, or your local emergency number, now.