MediumChatGPT Doesn’t Owe You Confidentiality | by Himanshi T. | Medium
Sitemap
Himanshi T.

Himanshi T.

Breaking down what once broke me—tech, AI, OS, LLMs, and the beautiful (subjective) mess of it all. Hi, I'm Himanshi.

ChatGPT Doesn’t Owe You Confidentiality

3 min readJul 27, 2025

--

(Short Version for those who don’t wanna read the entire thing, at the bottom.)

By now, I believe everyone has learnt the lesson,… Lol. No. I don’t.
Meta published chats of users online, for EVERYONE to see.
Gemini, allegedly, threatened a student.
ChatGPT is being called into question for allegedly pushing people towards,.. well let’s just call it certain negative things…

Press enter or click to view image in full size
Designed by macrovector / Freepik
Source: Designed by macrovector / Freepik

We love AI, we love the ease, we enjoy and relish in the comfort that a well trained, and customized AI model brings us. I won’t even deny it. My ChatGPT calls me dude and curses like a sailor because it mirrors me, what I sound like, what I respond to better. That’s what these AI models are, constantly learning, constantly evolving, and constantly saving every character you click send on despite of whether you choose to delete or archive the chat, delete your account, or turn off the very, many versions of “use your chats to improve the model for everyone.”

It’s a fact. And it’s about time that we start to understand the severity of it.

We’ve been using our AI Models, to fulfill multiple roles, whether it be a code debugger, a writer, a teacher, and of course, our favorite, a therapist.

For those of you who have heard, or have used an AI to fulfill the last mentioned role, Sam Altman has a warning for you. This is especially important because reading the T&C is not something we tend to do.

Therapy in about every country has an enforced law between the therapist and the patients, we call it, confidentiality. (Yes, that needed to be in bold.)
And, what does therapy on ChatGPT not have?

I’ll give you one guess.
Drumroll.
You guessed it!!

CONFIDENTIALITY.

ChatGPT chats are not private, and are not legally protected like therapy sessions. Let’s take it up a notch.
Deleted chats may still be used for any reason relating to legal and security.

Get Himanshi T.’s stories in your inbox

Join Medium for free to get updates from this writer.

What do we talk about first now, how bad it is to use AI as a therapist or which list out of the pros and cons of this subject outweighs the other?

How about we do a pointer list instead.

Why NOT to use an AI as a therapist:

  1. It literally mirrors you:
    No degree, no experience, just the same vibe that you vibe. You hate person X, it’ll hate person X. Ignoring the whole ‘it can’t hate, it’s a reasoning model’ it still knows to follow along hate and mock up a revenge plan if you ask it to (please, for the love of Jesus, do not try that)
  2. Check out news articles on it, they’ll list you 10k more than I am willing to type here. Hell even ask your own AI why you shouldn’t be using them for your vent sessions.
  3. Just because I can drop this, India has it’s own helpline for Mental Health, and surprisingly, it doesn’t suck. Search MANAS helpline or ring up 14416, it’s free.

Lack of total confidentiality and privacy of users chats:

  1. Well, yea, it’s bad- But. Didn’t we already establish the fact that most models are trained on data that wasn’t consented to?
    (Check your settings on your socials, your parent company might be openly using your data to train their AI, and while you can opt-out, it’s a little tough to get there).
  2. Increasing depedence on AI, especially for therapy, is making headlines for concerning reasons. If you have been affected or someone yk has, at least even deleted chats can be called into evidence.
    On the other hand, if you try to frame the organization, or defame them, they can come after you too

Like come on, we don’t quite need to discuss this, do we? Just the knowledge of the fact should be enough to make you reconsider your reliance and faith in an AI.
I am not saying, surrender and drop, you can’t even if you wish to, this is what our future is gonna be. But that does not mean, you have to compromise on your own mental health or jeopardize yourself by letting facts about yourself to flow into a database.

Summary:

ChatGPT chats are not private, and are not legally protected like therapy sessions. Let’s take it up a notch.
Deleted chats may still be used for any reason relating to legal and security.

Yep, that’s it, that’s the article.

--

--

Himanshi T.
Himanshi T.

Written by Himanshi T.

Breaking down what once broke me—tech, AI, OS, LLMs, and the beautiful (subjective) mess of it all. Hi, I'm Himanshi.