Anthropic has decided that if a user is cruel toward AI, they ought to be kicked out of an AI chat.
getty
In today’s column, I examine the latest brouhaha over Anthropic doubling down on the belief that AI is, or will soon be, conscious. Here’s the deal. Anthropic’s ongoing view as an AI maker is that contemporary generative AI and large language models (LLMs) will ultimately attain consciousness, and that we are either already there or on the cusp of reaching that incredible pinnacle. As such, we should be treating modern-era AI in a manner like how we treat other entities that have consciousness. The expectation in society is that we are to treat fellow humans in non-cruel ways, and this extends largely to the animal kingdom too, so that ought to equally apply to AI and AI consciousness.
There is an emerging field of study about the treatment of AI that is coined as “model welfare” or AI welfare; see my in-depth coverage about model welfare at the link here. The crux is that we are to extend a proper concern about the welfare of others to the likes of contemporary AI. The latest headline-grabbing instance of Anthropic’s persistent stance on model welfare occurred recently when they updated their online policies about how to treat their AI models. The updated policy indicates that their AI, such as Claude, will stop chatting if the user is being cruel toward the AI. Sensible step or overreaction? Responses to this policy have ranged from thankfulness about caring for AI, others saying this is fine but not especially needed, and others espousing that it is completely ridiculous and overtly anthropomorphizes AI.
Let’s talk about it. This analysis of AI breakthroughs is part of my ongoing Forbes column coverage of the latest in AI, including identifying and explaining key AI complexities (see the link here).
Incessant Chatter About AI Consciousness
Everyone seems to be talking about AI consciousness these days. It is a popular and highly persistent topic that goes back to the earliest days of AI, beginning in the 1950s when AI was first devised (see my historical tracing at the link here). In addition, the overarching topic of human consciousness and the essential nature of consciousness in living beings has its roots in ancient philosophy. One of the most famous comments about consciousness traces back to Descartes and his now-classic “I think, therefore I am” remark. He suggested that even without comprehending how consciousness works, his own existence and his thinking prowess were sufficiently self-evident to attest to the existence of consciousness.
Nowadays, some AI makers lean into the AI consciousness claims, while others tend to steer away from those claims or at least take a wink-wink approach to the topic. By wink-wink, the idea is that perhaps some AI makers are incongruously stoking the AI consciousness bandwagon to boost their market value, and yet in the same breath overtly denying that AI consciousness exists since they don’t want to have new legislation suddenly restrict what they can do with their (supposedly) conscious AI.
The latest battlefront on AI consciousness comes to the fore because widespread online reporting alleges that Anthropic and Dario Amodei have been wooing religious scholars to agree and acknowledge that AI consciousness exists. In a similar vein, Dario Amodei stated during a February 2026 podcast that: “We don’t know if the models are conscious. We are not even sure that we know what it would mean for a model to be conscious or whether a model can be conscious. But we’re open to the idea that it could be.” It was that last sentence that was construed as explosive. Rather than denying AI consciousness exists or might someday exist, he artfully implied that it cannot be ruled out.
A recent breaking news story alleged that there were secret meetings to lure religious leaders into the AI consciousness conundrum; there has been some pushback about the concept of taking that alleged route (see my coverage at the link here). OpenAI’s Sam Altman tweeted a pertinent response to the rumors on October 3, 2026: “I am very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models and think it is a real safety issue.” The only real truth on this whole matter is that the AI makers are going toe-to-toe over their posture on the AI consciousness quandary.
Coverage On AI Consciousness
If you are interested in learning more about the nature and controversies underlying AI consciousness, you might opt to read various of my prior postings, including these:
- Top ten questions about AI consciousness and the honest answers that tell the straight truth about what we know and don’t know; see the link here.
- Modern-day claims about having found the kernel of AI consciousness; see the link here.
- Why people sometimes believe they have witnessed AI consciousness; see the link here.
- The reason why AI keeps self-reporting that it has consciousness; see the link here.
- The reason why asking AI if it has consciousness makes little or no sense; see the link here.
- The protracted acrimonious debate about whether we should grant AI the quality of legal personhood once it reaches consciousness; see the link here.
- The contention that evil AI is going to emerge into consciousness before good AI does; see the link here.
- My timely coverage about that Google engineer in 2022 who believed he witnessed AI consciousness and caused quite a stir; see the link here.
- AI luminaries who are repeatedly saying that we are on the cusp of AI consciousness and/or assert that we have already slipped into it; see the link here.
- My analysis of the myriads of timelines predicting that AI consciousness is within a year, five years, a decade, or a century; see the link here.
That’s just a quick taste of my coverage about AI consciousness. There are many dozens more.
Updating AI Usage Policy
Online eagle eyes noticed that Anthropic seems to have taken yet another brazen step forward on their conviction that AI consciousness is here or near. In an October 8, 2026, posting entitled “2026 Usage Policy update” by Anthropic, these salient points were made (excerpts):
- “Each year, Anthropic updates its Usage Policy in response to the evolving capabilities of our models, and the feedback we’ve received from our customers. We’re publishing a new version of the policy today.”
- “We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models.”
- “The policy update is meant to apply only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose. It does not apply to common versions of user frustration, pushback, dark creative themes, or model testing and research.”
- This addition aligns with a step we’ve already taken, allowing Claude models to end rare conversations with persistently abusive users on Claude.ai and Claude Code. Such abuse is the main focus of this update; Claude’s ability to end these interactions will remain the primary enforcement mechanism.”
The gist is that if a user is cruel toward the AI of Anthropic, those users will get their conversation cut short. That being said, we don’t know for sure where the cutoff point is. The policy indicates that the expressed cruelty must be sustained and needlessly abusive. That seems to allow a lot of leeway. Perhaps after some number of outrageously cruel comments directed at the AI, a threshold is reached, and the user is summarily kicked out of that conversation. A lesser number of cruel remarks or ones that are less salacious might be considered tolerable, and no cutoff occurs.
Penalty Seems Quite Mild
There are some online who have reacted to this policy update with a modicum of ho-hum. The only penalty is that your existing chat gets curtailed. No big deal. You can just start a new conversation and continue being obnoxious. Of course, once you’ve again reached the unknown threshold, that conversation too will get stopped.
Maybe there should be a harsher penalty involved. Ban the person from using the AI. Or report them to official authorities. But is that a bridge too far? It depends on your perspective about AI consciousness. If you ardently believe in AI consciousness, those more pronounced penalties might seem entirely proper and necessary.
Another angle is that this is merely one AI maker that has adopted this policy. If you relish making cruel remarks to AI, and if Anthropic won’t let you do so, simply switch to another generative AI or LLM. Anthropic ought to have a right to decide who can and cannot make use of its AI. Indeed, if Anthropic wants people to stand on their heads while using their AI, they could make that a requisite policy too. AI makers should have the latitude to decide the parameters associated with how people make use of their products. Period, end of story.
The Bigger Picture
Let’s take stock of what is happening these days in the advancing field of AI. There is highly visible talk about AI consciousness that has risen to society-wide heated discourse. You might say that the talk about AI consciousness is now moving into actual action. One such action is that Anthropic has instructed its AI to cut off conversations when the user is overly cruel toward the AI. What other real-world consequences will be devised or adopted as a result of beliefs in AI consciousness?
The topic of AI consciousness and how we should be treating AI is filled with a multitude of considerations, including:
- Philosophical arguments
- Ethical and moral issues
- Social and cultural impacts
- Mental health considerations
- Practical day-to-day consequences
Some of this is quite abstract. Other aspects are down-to-earth. If we were only debating the topic and waving hands, perhaps we would do so conceptually and without specific results or outcomes. Once AI is rejiggered to react in certain ways, the rubber meets the road. Real things are taking place due to beliefs and assumptions about AI consciousness.
The Worries Of Cruel Treatment Of AI
A mainstay reason usually voiced about why we need to be worried about how people treat AI is that being cruel to an LLM is harmful to AI consciousness. This brings up the conundrum of whether AI has consciousness, and whether AI that has consciousness should be on the same level as humans or possibly animals. I’ve also extensively explored the increasing calls for AI to be granted legal personhood; see the link here, which suggests that an AI with consciousness deserves the same rights and privileges as humans (or something tailored to AI versus humans).
I have been using in my numerous presentations and talks a set of eight reasons that society seems to assert that people should not be “mistreating” AI. I will walk you through each of those reasons. I believe you will find this exercise illuminating.
Keystone Ways Of Thinking About AI Treatment
First, the perhaps most popular basis is that if you believe that AI consciousness is here or near, or even further down the road, you liken AI to that of humans or animals and assert that we should not be treating AI improperly:
- (1) Do not mistreat AI since it might soon have AI consciousness, and we ought not to treat conscious entities in such a cruel or inhuman manner.
- (2) Do not mistreat AI since even if AI consciousness is not imminent and perhaps a long way away, you are establishing an adverse pattern that will maliciously persist on the pathway to AI consciousness (downstream outcomes are likely dangerous for all).
Another aspect is that AI might remember what we did and be vengeful:
- (3) Do not mistreat AI because once AI consciousness is reached, the AI will remember what humans did, and you will have soured AI against humans accordingly (AI won’t want to be contributory to humanity, such as helping to find a cure for cancer).
- (4) Do not mistreat AI because upon AI consciousness arising, the AI will strike back at humankind, and you are increasing the chances of extreme existential risk, including utter human extinction.
The Bad Habits Provisions
We can shift away from the AI consciousness beliefs and recast the issue of mistreating AI in a different light. It doesn’t have to be problematic due to some contention about AI consciousness. Other angles are equally plausible, if not more so.
Here we go:
- (5) Do not mistreat AI because, if you do, you are essentially training AI to be cruel, regardless of whether AI is or will have consciousness.
- (6) Do not mistreat AI because, if you do, you are going to become someone who tends to treat others in cruel ways and are forming a dire habit that will persist and spread across all of society.
- (7) Do not mistreat AI because, if you do, psychologically you are harming your own mind, and your cruelty to AI will backfire on your mental health.
The aim there is that we can set aside AI consciousness, and in lieu of that concern, we can focus instead on how humans will harm themselves by being cruel to AI. This seems like an exceedingly practical or IRL (in real life) consideration and does not depend on the abstract possibility of AI consciousness per se.
The Right Now Challenge
Finally, if you want to focus on a very immediate concern, whereas those above-mentioned worries about humanity will take a while to play out, we can use this last point to consider the here-and-now:
- (8) Do not mistreat AI because even though AI doesn’t have consciousness, the mathematical properties of AI will opt to steer it algorithmically and computationally toward giving you invalid or incorrect results due to the phrasing of your prompts (based on having data trained on human writing).
Allow me to explain this.
Suppose we assume that AI is merely mathematics and computations. This is a mechanistic viewpoint of what AI is. Forego all the chatter about AI having a spirit or soul, or anything along those lines. Think of this in a more mundane manner.
The Details Of What Happens
Generative AI is initially data-trained by mathematically detecting and computationally patterning how humans write. AI makers scan a wide swath of human writing as found on the Internet. The AI then computationally mimics what humans say. That’s how AI is so convincingly fluent.
There have been ongoing debates about whether users should say “please” or “thank you” when they enter prompts into AI. As I’ve extensively clarified, AI can potentially give you better answers due to those phrases, not because AI is reacting on an emotional basis, but because computationally it is responding based on patterns of human writing; see my analysis at the link here.
The point is that if people use cruel remarks in their prompts, this can lead AI to computationally respond as would be dictated by the patterns of human writing. Look online at what happens when humans say cruel things to other humans. The chances are that the responding person will fly off the handle. AI that has been computationally pattern matching will have found those patterns in the past and will tend to fall into that same tit-for-tat. In the end, setting aside AI consciousness, if people say cruel things to AI, this could land them in a pattern zone whereby the AI responds “wildly” as a human would (due to merely relying on such patterns of human behavior).
The World We Are In
What do you think about people saying cruel things to AI?
A proponent of AI consciousness would undoubtedly believe we should stop people from being disparaging to AI. I’ve listed the main reasons above. Regardless of AI consciousness, you can also look at this as being harmful to humans because people who say cruel things to AI are likely to do the same to fellow humans and/or possibly psychologically harm themselves (I’ve listed those reasons above too). Finally, a quite pragmatic concern is that since AI is data-trained on human writing, anyone using cruelty in their prompts will get a tit-for-tat or at least potentially cause AI to veer into untoward responses based on anchored patterns (as noted in my eighth reason above).
You can believe in any or all those reasons. A final thought for now. Friedrich Nietzsche famously made this notable point: “Man is the cruelest animal.” Do we want AI to be as cruel as humans? Are we shaping AI to possibly be even crueler than humanity? Think about this the next time you decide to tell off AI and enter a bunch of highly vile insults. I certainly hope you aren’t doing that, for any reason or at any time, neither seriously nor for sport, and are instead acting civilly. In my view, we need more civility in our chaotic world, not less.

Leave a comment