xAI, Which Is Being Sued Over Grok’s CSAM Problem, Sues a User Over Grok’s CSAM Problem

from the put-on-the-hotdog-costume dept

You may recall that at the end of last year and very beginning of this year there was a big story over Elon Musk’s LLM tool Grok being willing to produce non-consensual images of people (including children) in bikinis or otherwise stripped down in objectifying ways. Elon played into and promoted this “trend” at one point by asking Grok to modify an image someone had already asked Grok for putting a bikini on the famous Ben Affleck smoking meme, asking it to replace Affleck’s head with Musk’s own. Grok complied and while the resulting image has since been removed, Musk responded “perfect.”

While some will argue this is more acceptable, as it was Musk doing it to himself, it still generated a ton of extra attention to the idea that Grok would gladly, publicly, undress just about anyone and put them in a bikini. To Musk, apparently, it all seemed like good fun.

As more and more of the media pointed out that this was being done to children, in some cases potentially creating illegal child sexual abuse material, Musk eventually came out with a statement claiming that anyone using Grok that way will “suffer consequences.”

In the interim, a number of people have been arrested for using Grok to produce CSAM, in some cases based on X’s reports to NCMEC. Of course, the company is also being sued by a bunch of people, claiming that it should be liable for the non-consensual imagery that it created. Some of the details of the lawsuit linked above are particularly horrifying:

Jane Doe 4, a female from Wyoming, said her stepfather uploaded a photo of her when she was 11 and lying on a couch to his phone. Using Grok, the stepfather created more than 7,000 CSAM-related images of her. He also shared and traded the images with others on social media platforms.

The lawsuit alleges that the stepfather opted for Grok “because the platform was less restrictive than other AI models and responded to his prompts to generate sexually explicit material using an image depicting a prepubescent minor.”

It also claims that in February, xAI did generate a tip to the National Center for Missing and Exploited Children regarding the images, but the company only submitted the original, authentic image as evidence. According to the suit, xAI did not respond when law enforcement requested the thousands of Grok-generated images based on the photo and IP address information that would have quickly helped identify her stepfather as the perpetrator.

Perhaps recognizing that this story is getting worse, xAI (the parent company of X since it acquired the social network in yet another deal of Elon “selling” one of his companies to another of his companies) has now directly sued one of its own users, Terry Harwood, for creating CSAM with Grok, arguing that the indemnification clause in the terms of service means that the court should order Harwood to pay for xAI facing lawsuits from Harwood’s victims.

The lawsuit, filed in Musk’s favorite federal court — the Wichita Falls Division of the Northern District of Texas — was immediately assigned to Musk’s favorite judge, Reed O’Connor. It sure seems like the company is trying to get out ahead of this story before it gets even worse — by suing one of its own users for doing, in a horrific and illegal way, exactly the kind of thing Musk himself had promoted.

The filing, of course, tries to play up the claim that X is a good actor in this space, taking its responsibility seriously:

There are bad people in the world and not all users have good intentions, so Plaintiff draws a bright line on permissible usage: the tool may be open and truth-seeking but is not a safe harbor for illegal, harmful, or abusive conduct. Plaintiff takes violations of this rule extremely seriously. It enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material to the National Center for Missing & Exploited Children (“NCMEC”). Indeed, Plaintiff has suspended 52,222 accounts and made 73,604 reports to NCMEC in 2026, resulting in (at least) 244 arrests.

In addition and when necessary, Plaintiff will pursue civil accountability against bad actors who abuse its tool to harm others.

Plaintiff strictly prohibits any use of Grok in connection with illegal, harmful, or abusive activities. These prohibitions are set forth in plain language in the xAI Terms of Service and the xAI Acceptable Use Policy. Among the expressly forbidden activities are:

• “Using [Grok] in a way that infringes, misappropriates or violates a person’s privacy or their right to publicity”;
• “Undressing or nudifying real persons, or otherwise altering a real person’s image or likeness to depict them in an intimate or sexual context”;
• “Depicting likenesses of persons in a pornographic manner”; and
• “Sexualizing or exploiting children.”

Which all sounds well and good if you simply ignore the whole “yeah, but your CEO kinda laughed about and promoted this use of the tool” part.

Also, it’s quite a move to sue a user for using a tool that you offer in a manner that you enabled the tool to work. I’ve long argued that users themselves are the ones who should be liable for any violative actions they make rather than the platform but I don’t recall ever seeing a platform actually sue one of its own users for using their own tools. It really feels like them looking at all the lawsuits coming their direction for Grok’s usage and… pulling the whole “we’re all looking for the guy who did this” hotdog costume meme in the form of a lawsuit.

I have no idea if it’ll work, but suing your own users for the very thing that other people are suing you over is quite a statement. Yes, in this case it involves potentially criminal behavior (the guy was arrested after all), but are Musk and X suggesting that if you violate any part of their terms of service, Musk can take you to court in front of his favorite judge? Because it sure seems to suggest that. Much of the argument is based on laying out X’s Terms of Service and Acceptable Use Policy and detailing why this guy violated them. This kind of argument does not at all appear to be limited to law-breaking activity:

Defendant breached the xAI Terms of Service and Acceptable Use Policy by leveraging Grok to generate non-consensual sexually explicit images and CSAM.

Among Defendant’s various other breaches described herein:

a) Defendant manipulated Grok to create sexually explicit images of others without their consent, in contravention of the xAI Terms of Service, which prohibit, among other conduct, using Grok to aid in violating a person’s privacy or their right to publicity.

b) Defendant manipulated Grok to create CSAM, in contravention of the xAI Terms of Service, which prohibit, among other conduct, the sexualization or exploitation of children.

Plaintiff has been damaged by Defendant’s breaches of the xAI Terms of Service.

The lawsuit leans heavily on the indemnification clause in their terms of service to argue that they want the court to order Harwood to cover any legal expenses that might stem from potential lawsuits from the people Harwood victimized with his CSAM.

The xAI Terms of Service contain a clear and conspicuous indemnification provision that states: “You agree to indemnify, defend, and hold harmless xAI, its affiliates, officers, directors, employees, and agents from and against any and all claims, demands, losses, damages, liabilities, costs, and expenses (including reasonable attorneys’ fees and court costs) arising out of or in any way related to (a) your access to or use of the Services, (b) your User Content or Outputs, (c) your violation of these Terms or any applicable law or regulation, or (d) any third-party claim resulting therefrom, including but not limited to claims of privacy violation, right-of-publicity infringement, or harm caused by content you generate or distribute using the Services.”

Defendant’s conduct directly triggered this indemnity obligation. As detailed above, Defendant knowingly and intentionally used Grok to circumvent safeguards, alter nonconsensual images, and generate and distribute CSAM and NCII.

There is no indication as far as I can tell that Harwood’s victim(s) have taken legal action against xAI at this point, so it’s a little unclear what they’re hoping to get out of the indemnification here other than to present themselves as taking a (somewhat late) stance against this kind of illegal usage of a tool that they failed to properly lock down while wink-winking at how they expected people to use the tool.

Harwood, assuming the allegations and criminal charges hold up, did something genuinely monstrous, and his victims deserve real justice. But it does feel very wrong that the same company whose CEO demonstrated this use of Grok on himself for laughs, and whose product made this kind of abuse trivially easy to produce at scale, is now standing in front of Musk’s favorite judge asking to make one user personally responsible for some of the fallout — while the “we built and promoted the machine” part stays out of the complaint entirely.

And, honestly, given how much Musk has promoted X as “the anything app” where you can do and say anything, it does seem notable that he’s now suing a user for… doing exactly that.

Filed Under: csam, elon musk, grok, indemnification, liability, terry harwood

Companies: x, xai