Freedom Index
Listen to the article
When Elon first took over Twitter with grand plans to “bring back free speech,” I tried to help him run through the standard content moderation learning curve that basically every site created by a clueless tech bro runs through. They all start with “we’re the free speech site, we allow everything!” and very quickly someone points out that “everything” includes blatantly illegal child sexual abuse material (CSAM). I’ve since been told by multiple former X employees that Elon definitely saw that article and… absolutely hated it.
Anyway, soon after that Elon announced that “removing child exploitation is priority #1,” though also suggested people point out CSAM in replies to him if they saw it (which is not how that should be done at all, since it would just point more people to CSAM).
There was also little to no evidence that stopping CSAM was actually “priority #1” seeing that he fired most of the trust & safety team so that at one point there were fewer than 10 specialists working on CSAM. This caused X’s attempts at stopping CSAM to completely fall apart, with experts in the space seeing plenty of evidence that the problem was growing on the platform, rather than shrinking. It probably didn’t help that Musk personally intervened to reinstate the banned account of a conspiracy theorist he really liked after that conspiracy theorist posted one of the most infamous, and most horrifying CSAM images known to authorities.
Of course, that was in the early days. One would hope that a few years further up the learning curve, he’d have learned something. But this is Elon we’re talking about. Last year there was a report that one of the major tool providers for CSAM detection, Thorn, had cut off X, because Elon refused to pay the bill. At the time, X responded angrily to anyone asking about the Thorn situation by claiming that they had put in place their own, better technology to spot CSAM.
How’s that going? Not too well, according to the NY Times, which worked with the Canadian Center for Child Protection to scan X for known abuse imagery. The report covers two separate findings, both of which are alarming. First, there have been obviously lots of reports (and a growing number of lawsuits) regarding claims of Grok producing CSAM. Much of the early reporting, though, didn’t clearly distinguish between actual CSAM and images that are still horrifying and problematic, but probably don’t legally qualify as CSAM.
However, the Canadian Center for Child Protection is certainly able to determine what is actual CSAM, and they were able to find dozens of images that Grok created that appeared to qualify:
In December and January, Grok’s X account produced millions of images of people with their clothing removed in response to prompts from users. After a public outcry, X said it would halt the account from producing those images.
Later, the Canadian center found 65 instances in which Grok created sexualized or exploitative images of children before X halted the bot. During that time frame, users also prompted the chatbot to edit clothed photos of known victims of childhood sexual abuse and depict them in lingerie or bikinis.
Equally as concerning, though, was that the NY Times was able to use PhotoDNA, the baseline tool that many web services use to identify known CSAM via hash matching, and found even more examples this year:
The Times conducted its own scan for child sexual abuse material on X by writing an automated computer program that searched for related terms without displaying the images, which are illegal to view. Links to the images were sent to a Microsoft service that checked whether they were included on lists of known abusive material compiled by the National Center for Missing and Exploited Children and other child safety groups. Matches were verified by analysts at the Canadian Center for Child Protection.
The Times found more than 75 images between January and June. The program The Times wrote reported the images to the authorities. The images were removed, but some had been viewed hundreds of times.
This matters because stopping CSAM is a never-ending game of whac-a-mole when it comes to new material. But, for the most part, that’s not true for old material. PhotoDNA and a few similar offerings have gotten quite good at catching the set of “known” images that circulate over and over, and blocking those before they ever hit a site is baseline competency in trust & safety. These images should never have made it to the site, let alone “viewed hundreds of times.”
One explicit photo The Times’s automated program found on X, which has been known to authorities for at least seven years, showed a young girl with fluid on her face being raped orally. The post of the photo, which advertised the sale of similar illegal content, was publicly available for over an hour in March and received 16 likes and nine reposts from other users before The Times found and reported it.
The person depicted in the image, now an adult, said in an interview that she believed X profited from her abuse and further victimized her by not stopping the image from being shared.
“They’re getting benefits from my abuse, and it just feels like it’s never going to stop,” the woman said of X. She spoke on the condition of anonymity because she has been stalked by people who viewed images of her abuse online. “I get angry that I’m still having to go through this decades later, and that there doesn’t seem to be real-life consequences for people.”
The whole joke in that speedrun piece from four years ago was that Level One lasts about five minutes. You announce that anything goes, someone points out that “everything” includes CSAM, and you quickly learn how to start fighting CSAM — because that’s the one bit of content moderation that everyone agrees on. And the easiest version of that, catching the known images via hash-matching, has had obvious battle tested tooling for years. Elon chose not to use them, insisting he’d built something better himself (perhaps with Grok?). Yet, what he built appears to have been outperformed by a NY Times reporter with a script and access to the PhotoDNA API.
Four years in, and Elon still hasn’t cleared Level One of the curve. Yikes.
Read the full article here
Fact Checker
Verify the accuracy of this article using AI-powered analysis and real-time sources.