The Worst Person You Know Just Filed A Good First Amendment Lawsuit Against A Very Badly Drafted Nudify App Ban

from the fire-the-people-who-drafted-this-bill dept

There’s been a bunch of news this week regarding Minnesota’s new law that purports to prohibit “nudification” technology, and the fact that xAI has sued to have the law blocked as unconstitutional. A few things need to be said upfront, because it’s very, very easy to just say the tech is terrible, that Elon Musk and Grok are terrible, and that of course Minnesota should ban it. But it’s also possible that, in the rush to attack very problematic apps built by very problematic people, Minnesota drafted a bad law that is ridiculously overbroad and pretty clearly unconstitutional. And… that is exactly what appears to be the case.

Let’s start with the basics: apps (mostly powered by various AI tools) that are used to produce modified imagery, especially stripping people of their clothes are… bad. They should be socially shunned. People using them to objectify or sexualize others are doing bad things, and people should judge those who use those apps accordingly. This is not a defense of those apps. Similarly, Elon Musk’s Grok and its widely promoted use of putting people (including children) in bathing suits definitely deserves social shunning as well. Norms take time to form, and the shunning here is still catching up to the technology.

But passing a badly drafted, obviously unconstitutional law does not help form those norms. Nor does it punish Elon Musk. Instead, it allows him to act like a First Amendment martyr.

It’s also worth clearing something up early, because a lot of the coverage has gotten it wrong: this is not a law about child sexual abuse material. CSAM is already quite illegal under both state and federal law, and nothing in HF 1606 is limited to images of minors. Had Minnesota drafted a law narrowly targeting AI-generated CSAM, it might have survived a constitutional challenge. That’s not what it did.

And if you want to pass a law to ban technology like this, there are rules under the First Amendment. And, in Minnesota, we even know what some of those rules are. After all, a decade ago, the state also passed a law criminalizing the dissemination of “nonconsensual private sexual images.” After some back and forth in the courts, the Minnesota Supreme Court finally blessed the law as constitutional in late 2020, but made it quite clear that the law went right up to the First Amendment line. It first noted that while the state wanted to claim there’s an entirely new category of unprotected speech (in this case, “substantial invasions of privacy”), the court refused to do so, citing the famed US v. Stevens case (about an attempt to outlaw animal “crush” videos) in which the Supreme Court made it quite clear that it wasn’t open to creating new categories of unprotected speech:

The United States Supreme Court has emphatically rejected “freewheeling” attempts “to declare new categories of speech outside the scope of the First Amendment.” Stevens, 559 U.S. at 472; see also Jorgenson, 946 N.W.2d at 604 (“The United States Supreme Court has been reluctant to expand these traditional categories of unprotected speech.”). It is possible, however, there are “some categories of speech that have been historically unprotected, but have not yet been specifically identified or discussed.” Stevens, 559 U.S. at 472.

To successfully argue for a new unprotected category of speech, the proponent must present “persuasive evidence that a novel restriction on content is part of a long (if heretofore unrecognized) tradition of proscription.” Brown v. Ent. Merchs. Ass’n, 564 U.S. 786, 792 (2011). This is a heavy burden to bear, and the Supreme Court has recently rejected creating new categories of unprotected speech for animal cruelty, Stevens, 559 U.S. at 472, depictions of excessive violence, Brown, 564 U.S. at 791–93, and false statements, Alvarez, 567 U.S. at 722–23.

In this case, we conclude that the State has failed to carry the heavy burden required to provide a basis to establish a new category of unprotected speech.

And yet, the law was still deemed constitutional, but not because it created a new category of unprotected speech, but rather because it passed strict scrutiny, in which the law is narrowly tailored to use “the least restrictive means” of addressing a compelling government interest. That is the test by which a law can still be deemed viable under the First Amendment, despite suppressing speech. In the case of the nonconsensual imagery bill, the law passed strict scrutiny because it focused very narrowly on a category of speech that is very likely to cause harm, and put in place a law that was narrowly tailored to only target that speech, and on top of that included clear exemptions for edge cases that likely wouldn’t be harmful.

Indeed, the court leaned hard on the fact that the law only reached images disseminated without consent, and only when the disseminator knew or reasonably should have known the subject expected privacy. Those two limits — consent and intent — are what kept the statute from sweeping in vast amounts of protected speech. Some quotes from the court which list out all the factors necessary to pass strict scrutiny.

First, the Legislature explicitly defined the type of image that is criminalized…. Furthermore, the image has to be “obtained or created under circumstances in which the actor knew or reasonably should have known the person depicted had a reasonable expectation of privacy.” Id., subd. 1(3). Images that do not clear each of these hurdles fall outside the scope of the statute.

Second, a defendant must “intentionally” disseminate the image. … This mens rea requirement means that a defendant must knowingly and voluntarily disseminate a private sexual image; negligent, accidental, or even reckless distributions are not proscribed. This specific intent requirement further narrows the statute and keeps it from “target[ing] broad categories of speech.”

Third, the statute has seven enumerated exemptions…. The statute allows for private sexual images to be distributed “in the course of seeking or receiving medical or mental health treatment.” Id., subd. 5(3). Advertisers, booksellers, and artists are protected because images “obtained in a commercial setting” for legal purposes fall outside the statute’s reach. Id., subd. 5(4). Journalists cannot be prosecuted because there are exemptions for the dissemination of private sexual images that involve matters of public interest and “exposure[s] in public.” Id., subd. 5(4)–(5).8 Educators and scientists are protected because there is an exemption for private sexual images disseminated for “legitimate scientific research or educational purposes.” Id., subd. 5(6). Accordingly, even if protected speech falls within the ambit of subdivision one and a disseminator acted with the requisite mens rea, that person may still be exempt from prosecution under these precise exceptions.

Fourth, to be prosecuted under the statute, a disseminator must act without consent…. This provision provides additional protection for commercial advertisements, certain adult films, artistic works, and other creative expression outside the statute’s scope.

Finally, this statute only encompasses private speech…. Unlike the overly broad statutes at issue in our recent decisions in In re Welfare of A.J.B. and Jorgenson, this statute covers only private sexual images and does not prohibit speech that is “at the core of protected First Amendment speech.”

It was all of that combined that allowed the law to pass strict scrutiny — something that is incredibly difficult to do. Most laws that have to clear strict scrutiny don’t. Here, this law survived with a careful roadmap from the court of how to do so.

One would think that Minnesota legislators would be aware of this ruling and the clear reasons why the law was deemed to pass strict scrutiny and then write an equivalent law with the same elements in trying to ban nudify apps.

But for reasons known only to the Minnesota legislators, they basically ignored every single one of those points.

Minnesota’s anti-nudification tech law is not limited to non-consensual content. This means, as legal commentator Kathryn Tewson noted, that if she uploaded a picture of herself and asked Grok to put her in a bikini, she could by her own hand, cause Grok to break this law. That… seems like a very problematic law.

And, again, the Minnesota Supreme Court has already told the state pretty much exactly how to make this law constitutional: focus on nonconsensual imagery, narrowly tailor it to just the deeply harmful content, include an intent requirement, and include clear delineated exemptions for things that should be allowed.

Minnesota legislators did none of that. Indeed, even the definition of “intimate parts” in the law borrows its definition of ‘intimate parts’ from an earlier statute, covering: “the primary genital area, groin, inner thigh, buttocks, or breast of a human being” — not much of which is inherently sexual, let alone harmful. Tewson offers another example: an edit of a Taylor Swift photo that changes the texture of her fishnet stockings to look more like skin. Whatever tool made that edit just violated Minnesota law.

This is, by definition, an overly broad, non-narrowly tailored law.

Another example: last year the TV show South Park did a deepfake parody of Donald Trump, showing a photorealistic version of him wandering naked through the desert, including his “intimate parts.”

Under this law, that video could violate HF 1606. That’s not narrowly tailored. That’s not dealing with intent or focused just on truly harmful content.

One lawyer I spoke to, after reading through the statute, wondered out loud whether the Minnesota legislature had deliberately drafted it in the dumbest way possible just to guarantee a successful challenge. That’s how poorly the law was drafted.

Of course, no one wants to hear that the law is badly drafted. Lots of people want to ban nudify apps and to yell about how ridiculous it is that Elon Musk has gone to court to challenge this law.

But… it’s the sort of thing he should be doing. Otherwise anyone can have Grok put themselves in a bikini and… Minnesota’s Attorney General can demand $500,000 for each such image created, even when the image was created deliberately, by the person in it, of themselves.

xAI (now a division of SpaceX) is right to challenge the law, not because nudify apps are a good thing, but because the law is terribly drafted and pretty clearly exceeds what’s allowed under the First Amendment. The complaint itself is worth a read. For one thing, it explains why xAI last week sued one of its own users for producing CSAM with Grok (which I had found perplexing at the time). It reads a lot like the company wanted a concrete example to put in this filing of how it fights back against those who use Grok in such ways (leaving out, of course, that Elon himself used the app to put himself in a bikini, thereby encouraging others to do the same).

It also explains why that complaint was focused on triggering the indemnity clause in X’s terms of service, which makes the user liable for any legal costs associated with their use of the product. What Musk is really signalling with that lawsuit is if Minnesota’s AG sues us under this law for your usage of the product, we’re going to sue you to cover our costs (which could include the $500,000 fine for any images created).

As the lawsuit notes, the law is just terribly written:

HF 1606 punishes AI platforms that allow users to alter images of real people to depict an “intimate part.” But the statute contains no knowledge, intent, or purpose requirement. It is a strict-liability statute keyed solely to whether a user succeeded in creating a covered image using the AI provider’s platform—regardless of whether the provider prohibits users from using its tool for such a purpose, regardless of how many mitigations the provider has in place, and regardless of how diligently the provider polices such conduct using its tool. There is no safe harbor for good-faith efforts of the provider of general-purpose AI creative tools to avoid harms. Liability attaches even if the depicted persons consented—or created the image themselves—and even if the image is never shared. Liability also attaches even if the image has artistic, scientific, political, satirical, educational, medical, or religious value, and (again) even if the company has deployed near-perfect, state-of-the-art technical controls to prevent the generation of nude images.

Additionally, the law’s definition of “intimate part” is exceptionally broad. Although the federal government and various states have enacted statutes that clearly define nudity for the specific context of AI-generated images, Minnesota rejected such a precise definition. Instead, it borrowed the definition of “intimate part” from a criminal sexual-contact statute. That definition was drafted for nonconsensual touching and thus covers the inner thigh, buttocks, or breast of a man or woman, as well as the groin and primary genital area. HF 1606 accordingly bans ordinary depictions of men without shirts, people in shorts or swimsuits, and other body parts routinely displayed in public—far beyond what an ordinary person would consider “nudification.”

Even worse, as the lawsuit states, the bill’s “principal sponsor” admitted that the law was designed to apply to consensual imagery:

A service used by an adult to edit a photograph of him or herself or a consenting individual is covered on the same terms as a service used to create an image of an unwilling stranger. The statute’s text draws no distinction among them. And this was by design. When a staff member of the Senate Judiciary and Public Safety Committee pointed out that the Act’s “prohibition applies to consensual images,” Senator Maye Quade (the bill’s principal sponsor) explained “that is intentional.”

That is the bill’s main sponsor stating, on the record, that she deliberately chose to leave out one of the very features Minnesota’s own Supreme Court had identified as necessary for a law like this to survive constitutional scrutiny.

That is legislative malpractice.

Since the lawsuit was filed, Maye Quade and other legislators have publicly defended the bill:

“I don’t see this as a free speech issue. This does not regulate content; it does not regulate art. It regulates conduct,” Maye Quade said. “Prompts are not art, and we protect art specifically in this law. It’s pretty audacious to sue to prevent a law that protects children from being turned into child sexual abuse material.”

She’s describing a law she could have written, but didn’t.

Notice what’s missing from that defense: any explanation of why the consent and intent elements the Minnesota Supreme Court specifically identified as saving the 2016 law were left out of this one. Also, she’s just simply incorrect that the law does not regulate speech. Again, if she simply read what the Minnesota Supreme Court said about the nonconsensual intimate imagery law, it spent pages analyzing the nonconsensual imagery statute — a law covering narrower material than this one — as a content-based restriction on speech that had to pass strict scrutiny to survive.

Similarly, the law does not actually “protect art.” Its one and only exemption is if the work “requires the technical skill of a user to nudify an image or video.” That could protect some art, but not all. And it defines art only in a case where a level of skill is needed, which itself potentially creates First Amendment issues in defining what is, and what is not art. There is plenty of modern art that people regularly complain takes no “technical skill” to create.

The complaint itself includes some other examples of what would violate the law, including this (gross) AI-generated image that Trump posted of a slimmed down version of himself, some of his cabinet members, and a randomly generated woman in a bikini sitting in a gleaming blue reflecting pool. Under the law, whatever tool was used to generate that image pretty clearly violated Minnesota’s law:

In this viral snapshot—which President Trump posted publicly— President Trump, Vice President J.D. Vance, Secretary of State Marco Rubio, and Secretary of the Interior Doug Burgum all are portrayed shirtless in the Washington Mall’s reflecting pool, along with an unknown (possibly fictitious) woman.19 An “intimate part” (the breast) of at least the President, Vice President, Secretary of the Interior, and the woman are “depict[ed],” with the Secretary of State also at least arguably included as well. The President posted this image on his personal account, presumably to make light of the public controversy surrounding repairs to the reflecting pool on the National Mall.

Nudify apps are gross. Musk’s encouragement of people to use Grok to de-clothe people is gross. People who use AI tools to “nudify” people are gross. But that doesn’t mean all laws targeting such things are good laws or constitutional.

In this case, despite having clear instructions from its own Supreme Court on how to write a constitutional law, Minnesota’s legislature deliberately chose to write an unconstitutional one. And thus, this lawsuit is the proper thing for SpaceX/xAI/Musk to do.

Supporting the lawsuit is not supporting Elon or Grok or nudify apps. It’s telling every legislature in the country the same thing: if you want the law to survive, learn to draft it in ways that aren’t unconstitutional.

Filed Under: 1st amendment, ai, free speech, grok, maye quade, minnesota, nudify, strict scrutiny

Companies: spacex, x, xai