Spain’s new AI ‘hate’ tracker raises familiar risks for online speech

Spain is no longer just talking about regulating online “hate.” Now it’s building an AI system to track it. Fresh off an announcement that he intends to pursue an under-16 social media ban, as well as regulations holding tech owners personally liable for hateful content on their platforms and algorithms that share that material, Spanish Prime Minister Pedro Sánchez is promoting the launch of an AI program to track online hate. 

The tool, named HODIO (Huella del Odio y la Polarización) and operated by Spain’s Observatory of Racism and Xenophobia, will reportedly “analyze large volumes of publicly available activity on social media to measure the scale and spread of online hate speech.” According to Politico, HODIO’s data will then “be used to track how hateful content evolves and spreads on platforms, and will feed into a public ranking comparing how much hate speech circulates on major networks.”

It’s valuable for governments to know and understand how prejudice operates and affects the citizens they govern, but this system risks expanding government pressure on platforms to censor lawful political expression.

First, there are significant preliminary questions of how “hateful content” is defined by the agency operating HODIO and to what extent an AI tool can be expected to track and understand that content. Will the definitions hew to Spanish law and precedent, broader language from other member states under the EU’s Digital Services Act, or encompass more nebulous concepts of hurtful speech that may nevertheless receive some legal protection? And how will these tools grasp the complexities and nuances surrounding contested political speech that some, but not all, users perceive as hateful?

FIRE warned about similar concerns last year regarding the Trump administration’s plans to use AI to search through the social media accounts of international students to find targets for potential deportation based on their posts about the Israeli-Palestinian conflict. With stakes as high as these, AI cannot be tasked with the responsibility of parsing speech about a high-profile conflict and fully understanding both the nuance of language — and the First Amendment protections — in play. We cannot outsource free speech rights to a technological tool.

Most important, though, are questions about how this data — the collection and classification of which will raise challenges already — could be used not just to track hateful speech, but to guide government policy on it. When announcing HODIO’s launch, Sánchez said his government “plans to introduce a legal offense for ‘algorithmic amplification’ of hate speech” and warned that “social media must be held publicly accountable for every piece of hate content they allow.” 

Taken together with these warnings, as well as earlier promises from Sánchez about personally punishing executives for posts and banning teens from social media, it seems increasingly likely that HODIO will function not just as a tracking tool, but as a component of regulating and policing the web. 

Unless careful guardrails and caution are exercised, government efforts to understand and track bigotry can quite seamlessly morph into official censorship campaigns that infringe upon protected political speech. Just look at the expansive use of the International Holocaust Remembrance Alliance’s Working Definition of Anti-Semitism in the United States. Its primary author Kenneth Stern said it “was intended for data collectors writing reports about anti-Semitism in Europe” and “was never supposed to curtail speech on campus.” It’s done exactly that.

As with any new or emerging communicative tool, there will be concerns about the ways social media can upend society, for good or ill. But we must push back against efforts to grant governments sweeping new powers to police expression on the internet — AI-powered ones included.