Zuckerberg Is Right About Open, Decentralized AI. He’s Also The Last Person You Should Trust To Deliver It

from the yeah,-sure,-whatever,-mark dept

The reaction to Mark Zuckerberg’s 6,500 word AI manifesto, “The Future is for Everyone,” has been all over the map. Given the messenger and the topic, there’s plenty of reasonable skepticism. 404 Media called it “deranged.” TechCrunch pointed out that Zuck’s manifesto is “exactly why people don’t like AI.” But then you have groups like Fight for the Future (certainly no fans of Zuck) pointing out that Zuck is right about the importance of open source AI and the perils of centralization.

And they can all be right. As with any 6,500 word manifesto, there are plenty of things in the screed that are silly or out of touch. But there are a few important things, including on decentralization and open source. Ignoring the silly framing of “superintelligence” (a marketing term doing a lot of load-bearing work for tools that are, so far, getting pretty good at automating certain tasks, and still really bad at lots of other stuff), he repeatedly calls out his concerns about having these new tools be too centralized:

The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems inherently problematic. Historically, hoping that an absolute power will benevolently provide for humanity if sufficiently enlightened has not led to safe or positive outcomes.

And, sure. I mean I’ve written a lot about this exact thing recently. But, um, it’s Zuckerberg and Meta that I was warning about.

Because in the social media realm, Mark Zuckerberg is the guy who is most responsible for taking us from an open, decentralized web where anyone could build to a closed, centralized data silo where if you tried to build on top of it, he’d sue you out of business.

So forgive me for not taking him seriously here. Warning that an “extreme concentration of power” is dangerous, and that hoping such a power “will benevolently provide for humanity if sufficiently enlightened” never ends well, seems like an astounding lack of self-awareness of Zuckerberg’s own role in centralizing the social internet, and positioning himself as society’s benevolent overlord.

It seems the only reason he’s pushing this vision now is… he’s losing the AI race. Badly. Funny how Zuck apparently discovered the perils of concentrated power at almost exactly the moment Meta stopped being the concentration of power that mattered, as the center of gravity shifted from social to AI.

Same with his support of open weight models. I’ve been talking about that as well for quite some time, and agree with basically everything he says here:

It is also important that the US and its allies lead the open source AI ecosystem that will make up a large percent of global AI use.

Foreign labs currently hold several advantages here since American labs have to comply with many additional restrictions on training data. US policy must reduce this additional friction if we want American open source models to lead over time. I do not believe restricting access to foreign open source models is an effective solution. Our goal should be for American open source models to be the best globally. This requires removing the hurdles that make it harder for American open source models to compete. Restricting people and companies from using the leading open source models — wherever they come from — will reduce the quality of AI accessible to them, and centralize AI rather than putting more power in people’s hands.

For the US to lead in open source, we will need to rethink our policies in several areas, including distillation and data use in training. The ability for models to learn from other models is an important principle of how the open source ecosystem works. All AI models are derived from human knowledge. Some have tried to frame distillation as harmful, but I think it is important to protect the principle that you can learn from anything you can observe. This is how the world works, and the US will not be able to lead if we restrict ourselves on this front.

But, again, Zuckerberg is the exact wrong messenger, and clearly not someone you can trust. While Meta has (like any big tech company) released and supported certain open source projects that are widely used today (mainly React and PyTorch), the company’s willingness to support openness almost always seems purely for the sake of market positioning.

Take, for example, Meta’s supposed support of the decentralized ActivityPub standard with Threads, its microblogging platform. The company initially announced that it would be building on the wider open social web and would contribute back to ActivityPub. This was positioned as being done for the good of the wider open social space. The more cynical take was that Meta knew it was way behind some others, and needed a hook to catch up. But as Threads became more successful, the company basically put that whole effort into “maintenance mode,” after the head of Threads gave an interview more or less admitting that ActivityPub integration just wasn’t that important to them any more.

And that’s before we get to the genuinely nutty parts — like the thought experiment where everyone having a superintelligent lawyer produces fairer justice rather than an unprecedented legal arms race, or the section where the answer to communities worried about data centers is a “community compact,” illustrated with $50,000 teacher bonuses in Louisiana.

Is it nice that Zuck is mentioning decentralization, less concentration of power, and open source? Sure. It’s better than the alternative. And, it’s great that Meta is now releasing new open weight models, meaning that both Google and Meta have released decently sophisticated open models that can run locally and that can compete with some of the powerful Chinese open models.

But do I actually think that Zuck cares one bit about those things? We have twenty plus years of data showing he cares about them right up until the moment he can exploit them to concentrate his own power and control.

None of that makes the good parts of his manifesto wrong. Open weights absolutely matter. Distillation restrictions are bad policy. Concentrating this stuff in three or four companies is a terrible idea. But those things were obviously true before Zuckerberg discovered them, and they’ll stay true after he abandons them.

So sure, welcome to the open, decentralized revolution, Mark. But we all know you’ll be the first to switch sides if you have the opportunity. On this stuff, principles matter and you’ve shown you’ve got none.

Filed Under: ai, centralizataion, decentralization, distillation, mark zuckerberg, open source, open weights, power

Companies: meta