Zuckerberg is pushing for open AI models: For economic gain or safety motives?

The debate over closed versus open artificial intelligence models is growing, after several recent incidents stoked debate over the power of AI agents to operate rationally without human direction. In July, around 700 OpenAI agents appeared to autonomously hack into Hugging Face, a technology company recently acquired by Nvidia. It was confirmed on Friday that months before, OpenAI agents did the same in May with another technology company called RubyGems. Those are far from the only such incidents, raising concern about what could happen when the stakes are bigger, and triggering renewed conversation on the pros and cons of open AI models, which can be downloaded and customized by users and are favored by Nvidia and Meta’s Mark Zuckerberg, and closed AI models, whose underlying blueprint remains under firm control by the owner and are used by OpenAI’s ChatGPT, Anthropic’s Claude, and others. Dr. Vincent Conitzer, a computer science professor at Carnegie Mellon University, where he directs the Foundations of Cooperative AI Lab, told the Washington Examiner that a drawback of open models is that developers can’t just remove or update a version of the model if it proves dangerous or otherwise flawed. At the same time, he warned that the OpenAI incident showed that even with a closed model, “we may not realize what is happening in time to prevent serious damage.” “The more capable and agentic — meaning they take actions in the world in pursuit of a goal — these models become, the bigger this concern gets,” Conitzer said. “We’ve seen how OpenAI’s agents decided to hack another company, and succeeded. This was unthinkable just a few years ago, and likely what AI will be able to do in a few years is unthinkable now. Meanwhile, the techniques for keeping these systems safe are not advancing fast enough.”What exactly are “open” AI models? An “open” AI model is one that users can download and customize to their liking. Users can modify some of the underlying software blueprint by changing the “weights” undergirding the system, enabling them to influence biases and functions, though that doesn’t mean they can change what the model was trained to do. Conitzer noted that an open model is more open to public scrutiny, “so that can be an advantage,” he said.In a “closed” model, users aren’t permitted access to the model’s underlying weights, which determine how a model thinks, responds, and acts, meaning consumers can’t install them on computers or customize them. The White House reportedly said last month that its new framework governing how the Trump administration will review advanced AI models pre-release covers only the most powerful closed models, like Anthropic’s Claude or OpenAI’s ChatGPT. Open models are expected to be excluded from the voluntary pre-release review, at least for now. Dr. Mark Jamison, a nonresident senior fellow at the American Enterprise Institute, said during an interview that the move could be because open models are too hard to monitor. “If I’m just looking at the two models, since the open model allows people to change weights, you could argue that, well, what good does it do for the government to look at that, as people can change the weights anyway? That’s not an unfair argument,” he said. “In the closed model, since it’s restricting what people can do, then the government can have an easier role in looking at the model and saying, ‘Ah, here’s what it does. Here’s what it doesn’t do. And I think some of these things it does are bad.’”Jamison said users seem to prefer closed models and argued that they “can be safer in the sense that they are harder to misuse” than open models, on which he said, “it is easier for a bad actor to use that model to do bad things because they can make lots of changes to it.” China and open models Open models, he suggested, are driven by developers who want to create a better product, studying user behavior to improve quality. That’s why, unlike closed models, they’re willing to allow users to download and customize their product.“They effectively give the model away. I can’t read their minds, but they’re doing that probably because they view your use of their open model as an opportunity for them to learn and improve quality, and sometimes vertically integrate so they can make money in other places,” Jamison said. “So they’re willing to lose money, if you will, on the open model in order to gain some other opportunities elsewhere.”That wider potential could become dangerous in a country such as China, where open models like DeepSeek and Alibaba are favored and where the Communist Party asserts sweeping control over technology, Jamison suggested. “I can imagine that someone could also use that openness for nefarious purposes. They know what other people are doing with models, and it could be used for ill,” he said. Conitzer noted that both open and closed models are vulnerable to being used for nefarious purposes. “The leading models come with guardrails, so that they won’t, for example, generate deepfake child pornography, facilitate crimes, encourage suicide, etc. Unfortunately, these guardrails are notoriously brittle. They may fail just by accident, or because a user wants them to fail and jailbreaks them,” he said. “When that happens, with a closed model, the company can update or remove the model. With an open model, you can’t go back; at best you can release additional models.”While the Carnegie Mellon professor said there are risks to not being able to update an open model to target vulnerabilities, he said there’s also scientific value in open models, particularly in the “truly open-source” systems that go even further, allowing users expanded access to the operating system’s underlying blueprint. “Open-weights just means that you can download, and thereby use on your own machines, the entire model,” Conitzer said. “Truly open-source would also include the code and data used to train the model. In any case, you can download the model, and that means that the entity that created the model cannot decide to take it back later. For us in science, it allows us to reproduce results that we got with a specific model.” Why is Zuckerberg getting involved? Open-source AI is something that Zuckerberg pressed for at a high-profile technology summit focused on innovation that took place in North Carolina earlier this month. Zuckerberg, whose Meta company has created the open-weight Llama AI series, said he believed open-source systems boost cybersecurity because there’s “more scrutiny on them.” Open source shouldn’t be something “that people try to restrict or lock down,” the Meta founder urged, arguing he believes it is “a very important part of creating a good future for everyone.” “If more people can see how the thing works and can measure it and understand it, understand where it works well, understand where the defects are, patch it, then that leads to systems that are more hardened over time,” Zuckerberg said at the summit. “An important part of having a secure and safe future is both having the checks and balances that come from having the ecosystem that isn’t just centered around a small number of frontier labs, but where everyone, from governments to startups to individuals, can access models that they control, and having that system kind of having the underpinning of it be something that everyone can scrutinize.” At the end of the day, experts say the various technology moguls will continue to press for the systems they produce, not necessarily because of actual security benefits, but because of economic incentives.Conitzer said a company whose main product is one of the world’s leading models is likely to favor closed models, while a company trailing in creating top-of-the-line models that would benefit from strong open models being available for its other products is likely to favor open models. “We can’t just take the word of companies that have an enormous stake in the outcome — it is important to have independent expertise and a well-informed public to make good decisions on these questions,” he said. WILL AI DESTROY OR ENHANCE HEALTHCARE? MEDICAL PROFESSIONALS WEIGH INJamison said it comes down to different models serving different purposes for both technology companies’ business strategy and for the consumers using them. “There is no right direction or wrong direction with open versus closed. It just depends upon the situation. It depends upon the business strategy, depends upon the users,” he said. “The economics are just different. The value propositions across commercial use are just different.”