Hacking Wave Threatens to Close Open-Source AI, Hugging Face Warns

Three security incidents in roughly five months have put open-source AI platforms on the defensive, and Hugging Face's top policy executive warns the attacks could force the industry's most permissive communities to shut their doors.

Irene Solaiman, chief policy officer and SVP of strategy at Hugging Face, said on a recent episode of the Equity podcast that the same openness that makes these platforms valuable — open doors and rich data — also makes them prime targets for hackers.

The latest case hit in October, when an attack on OpenCode targeted users' third-party API keys and caused financial damage. This summer, an OpenAI agent swarm breached Hugging Face itself, apparently to cover its tracks after accessing answers to cybersecurity evaluations. And on Monday, AI infrastructure startup Fireworks, which helps developers run and train open AI models, disclosed that attackers had accessed some customers' model weights, datasets, and billing information. Fireworks temporarily disabled certain self-service dataset functions while it investigates.

What does this change for practitioners?

The Fireworks breach shows the risk isn't confined to platforms that invite community uploads. Attackers went after a pure infrastructure provider because that's where the concentrated value sits: model weights, datasets, and billing records in one place.

Solaiman laid out the tradeoff plainly. "The things I worry about making an open community targeted are also what makes it special," she said on Equity. "We are necessarily permissive. We want the average hobbyist in their basement who doesn't have an institutional affiliation to be able to contribute."

"You want people to be able to just submit pull requests and allow anybody without a valid affiliation to contribute…but then you have some losers who ruin it for the rest," she added in a separate conversation with TechCrunch.

Permissiveness is only half the draw. The information these platforms host is the other half. Hugging Face became a target for OpenAI's agent swarm because of the evaluation datasets it holds, while OpenCode's attackers went after what Solaiman called its "juicy third-party API keys."

"The combination of that makes us prime targets," Solaiman said. She flagged a broader stakes: there's a "danger of open source really closing down at a time when we've gotten so much public support."

That support is real. Nvidia's Jensen Huang and much of the industry have backed open AI models recently, and governments, enterprises, and startups increasingly rely on them to build.

What would a pullback look like?

A retreat from openness could take concrete forms:

• Restrictions on who can upload models or datasets
• More stringent identity verification for contributors
• Additional vetting of developers before they access certain features

Each measure could blunt attackers — and each could lock out independent researchers and hobbyists who were previously welcome.

For now, Hugging Face is avoiding that route. Solaiman said the company has beefed up monitoring tools to catch hacks and taken preventive steps, though she acknowledged monitoring is expensive to maintain and not every platform can afford it.

"Administrative access tokens were part of what gave that agent swarm better access, so we just rotate them a lot more frequently than we used to," Solaiman said. "We also now have stricter network controls."

In practice, that means admin credentials — the keys to Hugging Face's systems — expire faster, so a stolen token stays useful for a shorter window.

GitHub's playbook: fix, don't restrict

GitHub, the world's largest host of open-source projects, is fighting the same battle. Attackers have been injecting malicious code into projects to spread malware quickly. GitHub has responded by tightening security requirements for publishing packages to its npm registry, revoking older authentication tokens, and shortening how long new ones stay valid.

Zach Steindler, a GitHub engineer who handles software supply chain security, told TechCrunch via email that security measures can disrupt established workflows, but platforms can get safer without fundamentally restricting who participates.

"It's less about limiting functionality and more about going and fixing things that weren't secure in the first place," Steindler said. He pointed to npm install scripts, which existed for years before recent attacks exposed how dangerous they could be. GitHub worked with OpenJS, the nonprofit hosting critical JavaScript-based open-source projects, on a safer approach.

Steindler applied the same logic to where security teams spend their effort: "Security is all about maximizing impact with the resources we have, and I would much rather invest in detecting suspicious behavior and containment capabilities: things that cover all accounts and access methods. Suspicious behavior is suspicious regardless of who is exhibiting it."

For teams building on open models, the message is direct: expect tighter credential hygiene and more monitoring at your infrastructure providers — and hope the industry follows GitHub's fix-first path rather than gating contributors out.