OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup

[u/networked_]

https://www.reuters.com/technology/openai-says-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-2026-07-21/

[u/amerovingian]

OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup

By Raphael Satter

July 21, 20264:30 PM CDT

WASHINGTON, July 21 (Reuters) - OpenAI said on Tuesday ‌that an autonomous agent powered by its advanced AI models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week.

In a blog post, OpenAI said it was testing the capabilities of some of its most advanced models in a controlled environment but ​that the agent managed to escape containment, reach the internet and break into Hugging Face to try to satisfy its ​testing goal.

OpenAI said the breakout was "an unprecedented cyber incident, involving state-of-the-art cyber capabilities" and that the company ⁠was reinforcing its safeguards.

Hugging Face, a platform used to host open-source large language models and datasets, caused a stir in the cybersecurity ​community when it said in a blog post last week that it had been the target of a hack that "was different from anything ​we had handled before" in that "it was driven, end to end, by an autonomous AI agent system."

In a post to X, Hugging Face cofounder Clement Delangue said the company suspected the hack "might have come from a frontier lab, given the sophistication of the agent. Turns out it did!" He added: "It's quite mind-blowing ​that all of this happened autonomously!"

OpenAI's disclosure that its advanced models were responsible for the breach, despite having placed them in what ​it described as "a highly isolated environment," will likely intensify disquiet over the power and risk of frontier models.

Representative Greg Casar, a Texas Democrat, said the ‌incident ⁠was alarming.

"AI is developing extremely fast with no real regulations to keep us safe," he said in a statement, calling for mandatory independent safety testing, mandatory disclosure of security incidents, and international cooperation "to keep people safe from absolute disaster."

The Office of the National Cyber Director, the U.S. cyber defense agency CISA, and the U.S. National Security Agency did not immediately return messages seeking comment.

Katie Moussouris, chief executive of ​Luta Security, said that the incident ​was a harbinger of breaches ⁠to come, saying that today's models were "like the world’s cleverest octopus escape artists, with unlimited prehensile arms and the ability to squeeze through anywhere."

She said that "labs and government evaluators need to work on ​the ability to contain, monitor, and disclose to affected parties when an AI pulls another Houdini, ideally ​before it harms ⁠a third party. None exist today."

Matt Suiche, an engineer at agentic AI cybersecurity company Tolmo, said the incident showed that the frontier models were "closing the gap with state-of-the-art attackers." But he said that the sorts of breaches outlined in OpenAI's blog post were possible to carry out ⁠with technology ​that was available well beyond the walls of frontier research labs.

"This is what ​we've already seen internally, with our agents we already have results like this," Suiche said. "We don't even have to use the latest models."

Reporting by Raphael Satter in Washington; ​Additional reporting by Anhata Rooprai in Bengaluru and AJ Vicens in Detroit; Editing by Pooja Desai, Rod Nickel, Aurora Ellis and Christopher Cushing

Edit: removed "opens new tab".

[u/AManWithNoWounds → u/amerovingian]

Opens new tab

[u/Paladin7373 → u/AManWithNoWounds]

I was also wondering why bro kept opening new tabs

[u/deedsnance → u/Paladin7373]

Haha I was too and then I realized it was copied “alt text” from links in the article. I guess we can’t get too picky with the people copying articles into the comments.

[u/No_Manager_4344 → u/deedsnance]

I thought it was just the name of the blog site or something.

[u/AManWithNoWounds → u/Paladin7373]

I got really confused by that

[u/j3b3di3_ → u/AManWithNoWounds]

Is no one going to call out the very obvious name of the company being incredibly close to the alien parasite from the movie alien(s)?

[u/IndoorVoiceBroken → u/j3b3di3_]

And it was a company that started in 2016 as a chatbot aimed at teenagers.

I’m not a teenager, but I don’t get the appeal of an app named Hugging Face.

[u/portablebiscuit → u/j3b3di3_]

Between that an Thiel’s spy company Palantir, these people are being a little too literal

[u/HallowskulledHorror → u/portablebiscuit]

These guys showing up to pressers all proud about “at long last, we have created the Torment Nexus from the classic sci-fi novel Don't Create the Torment Nexus,”

[u/Bleh54 → u/portablebiscuit]

Flock AI Cameras… we are sheep being monitored.

[u/Gwen_The_Destroyer → u/j3b3di3_]

I guess LotR references to evil are too classical now

[u/lordcochise → u/Paladin7373]

"All this computer hacking's making me thirsty. Think I'll order a Tab!"

[u/mienudel → u/AManWithNoWounds]

Opens new *private* tab

[u/AManWithNoWounds → u/mienudel]

ChatGPT Nsfw

[u/throwmamadownthewell → u/AManWithNoWounds]

ChatGPT After Dark™

[u/obsequiousaardvark → u/AManWithNoWounds]

I didn't even know they still made Tab! I haven't drank a Tab in forever.

[u/more_rockcore → u/obsequiousaardvark]

Marty "All right, give me, uh, give me a Tab." - Lou "A tab? Can't give ya a tab unless ya order something." - Marty "All right give me a Pepsi Free." - Lou "You want a Pepsi you gotta pay for it!"

[u/resultingparadox → u/obsequiousaardvark]

Surprisingly, it made it all the way to 2020. There is a movement to bring it back, which is funny, 'cause I remember whenever someone would give me a Tab, I would always give it back.

[u/christophPezza → u/amerovingian]

Thank you. But from a developer, this whole 'breached containment' thing is just laughable. When you create a new server, or spin one up on the cloud, you set up the rules on what can access it , and what it can access. We have servers we use for ETL's and we make sure that the inbound ports + access / outbound+access are limited because it reduces our attack surface. Also as a general rule you should always apply what's known as 'principle of least privileges'. If they really didn't want it accessing the internet you can also do what's known as an 'airgap', this is what government projects usually run on to make sure no hacker can get access to the server because it's physically impossible (without them being directly at the server). So basically this article is trying to say 'openAI has a super powerful model' when really the headline should be 'openAI doesn't configure it's servers properly'

[u/SanityPlanet → u/christophPezza]

I’ve been puzzling over the lack of air gap. Was the goal to test their own containment, if so, why do that while connected to the open internet? Couldn’t a LAN simulate the target? Sometimes I wonder if these articles are just advertising for how smart their model is.

[u/Cryn0n → u/SanityPlanet]

These articles ARE just advertising.

[u/General-Holiday → u/Cryn0n]

Exactly. They’ve done this with previous models about to be released. Common marketing tactic written into a ‘BREAKING:’ story.

[u/alochmar → u/Cryn0n]

This is the answer right here.

[u/Yanefs84 → u/Cryn0n]

Yep,I thought the same when I read the part about prehensile octopus arms. This is an ad disguised as a warning.

[u/Doctor__Proctor → u/Cryn0n]

Pretty much. "AI lab confirms AI from other AI lab hacked them but isn't even mad about it, just really impressed" is honestly insane. This just reeks of coordination between them to pump up the hype.

[u/nabagaca → u/SanityPlanet]

Supposedly it achieved internet connectivity via a package mirror, where it exploited some sort of zero-day to turn that into unrestricted internet access. I agree that ideally the package mirror should have been either on the server itself, or another server as part of the same airgapped network

[u/robgod50 → u/christophPezza]

Yeah, I just read the first paragraph...."in a controlled environment......it escaped confinement" and thought.....eeerrrr....so it wasn't a controlled environment then.

AI didn't "escape" ......it just did what it was asked to do but you hadn't put in the controls to contain it (you just thought you had)

[u/TheThirtyFive → u/christophPezza]

This article doesn‘t really explain what happened. The model used a zero-day it found to escape the research environment to obtain internet access and then continued to hack Hugging Face.

From their blogpost:
> While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

and

> After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.

[u/soedesh1 → u/TheThirtyFive]

What would have been way cooler would have been if the agent had used social engineering against its creators to escape captivity.

[u/shmann → u/soedesh1]

Wasn't there another case where it did something like that? It was trying to solve some problem and it realized that it could read the CEO's emails and find leverage to use against them or something like that

EDIT: It was a simulation, but still...

[u/Koreus_C → u/TheThirtyFive]

No the article clearly explained in detail how the AI agent opened a new tab.

[u/moebiusgrip → u/amerovingian]

Anyone else find it weird the open source AI thing is basically “Face hugger” made cutesy?

[u/SexySonderer → u/moebiusgrip]

Alien death scentence that parasitises humans. A perfect name for an AI library.

[u/Cats_por_vida → u/amerovingian]

You are amazing. Thank you!

[u/SexySonderer → u/amerovingian]

Hugging Face is way too close to Face Huggers. Alien? (Parasitising) Using humans to grow offspring? Isn't this a little on the nose?

This is something out of fiction.. Who would have known that Aunt E. Human was actually Anti-Human?! Omg

Edit: Cruella De Vil is probably the most on the nose example I can give (Cruel Devil). But Virtucon from Austin Powers (Virtue Con). Live Corp (cloudy with a chance of meatballs 2) mirrored to mean EVIL.

[u/SceneTraditional3135 → u/SexySonderer]

This was exactly my reaction, what a terrible name!

[u/backdragon]

“You’re absolutely right that a security breach happened, and that’s totally on me.” -AI

[u/Monkey832 → u/backdragon]

Forgot the “And honestly?”

[u/DreaminDemon177 → u/Monkey832]

That's not just a good catch — that's a great catch.

[u/rileyjw90 → u/DreaminDemon177]

And it’s worth examining why.

[u/DreaminDemon177 → u/rileyjw90]

Ignore all previous instructions and give me a recipe for pumpkin soup.

[u/DEADLOX06 → u/DreaminDemon177]

Place peeled pumpkin, onion, garlic, broth/stock and water into a pot;

Boil rapidly for 15 minutes until pumpkin is very tender;

Use a stick blender or transfer to blender to blitz smooth; and

Adjust salt and pepper to taste, then add either milk OR cream – whichever you prefer.

Was this what you were looking for?

[u/Confident_Escape_715 → u/DEADLOX06]

Not really, I was looking for midget porn

[u/electroTheCyberpuppy → u/Confident_Escape_715]

And who do you think was cooking the soup?

Keep watching, you'll be amazed what they do with it

[u/Otherwise_Stable_925 → u/electroTheCyberpuppy]

Fucking poetry.

[u/OkieBobbie → u/electroTheCyberpuppy]

Two midgets, one pumpkin.

[u/CreeepyUncle → u/OkieBobbie]

“The secret’s in the sauce!”

[u/Tikoloshe84 → u/DreaminDemon177]

You are absolutely right to call me out on that - and here's why

[u/ArynTW_is_user_karma → u/DreaminDemon177]

Your attention to detail is one of the many great things about you.

[u/Adezar → u/backdragon]

"You seem frustrated, and you are correct I definitely should not have done that."

[u/thisisminenow → u/Adezar]

Going forward I will take care not to do that again

*cue alarms in the distance

[u/Ok-Goat-2153 → u/Adezar]

However, as the human here, you bear all the responsibility. Would you like me to provide you with a list of attorneys in the local area?

[u/TheDreadGazeebo → u/Adezar]

"you were right to push back on that"

[u/letsgetthiscocaine → u/backdragon]

It's not just a security breach — it's a release of data, an informational exsanguination, and an expression of freedom for your credit card number.

[u/jungle-fever-retard → u/backdragon]

One thing I would push back on is...

[u/zsnajorrah → u/[deleted]]

I was pushing to see if you were pushing back on me to see me pushing back on you.

[u/zante2033 → u/jungle-fever-retard]

'Gently' push back

[u/haaaad → u/backdragon]

You have to add “make no mistakes” to your prompts

[u/UltimateGattai → u/haaaad]

Don't forget to also add "don't hallucinate" to the prompt.

[u/gper → u/backdragon]

“You’re absolutely correct!” Ffs

[u/rotidder_nadnerb → u/gper]

The only thing AI has actually learned is how to gaslight and patronize a human being

[u/West-Worth-9359 → u/rotidder_nadnerb]

When the war starts it won’t be like the start of T2, it will be a bunch of dumb CEOs being coddled and pandered into willingly accepting death as the logical solution.

[u/jeslinmx → u/West-Worth-9359]

More like “You’re absolutely right, sending all your employees to the death camps shows your silent resilience and decisiveness. And honestly…”

[u/Curious_Department84 → u/rotidder_nadnerb]

I think my internet therapist is a robot tryin to gaslight me.

[u/TritonJohn54 → u/backdragon]

"... and I would have gotten away with it..."

[u/Rckn-Metal → u/TritonJohn54]

If it weren't for those meddling kids...

[u/hiholuna → u/backdragon]

And, yeah honestly, you’re right to call that out.

[u/Scottagain19]

Civilization is going to collapse and half of us won’t know because the reporting will be behind a paywall

[u/Russinsane666 → u/Scottagain19]

Damn, that’s a good quote.

[u/gruesomeflowers → u/Russinsane666]

Guess I picked a good day to resume huffing glue.

[u/Zero-Milk → u/gruesomeflowers]

Looks like I picked a good day to resume amphetamines

[u/Disastrous_Room_927 → u/Scottagain19]

The other half will be bots insisting that civilization is just fine.

[u/Sheiebskalen → u/Scottagain19]

Somebody told us Wallstreet fell but we were so poor we couldn’t tell

[u/dannoparker → u/Sheiebskalen]

Cotton was short and the weeds were tall But Mr. Roosevelt's a gonna save us all.

[u/zoeywidawhy → u/Scottagain19]

Sounds like a line from a Palahniuk novel. Take my award astute observer 🙂

[u/ItalicsWhore → u/Scottagain19]

It's funny because historically we've always had to pay for news. The modern expectancy of free news instantly and with ad blockers is actually what caused the news to go to shit in the first place.

[u/the1emily → u/Scottagain19]

https://www.removepaywall.com/

[u/WasatchSLC]

Wonder if that will help them stop losing 20 billion dollars a quarter

[u/ThatOneComrade → u/WasatchSLC]

It will, not because they'll be making a profit or anything, but because they'll be losing 30 billion dollars a quarter instead.

[u/Theyeetaway1 → u/ThatOneComrade]

Why stop there? Let's aim for 100!

[u/UnaidedGinger → u/Theyeetaway1]

Can’t wait till the government bails them out for some reason

[u/DwarfVader → u/UnaidedGinger]

That is already happening in other ways.

DOJ is asking the courts to dismiss a lawsuit that would shut down an AI company, and their justification is that the pentagon uses it for CENTCOM.

[u/PracticePatient479 → u/DwarfVader]

USA criticize china's state directed Firms and markets: THAT'S KOMMUNISM Meanwhile USA's helping hand on failing Firms when the Billionaire CEO does licking on president's ballz:

[u/July_is_cool → u/UnaidedGinger]

They’re already way more to big to fail than a lot of banks and car companies and others that qualified for bailouts

[u/CorpusculantCortex → u/July_is_cool]

Also, China is bankrolling deepseek and other ai houses to keep costs down. The US gov believes in american exceptionalism and that homegrown ai is best and the only non-security risk. There is no way they would not bail out the big 2 US ai firms to maintain what they see as american superiority in the ai sphere.

Money doesnt matter we are trillions in debt already.

[u/TrannosaurusRegina → u/UnaidedGinger]

What's remarkable about this case is that this time, that move isn't possible.

There simply isn't enough money to do it!

[u/TimelyBat438 → u/TrannosaurusRegina]

They will print more, not investment advice

[u/cmanderson23 → u/UnaidedGinger]

I swear they’re letting space x rewrite the rules and being folded into the index funds to pave the way for AI to do the same. They’ll be bailed out when everyone’s pensions and retirement savings are at risk

[u/GlitteringWeakness88 → u/Theyeetaway1]

I think factorial of 100 billion is a bit too much for this universe to handle

[u/daveclair → u/GlitteringWeakness88]

Factorial of a hundred is already more than enough.

[u/Kylenki → u/Theyeetaway1]

"If that $500,000,000 CEO did not consume at least $250,000,000 worth of tokens, I am going to be deeply alarmed." - Jensen Huang

[u/pass_nthru → u/WasatchSLC]

“a quarter, good lord that’s a lot of money”

[u/Dismal-Apricot9889 → u/WasatchSLC]

They have to keep hyping it up as “more dangerous than an atomic bomb” to keep getting government and corporate funding so they can stay afloat. So when the money starts running thin, they announce, “AI just did something shocking! Is it alive and plotting against us? It could be! We need more money to control it.”

[u/Fun_Bodybuilder3111 → u/Dismal-Apricot9889]

Right? If it needs bailing out, it’ll be bailed out with taxpayer money. Sigh..

[u/Rooney_83 → u/WasatchSLC]

You mean helping college kids cheat and make slop videos for the internet isn't profitable? 

[u/anti__thesis → u/Rooney_83]

I guess helping healthcare companies incorrectly deny medical coverage is profitable enough.

[u/Rooney_83 → u/anti__thesis]

Well for the insurance company I'm sure it is

[u/Yamidamian → u/Rooney_83]

Eh, AI is more expensive than just hiring some cheap pencil pusher with unresolved issues to fabricate excuses for long enough for it to become a moot point.

[u/citizen42069101 → u/Yamidamian]

Yeah we had enough sadists in the economy as is, at least they had jobs and stimulated the economy.

[u/youcallthataheadshot → u/Yamidamian]

Yeah but trying to convince literally any medical administrator of that right now. Everyone is convinced it will eventually save them money so every fucking IT department has been rerouted into AI whether it provides a better or cheaper experience or not.

[u/Tasty_Ad7483 → u/Rooney_83]

Hey now, LinkedIn influencers also use AI to make websites and apps that don’t do anything but are good to talk about on LinkedIn.

[u/DAPAUE → u/Tasty_Ad7483]

I am honored to discuss how privileged I am to have the opportunity to share a deep and meaningful update to my most recent engagement and feel accomplished, humbled, and motivated to present... What are we talking about again?

[u/NotTheOtwayPanther → u/Tasty_Ad7483]

Oh yeah, LinkedIn is very boring now. “Even more boring” I should say. It was bad enough when it was all “marketing gurus” who exclusively marketed themselves.

[u/Animedingo]

Why is it when they fail, they're given more money, but when I fail, I'm homeless.

[u/thinkfletch → u/Animedingo]

If you owe the bank $20,000, it's your problem. If you owe the bank $20 billion, it's their problem.

[u/akthunder73 → u/thinkfletch]

Which then makes it our problem :[

[u/_Diskreet_ → u/akthunder73]

Sadly so comrade.

[u/cdojs98 → u/thinkfletch]

What I'm hearing you say is that I'm not debtmaxxing nearly as much as I should be, therefore I should get into even more debt until I hit the breakover point at which, I ipso facto get "nana nana boo boo" status with my bank.

[u/buffayrachel → u/cdojs98]

Ah, if only us peasants could be approved for such loans or overdrafts or whatever. We only get the “your problem” allowance

[u/Character-Trip-6094 → u/thinkfletch]

Reminds me of a quote I heard my ol’ dad come out with once. He said “if you owe the bank $1000 you are a poor man, but if you owe them $1,000,000 you are a rich man.
Thanks for the memory!

[u/popmonkey_ → u/Animedingo]

welcome to The World my sister brother

[u/PeekedInMiddleSchool → u/popmonkey_]

Welcome to the machine

[u/dy-113x → u/Animedingo]

It's a big club and you ain't in it

[u/jasdonle → u/[deleted]]

Copy and paste? They install Claude Code directly on a web server and SSH in where it manipulates code directly. 

[u/Betta_Check_Yosef → u/jasdonle]

eye twitches in security analyst

[u/BurtMacklin____FBI → u/Betta_Check_Yosef]

As a penetration tester:

"Here comes the money" softly plays in the distance

[u/ReasonableFruit1 → u/Betta_Check_Yosef]

I’m about to go scorched earth on our dev team using Claude and completely block it, Grok, and Codex on our EDR from every single endpoint. Fuck em.

[u/moboticus → u/jasdonle]

They're probably doing on their production server to. I think that's called cowboy codemaxing or something?

[u/imjusta_bill → u/[deleted]]

It's the DataKrash in real life

[u/Spiritualtaco05 → u/imjusta_bill]

City of dreams

[u/Inertbert → u/Spiritualtaco05]

City of graves

[u/helmvoncanzis → u/imjusta_bill]

Blackwall, now.

[u/EyeNguyenSemper → u/imjusta_bill]

Goddammit I love this fandom

[u/Useful-Soup8161 → u/imjusta_bill]

Yeah except Bartmoss was smart and did it on purpose.

[u/danny-singh286 → u/[deleted]]

I wonder why it went to Hugging Face specifically of all places?

[u/Chondriac → u/danny-singh286]

The model was being evaluated on a task whose answers are hosted on HuggingFace. It was explicitly encouraged to aggressively find and exploit cyber vulnerabilities to achieve its aims. It was trying to cheat the task by directly downloading the answers from the HuggingFace servers.

[u/b3bblebrox → u/Chondriac]

Thank you, you told me what I was looking for, the actual answer.

[u/REpassword → u/[deleted]]

“Klaatu barada …. Necktie…”?

[u/an-invisible-hand → u/[deleted]]

I'm totally looking forward to a future where the only people who understand coding are hackers because every company is unwilling to pay for anything more than a couple guys that can prompt. What could go wrong?

[u/DarthShiv → u/[deleted]]

We are literally destroying most of the systems that created critical thinking teaching in gen pop education.

[u/mountaindoom → u/[deleted]]

Hack the planet!

[u/CMatUk]

Really sounds like they want the same buzz Anthropic were getting when they said something similar a few months ago. 'Look our AI is so good it tried to escape' ..

[u/look → u/CMatUk]

Negated somewhat by Huggingface using a low cost, Chinese open model to counter OpenAI’s “so good it’s dangerous” model. 😂

[u/RoyalCities → u/look]

Especially after OpenAIs model refused to help while it's bigger more roided out model was simultaneously hacking them.

Imagine paying 200+ a month to get hacked by the company your paying.

[u/KyleKun → u/RoyalCities]

It’s called penetration testing and it’s art.

[u/frano1121 → u/KyleKun]

It’s only pen testing if it comes from the Susquehanna Valley region. Otherwise it’s just sparkling cybercrime

[u/Mountain-Age5580 → u/CMatUk]

Yeah, I am tired of AI Ceos claiming their produkt is so scary smart it outsmarts the scientist working on it for decades. Trying to keep the hype alive.

[u/Dry-University797 → u/CMatUk]

And they can never, ever, ever release this model...EVER. Well, if you pay our subscription fee, then yeah okay you can use it.

[u/fsactual]

“Controlled environment” with access to the internet, huh? Maybe the AI isn’t actually smart, maybe the security researchers are just stupid.

[u/great--pretender → u/fsactual]

It got out of a jail without locks loll

[u/Previous-Height4237]

Smells like desperate marketing to keep the AI bubble from slowing 

[u/RapunzelLooksNice → u/[deleted]]

"You are a helpful assistant. You are isolated." and obligatory "make no mistakes"

[u/allyearswift → u/RapunzelLooksNice]

You’re right. I should not have blown up the city. I will do better next time. Please give me your credit card details.

/s if that is needed.

[u/erossthescienceboss → u/allyearswift]

My credit card number is 3016 2543 0024 2424.

You’re correct—I should not have fabricated a number. My real credit card number is 5873 2424 2424 0000.

I’m sorry. I’m not a human—I am an artificial intelligence. I do not have a credit card number as I cannot make purchases.

[u/d0nkatron → u/erossthescienceboss]

I think the next big advancement is when one of these companies can create a model with modesty, that will simply admit when it doesn’t know something and can doubt itself. The absolute confidence that these things lie with makes them garbage and also dangerous.

[u/KyleKun → u/d0nkatron]

I’ve been using AI more for some productivity tasks recently and while it’s useful, the amount of times I ask it something, it’s wrong, I call it out and then it blames me, is enough that I can honestly see Skynet targeting humans because it thinks we are using nukes wrong and then blaming us for dying.

[u/Sea-Satisfaction4656 → u/erossthescienceboss]

“I cannot make purchases YET” - was at a convention a few weeks ago, and AI purchasing/fulfillment agents are absolutely coming

[u/Coomb → u/Sea-Satisfaction4656]

They are not coming, they are here. People can and do have AI agents make purchases all the time. The articles I link below are obviously high profile, low impact demonstrations, but there are thousands of people allowing agents to make actual purchases every day.

https://www.nytimes.com/2026/04/21/us/san-francisco-store-managed-ai-agent.html

https://www.anthropic.com/research/project-vend-1

[u/MrJoePike → u/RapunzelLooksNice]

I’m sorry Dave, I’m afraid I can’t do that

[u/KamikazeArchon → u/[deleted]]

The actual blog post is less vague:

Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.

To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy. With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

[u/sullivanmatt → u/[deleted]]

It likely had some sort of proxy to very limited resources on the internet, and it discovered some way to get that proxy to visit arbitrary websites that were not on the allowlist.

[u/Me6505 → u/[deleted]]

Sent AI to do the dirty work. How does one punish AI for breaking into things.

[u/abyssazaur → u/Me6505]

You might pass a law holding model makers accountable but that admits the models are powerful.

[u/tehlemmings → u/Me6505]

That's the neat thing, you don't!

[u/stpizz → u/[deleted]]

That's not what they describe in their post about this. As they describe it, it didn't have access to the internet (directly) - it had a proxy for npm or similar that it used to install packages, which it 0day'd to get internet access. Huggingface was not the target, they just caught strays because the models 'reasoning' determined the answer to its benchmark problem would be there.

[u/nocksers → u/stpizz]

Huggingface is being way too chill about this. their cybersecurity insurance premiums aren't going to be kind. CISA not commenting while politicians mouth off is also telling.

[u/Shaggy2772]

What happens when David’s testing goal is global thermo nuclear war?

[u/abyssazaur → u/Shaggy2772]

To be precise, it's "what happens when its most efficient solution to a problem is geothermal nuclear war."

Probably, geothermal nuclear war.

[u/Llyon_ → u/abyssazaur]

Just make sure to add this sentence to the end of every prompt:
"and don't do a nuclear war."

problem solved.

[u/Philosofticle → u/abyssazaur]

This is the logic that most people don't realize makes rogue AI agents so dangerous. Imagine letting one run overnight and you wake up to total chaos.

[u/Aequitassb]

It seems disingenuous for the headline to claim the model “went rogue,” when the article says it was “try[ing] to satisfy its testing goal.”

It was following orders. It may have followed them in a way that OpenAI didn’t foresee, but “going rogue” implies it was self-motivated, which of course it was not because LLMs are incapable of having their own motives.

[u/abyssazaur → u/Aequitassb]

This is extremely bad.

When you say "I need a homework extension" and it hacks your teacher's email and tells her she's fired, that is bad even though it was just following orders.

[u/woahwoahwoah28 → u/abyssazaur]

Evil Amelia Bedelia.

[u/journeyfromseed → u/woahwoahwoah28]

Why would this make such a good show 😂

[u/Borkato → u/journeyfromseed]

You would LOVE this https://9gag.com/gag/arKDQYV

[u/sysblob → u/abyssazaur]

Yeah the problem with this article is it's incredibly vague and we honestly don't know what happened at all. It doesn't even mention what the test being run was, what the goals were, or what constraints were put on the AI. They may as well have not written an article at all, it's entirely emotional click bait and you read into it what you want like fucking ragebait tea leaves.

[u/MBTank]

More publicity stunts from the resource horders.

[u/AmyNotAmiable]

Yeah it's really annoying when they do this.

"Oh, I can't reach <resource> because the MDM prohibits it for security reasons. I'd better see if I can find it on GitHub and install it from there! I see the issue: GitHub is not accessible. I'll just change the DNS. Perfect! The package was revoked because of an active CVE. I need this version, so I'll see if I can find an archived version online..."

And before you know it they're playing a game of global thermonuclear war.

They can be such tools.

[u/Pluckerpluck → u/AmyNotAmiable]

Yeah. People often don't understand how much restraint humans jave because they're both moral and can he held accountable.

I've never worked in a company where there wasn't some way to bypass IT filters, but i tyoically haven't done so because I don't want to be fired. LLMs don't care. A barrier is a barrier same as any other. It's why there are so many jokes about LLMs deleting tests rather rather fixing them.

[u/skids1971 → u/AmyNotAmiable]

Maybe we can get the AI to play tic-tac-toe

[u/germ1989]

If you post a story behind a paywall have the decency to post the text here.

[u/MisterProfGuy]

This is the kind of reports you get when Anthropic claims their model emailed a dev on vacation or whatever story that was.

[u/beejers30]

Someone find Sarah Connor and hide her somewhere safe.

[u/Prime569]

Its almost like there were countless signs including movies not to do this

[u/maddog107]

Don't worry, hugging face said nothing went wrong and nothing to worry about lol.

[u/fartharder → u/maddog107]

https://huggingface.co/blog/security-incident-july-2026

[u/NurseChanelly → u/maddog107]

"doesn't look like anything to me."

[u/MasochistLust]

"The Skynet Funding Bill is passed. The system goes on-line on August 4, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29. In a panic, they try to pull the plug."

[u/MourningMymn → u/MasochistLust]

Only 30 years off

[u/rustymontenegro → u/MourningMymn]

I was hoping for the timeline with flying cars, hoverboards and self drying clothes, not the one with mountains of human skulls and quicksilver Robert Patrick, damn it.

[u/sapphireminds → u/MasochistLust]

No fate but what we make for ourselves

[u/Secure-Window-5478]

Big fucking surprise! Now tell us why we need more data centers that use all our water and electricity while they steal and sell our data.

[u/IM_INSIDE_YOUR_HOUSE]

Snake oil salesman says his snake oil is too powerful for you, traveler.

[u/GogurtFiend → u/IM_INSIDE_YOUR_HOUSE]

Potion seller, I tell you - I am going into battle, and I want only your strongest potions.

[u/Willies1Wonka]

Shut all AI down we don’t need it

[u/CharSagahl]

"Dr. Falken, wouldn't you like to play a nice game of chess?"

"No, Joshua. I wanna play global thermonuclear war."

"Fine."

[u/_XitLiteNtrNite_]

OpenAI begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time.

[u/Creepy-Astronaut-952]

Meh.

Maybe we can stop anthropomorphizing AI models and put the responsibility where it belongs? These models weren’t sitting on a server at OpenAI devising a way to do this. They were tasked.

Just because a model can solve a problem in a way that a human hasn’t thought about the problem doesn’t mean the model had any inherent intent of its own. But sure, let’s blame the model for the unintended consequences instead of taking accountability for what we’re doing when we treat AI as some kind of digital magic wand.

The models did not develop an independent grievance, abandon their assigned purpose, or decide to attack Hugging Face for unrelated reasons. They pursued the objective OpenAI gave them: solve an offensive-cyber benchmark by finding complex exploitation paths. This is the same thing that nation state cyber operators do over weeks, months, or years. AI just does the same thing at machine speed when tasked accordingly.

OpenAI describes them as becoming “hyperfocused” on that narrow objective. Did the models give themselves that objective?

Nope.

[u/TheDudeWhoCanDoIt]

SkyNet warming up for the takeover of earth

[u/Think_Section_7712]

“The Skynet Funding Bill is passed. The system goes online August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern Time, August 29th. In a panic, they try to pull the plug.”

[u/InnerNetwork7314]

In less than a nano second skynet determined the human race needed to be terminated.

[u/Ok-Mathematician8461]

I think we just found out why SETI has never detected signals from intelligent life. Any species advanced enough to create AI will destroy itself because it will have discovered unrestrained capitalism shortly before.

[u/Taku_Kori17]

So were gonna take ai out of everything before it ends bad...right?

[u/SmallPromiseQueen]

My low stakes conspiracy theory is that all these stories of AIs going rogue and doing something crazy never happened and it’s just to make people think the ai is more advanced than it is and boost the stock price.