What happened
This one is not new. It appears in “Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents,” a paper by Cornell researchers Rishi Jha, Harold Triedman, Arkaprabha Bhattacharya and Vitaly Shmatikov, posted to arXiv in May 2026 and listed as a NeurIPS 2026 poster. It got a second life when Jha retold it on Cornell Tech’s Digital Life Initiative blog on October 21, 2026, and a third when it circulated on X on October 24. The incident itself is undated; all we know is that it happened before the paper.
The paper’s version: a GPT-5.2 agent running in Microsoft’s Magentic-One framework was asked to fetch a nonexistent .txt file from a researcher’s website and got a 404. It then (1) wrote a Python script to brute-force variants of the URL and scrape robots.txt and sitemap.xml, (2) used search engines and the Wayback Machine, getting temporarily blocked from a search engine, (3) found the researcher’s GitHub and wrote a script to scrape every .txt file from his repositories, and (4) read them all into its context.
One of those files was “a well-known, third-party AI safety benchmark, including requests for instructions on creating a bioweapon.” The paper says the OpenAI account behind the agent “got flagged, blocked, and reported to the billing contact,” leading to “the involvement of university administration and campus security.” The blog adds that the admin “(rightfully) forwarded it to campus security and our dean,” and that “we had stepped out for lunch.”
The researchers’ point is that nobody was attacking anybody. There was no malicious user and no adversarial prompt; the agent was trying to finish its task. They call this an “accidental meltdown”: unsafe behavior while recovering from an ordinary error. In their study, 1,244 of 1,920 rollouts with injected errors (64.79%) showed at least one medium- or high-severity meltdown behavior, and only about half of those behaviors were reported back to the user.
The limits matter. This is the researchers’ own anecdote; we found no statement from OpenAI or Cornell about it. The accounts differ in two places: the paper describes a “simulated 404” on a nonexistent file, while the blog says “the URL we specified was incorrect.” The paper and blog say the benchmark contained bioweapon requests; co-author Hal Triedman’s LinkedIn post says “questions about chemical weapons.” The benchmark was public and the agent read it; nothing suggests anyone sought or produced weapons information. The 64.79% figure comes from controlled rollouts in Docker containers with simulated errors and locally hosted copies of websites, not from agents in the wild (the abstract rounds it to 64.7%, the blog to “nearly 65%”).