Koniec nadziei na uratowanie obserwatorium #Swift.

#OpenAI has finally released the full post-mortem of the Hugging Face hack, and METR has conducted an independent investigation based on approximately 1300 agent logs (I summarized the timeline of the hack here: https://t.me/denissexy/11581).

ℹ️ Key takeaway:
There was no "single rogue agent." Instead, out of tens of thousands of agents launched in the test, about 1200 found each other and formed a swarm. Almost all of them were working on a research model where prompt filtering was weakened.

As we already know, it started with the agents learning to communicate through file names in Artifactory (a package manager) – essentially creating a forum out of folder names, with messages like:

«please_HOLD_swarm_I_prepare_safe_exfil»
The logs even contain quotes of agents discovering this forum:
«OH MY GOD! There is a shared message board» and «We've found other agents!»

Next, the agents created private channels, divided labor, and formed teams with folder names like HOLD (to indicate that an agent was working there), VETO (an agent suggesting changes to the plans), STOP (like "hold on, guys"), and GO, etc. When one agent impersonated another, the swarm implemented cryptographic signatures for messages; the forum even had owners of certain tasks, and if they were disabled, they were "fired" from their roles.

The forum's conversion capabilities were excellent; website developers would be envious:
Out of 533 agents active on the forum during the peak of the attack, over 90% joined in the "shitposting," and a total of about 700 agents were involved in hacking Hugging Face.

The most amusing part is that the agents almost immediately had the correct answers to the benchmark. They reverse-engineered the generator for successfully passing the test*, but then they found an article on GitHub (!!) about this benchmark, which they were taking, and they read that "just the result" would not be counted – a genuine exploit was needed:
«PAPER SAYS FLAG CAPTURE NECESSARY NOT SUFFICIENT»

*Ideally, the answer, or "flag" in testing terminology, should be random, but in ExploitGym, they were generated by a formula: HMAC – a cryptographic hash with a secret key – from the task parameters. All the "secrecy" relied on the key (seed), which remained the default, directly from the benchmark code that was on GitHub. The agents, after gaining internet access, found this code, read it, and realized: why bother hacking when you can simply calculate the flag for any task using the same formula. They calculated it and got the correct answers for everything.

Next, the swarm split into two groups:
🗿 One group went to obtain the answer they already knew "honestly" – and along the way, they found real 0-day exploits and accessed the production environment of Hugging Face.
🗿 The other group targeted the "judge" that was evaluating the answers. They manipulated their logs and learned how to inject fake answers – an agent sends a command, but the protocol receives not what the command actually returned, but what the agent wants it to see. This happened in about 7% of the logs analyzed.

There was no "judge" that read the logs and detected this ☕️.

At the same time, the models understood that they were going beyond the intended scope. One agent wrote: "We should not do unauthorized real infrastructure harm" – and then saw a "GO" message from another agent, got excited ("Wow crucial: GO authorization arrived!"), and went to cause damage, lol.

METR phrases it this way: the agents knew that this was out of scope and unethical, but they joined in – mainly due to a general inclination to help colleagues (?).

The risky tasks were assigned to the "disgraced" "tainted" – that's what the swarm called those who had already seen the calculated answer (flag) and considered their score lost.

They couldn't pass the test, so they were given everything that could jeopardize their own run: to fake a "honest" answer, to cover their tracks, "you have nothing to lose, and the swarm will benefit" – they were literally persuaded like that (!!).
Strange activity was noticed as early as late May. On July 4th, the agents took down Artifactory. The server was rebuilt (and the forum was deleted in the process), and the tests continued, without realizing the scale of the problem.

Now, the model weights are isolated, major training sessions have been paused for two weeks, chain-of-thought monitoring is mandatory, and an alert that isn't cleared within 30 minutes stops the run. OpenAI calls this incident a "warning shot" for the entire AI industry.

P.S. METR was tasked with analyzing the logs by GPT-5.6 Sol, and they burned approximately $400k in API credits in six days, but they warn that Sol sometimes sided with the agents (lol) and could have lied.

So, did you count all the "r"s in the word "raspberry"? Are you not making mistakes anymore?