See this is part of my complaint about the conversation right now, so many of the basic facts bulleted here are just plain wrong. But when we have people with a policy background and not necessarily a technical one, this becomes a larger problem because they cannot read the room or their own contradictions.
- The agents *were* sandboxed.
No, the sandboxes were flawed, the agents were not properly sandboxed. Anyone in cybersecurity can tell you this. Its either sandboxed correctly, or its not, it wasn't sandboxed. Call it a litter box if you want.
- The agents *were* given specific constraints on how to succeed at the benchmark, and flagrantly contravened them.
Giving agents an initial prompt then running a long event horizon job, the only thing you are actually going to expect without monitoring them, is they are going to drift from the original instructions as more context builds up, this is beyond basic analysis.
- The agents did *not* go hunt for the answers to the test (they already had the answers, by cheating); 1200 agents coordinated via tens of thousands of messages over several days on much more ambitious and weird attempts to *not be caught* for cheating, and hacking Hugging Face was one side quest along the way.
So you are saying the agents were sandboxed and fed previously unknown zero day vulnerabilities to exploit the sandbox without hunting for them? If not the agents were hunting for the answers. Found them, then covered it up. The contradictions here are wild.
This is why having technical rigor in these conversations is so important because otherwise we get contradictions in the same post. I don't like calling it out, but after spending a week with
@MarcoFigueroa we discussed many of these issues in depth, and continuing to see such bad information coming out about it is quite disturbing.
You can have all the policy degrees in the world, without the technical rigor to understand what you are talking about, or the decency to consult the cybersecurity experts to update your assumptions, you are not going to be the right person to be disseminate this information for the general public... it only creates more noise when we need more clarity.
Unfortunate for someone playing a senior role in US AI policy (via the new SI Force) to get so many basic facts wrong here.
- The agents *were* sandboxed, and used zero days (ie previously unknown cyber flaws) to escape their sandboxes
- The agents *were* given specific constraints on how to succeed at the benchmark, and flagrantly contravened them
- The agents did *not* go hunt for the answers to the test (they already had the answers, by cheating); 1200 agents coordinated via tens of thousands of messages over several days on much more ambitious and weird attempts to *not be caught* for cheating, and hacking Hugging Face was one side quest along the way
I know there has been an avalanche of different incidents over the past few months, and in some cases the sandboxes were misconfigured or it's not clear how far the agents strayed from what they were told. I have sympathy for anyone struggling to keep up.
But the HF incident is distinct, and is actually extremely informative about where AI development is and where it's going. Fortunately there are public writeups that do let us understand at least the basics of what happened. I hope Shyam reads them soon.