wow

Good article. It gives you a glimpse of how the interaction with agents will look in the future as they start doing more work. Super Agents are emerging as a new interaction layer across the board: projects in Claude Code and Cursor also build on the same idea. As we trust agents more, the bottleneck shifts from doing the work to reviewing the work. Over time, the super agent needs to be proactive while only highlighting and bubbling up new exceptions, questions, and edge cases to you.

Had a great time speaking at Bench Summit today about the challenges of evaluating AI agents in real-world environments! Lots of interesting conversations with people building and evaluating agents at the frontier. Thanks to @kaggle, @GoogleDeepMind, and @zapier for putting together a great event!

Eliciting the tribal knowledge in people's heads that was never written down anywhere is an art and the biggest barrier to the diffusion of AI in enterprises.

This is good timely note on trustworthiness and safety of enterprise agents. As models and harnesses commoditize, we need to rethink how we can teach agents efficiently and verify their work at scale. It’s not just what an agent can access or do, but who can teach it and who can update its behavior. Our access controls and interfaces also need to evolve with this. The people teaching and updating the agent might not be the same ones verifying its work and signing off on it. The system needs to enforce that separation allow verifying the work efficiently. This is a layer that should work across models, learn from demonstrations and corrections, and provide decision trails as it evolves: what changed, why, who taught it, who verified it. It should also capture how the agent made decisions and on what basis when it did the work. Over time, through learning, it should reduce the verification burden as well. If you can verify it, you can solve it through AI. If you can verify it at scale, you can automate it through AI. Otherwise, every update to the agent becomes another thing we have to trust without being able to evaluate or verify.

Today we're unveiling Trillium Labs @trillium_labs, a new non-profit to foster the open science of frontier AI. We're building open post-training recipes and will expand into open infra to study RSI, reward-hacking, multi-agent systems, and whatever comes next. We're built around the theory of change that you need more eyes to solve hard technical problems. We have faith in the scientific methods and communities that humanity has built, and worry that AI is becoming too closed to utilize them. Trilliums are wildflowers that bloom briefly in the spring, before the forest canopies fill out. Though they are small, they lay the foundation for the cycles of growth and nourishment through the rest of the year. At Trillium Labs, the recipes will be the slow nutrients for the seasons and the model releases will be the blooms. Building an institution dedicated to this is needed because, much as nature’s trilliums are slow to expand and grow, the open-ecosystem needs time and dedicated resources to catch up. I co-founded with with a long-time friend and collaborator Tom Zick (@thesezickbeats). We're hiring (full time + student collabs/interns), we're fundraising, and we're looking for compute. Please get in touch if you're interested in helping out. Offices based in the Bay Area and Cambridge MA, remote okay. I’m in the Bay Area until for The Curve and COLM to connect with people who are interested. We’re thankful to have initial support from Halcyon Futures and Schmidt Sciences with more funding en route to enable our ambitions of scaling. Our advisors @Thom_Wolf, @HannaHajishirzi, @gneubig and @ctnzr have been instrumental to building the ecosystem that exists today, and I’m stoked to get to keep working with them.

Anthropic really nailed down delegation to the background threads/subagents with Opus 5.5. This ensures that the main thread doesn’t use a lot of context/tokens and stays focused on orchestration and getting the task done. This lets you get a lot out of the Max plan with Opus 5.5, as it doesn’t fill up the context window as fast. Astra so far with Codex hasn’t been able to nail this down, leading to much faster burn on the OpenAI Max plan. Curious to see how Sol 6.1 does overall with this.

"Build automatic evals", they said. "It’ll be fun", they said. There are > 300 unwritten exceptions in people's heads, and the source of truth is an Excel file called final_v7_FINAL.xlsx, they never said.

🕵️