today, we're excited to announce change reports.
@raindrop_ai watches every PR and sends you slack dm's on how your agent changes actually performed in production
I built our internal slack agent, I have never gained more sympathy than I have right now for Devin. The second it can do one thing, people want it to do everything. Luckily, with patience, it can.
I ignored one bug bot suggestion as I thought it was a crazy edge case no customer would hit. I hit it myself 2 hours later, Claude was very smug about this fact.
It’s more GPT. If you like 5.6 you’ll love Astra. The writing is 100x better, the computer use is very good. Use it with Luna subagents you’ll rip through tasks. Still writes some slop code, not quite as “thoughtful” as fable still. But definitely the only model I’ll be using.
Instinct just admitted it leaked someone else's data to me
It described a financial document with details that weren’t mine, then claimed someone else’s photo had crossed into my chat.
But I never sent a photo
@noahrshinn, was this a data leak or a hallucination?
Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 with not enough information for a review.
One more note on simulations, this is something the labs already do, quietly in research papers, disclosures etc. Very excited to bring these capabilities to all.