I’m stuck in this loop:
“Use LLMs to find all the bugs”
Okay, which model?
“Model X found a bunch of bugs”
“Wait, another lab’s model found bugs that one missed.”
“Actually, a startup with a better harness found more using the same model.”
“Now another startup with a local model, a different harness, and talented researchers found bugs everyone else missed.”
“Hold on, the next frontier model just found another batch.”
All in the same attack surface that’s already been reviewed by everyone above.