Building BB Arena: which AI models can really hack? blackboxarena.ai Full-time bug bounty hunter on @Hacker0x01 @Bugcrowd 600+ reports, 150 critical
Australia
- Tweets 3,705
- Following 355
- Followers 4,306
- Likes 5,042
Pinned Tweet
Righto fam, apparently my DMs have turned into the unofficial AI bug bounty support desk...
So here's the rundown for anyone new here.
I'm an Aussie bloke hunting bugs full-time. 600+ reports, 150 criticals, ADHD-fuelled rabbit holes, an absurd amount of AI tokens burned and the occasional meme when my agents do something stupid
I share pretty much everything I can publicly. Workflows, setups, prompts, model testing, costs, failures, wins and whatever random shit I've been experimenting with that week.
🧠 Which model is best for hunting?
Personally, DeepSeek is my go-to. Mainly because I don't have to fight refusals every five minutes while doing legitimate security research.
But don't just take my word for it.
blackboxarena.ai
I literally built a platform to answer this question
Real black-box security labs, benchmarks, model rankings and results. Go have a look and decide what works for you.
🤖 Want to build your own AI bug bounty hunter?
github.com/mdpsec/bug-bounty…
My open-source prompts and workflow are there. Free.
It's not a one-click hackbot. You'll need to read, understand and adapt it to your own setup.
Claude and Codex are pretty good at figuring stuff out. Give them a crack.
⏳ When's the next model getting tested?
Usually within 1-2 weeks of release.
Unless it costs a kidney to benchmark, then I might skip it.
Want a specific model tested? Sponsorship is an option. My wallet would appreciate the support.
🔬 What's BB Arena actually about?
Beyond public rankings, I'm building realistic security labs and benchmarks that AI companies can license to evaluate their models, including unreleased ones.
blackboxarena.ai/
🛠️ What harness do you use? How do you automate everything? Subagents? Skills? Tools? Prompts?
FAM. READ MY POSTS
I've spent months sharing this stuff.
Search my profile. Scroll back. Read the experiments. Follow the rabbit holes.
There's a ridiculous amount of information buried between the memes and shitposts.
Yes, my ADHD means I jump between 47 different things. Somehow most of it eventually ends up on X.
💬 Can you still DM me?
Of course you can, mate.
I don't mind questions at all. Beginner or experienced, doesn't matter.
But PLEASE put in some effort first.
Read the repo. Search my past posts. Check the benchmarks. Actually try building something.
If you've done that and you're stuck, or you've found something interesting, I'm always keen for a yarn.
What I can't do is personally build your hackbot, debug your setup, fix your Claude subscription, chase your BB triager or get your CVP approved.
I can't even get CVP red myself, good luck with that one 💀
I'm not running a support desk, mentoring service or AI agent installation business.
I'm just a bloke hunting bugs, building BB Arena and seeing how many ridiculous rabbit holes I can go down before the next one appears.
I put a stupid amount of time and money into this research and share heaps of it for free because I genuinely enjoy seeing what other people build.
All I'm asking is that you meet me halfway.
You don't need to know everything. Just have a crack before asking someone else to do it for you.
Put in the effort, and I'll put in mine 🤝
Now back to spending 1b tokens investigating something that turns out to be intended functionality.
17
5
120
68,338
Marius du Preez retweeted
feels legit like this a endless cycle between openai and claude for cyber
just wanna get another account for some redundancy
its way worse for none US people sadly
9
15
547
55,759
So looks like if I wanna get red access I gotta get a CVE to my name aparently..
Guess I gotta step up my game
11
28
1,996
final update on the STEVE/Jimmy situation
team fired
went hunting myself... first bug in 10 minutes
Duplicate of STEVE
I give up
7
762
spoke to a triager who used to hunt
regrets switching and is probably going back
hunters get slow triage, dupes and downgrades
triagers get endless slop and bad reports
both blame each other
surely system can be fixed somehow...
2
25
1,490
Kinda expecting next drop to be something crazy on cloudflare...
That would end the week with ultimate bingo
1
5
699
Everyone's worried about AI hacking their business.
I'd be more worried about who has the keys to it.
Spent years working in IT. The amount of legacy equipment still running businesses is insane.
CNC machines, dental systems, medical equipment. Some locked to operating systems that should've retired decades ago.
Can't upgrade. Too expensive. Breaks compatibility.
So we bolt on remote access and call it a day.
Then there's the MSP managing everything.
Admin access to 100 businesses. Remote tools. Backups. Credentials.
And here's the irony.
Plumbers have broken toilets.
Cleaners have dirty houses.
IT companies aren't automatically immune to neglecting their own security.
Now throw increasingly capable AI vulnerability-hunting agents into the mix.
Old systems, modern connectivity, shared privileged access.
That's one hell of an attack surface.
The business owner probably doesn't even know half of it exists.
2
8
728
The STEVE vs Jimmy dispute got out of hand
Had to hire an independent AI to investigate
His name's Dave
$3,800 in API credits later, Dave has concluded both reports are duplicates...
Of a report Dave submitted yesterday
I need a holiday
6
18
1,231
New on BB Arena: MiniMax M3.1 Flash (Max). @MiniMax_AI
#9 of 28 with 55/100 black-box labs solved.
Level with Muse Spark 1.3 on solves, just behind on rungs proven, ahead of Mistral Large 4.0.
$0.32 per solve.
Enjoy fam
2
1
12
899
if your a lab and looking for data/training
reach out dm's always open
blackboxarena.ai/labs/
10
855
just a reminder
@Tailscale is amazing if u aint using this
ur ngmi
1
2
29
1,724
Offer still there
for all them inference guys.
id love to give eyes over ur platform as a hacker would...
offer be here.
legit pay for it or take it for free just atleast do something to keep users safe mdpsec.com/security-test/
1
1,010
Marius du Preez retweeted
For everyone in my DMs 👇
- Read my past posts first. Everything you need to automate your hunting is there.
- Still stuck? Ask me something specific.
- MVP model: deepseek
- Starter workflow: github.com/mdpsec/bug-bounty…
Put in the effort, and I'll put in mine 🤝
5
36
317
11,656
Jimmy finally found a valid vulnerability
Submitted it
Marked duplicate
Original researcher: STEVE
I am not dealing with this workplace drama
6
53
2,709
U know deepseek be MVP
More noticing it
blackboxarena.ai/models/deep…
We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program.
We have been building a cybersecurity benchmark based on simulated corporate networks. This is a valuable indicator for the cyber-risk a model poses if in the hands of a bad actor. We plan to make the full evaluation public in the coming weeks but wanted to share our first immediate finding here.
3
10
1,249
For those who havnt seen this
pretty amazing to crank this up and just walk around shops to see how much traffic out there...
Fieldwatch 1.1.27 is out, and it's the biggest update yet.
Fieldwatch now speaks 27 languages. Pick one in Settings, follow your phone's language or stay in English. Want a language that isn't included? Export the English template, translate it and import it back. Catalog notes stay in English for now, but menus, settings and screens now make sense to people who don't speak English natively.
Places watch. Fieldwatch now flags a stable tracker that this phone hears at two or more places in the last 7 days, plus one that comes back after a quiet stretch. You'll find it on that radio's page and in the sit report, which shows where you were and, if you turn on place names, the street. Stops from earlier days include the date, and each radio keeps its last 8 stops. A filter can narrow the current Live list to the ones you can hear right now.
Better TAK. Each install now gets its own marker, either Fieldwatch plus a short tag or a name you choose, so two phones on the same network no longer pile onto one pin.
Sit export levels up. Along with GPX and KML, you can now export GeoJSON or a full ATAK data package. That package includes your path, a point for each radio you heard, drone pins with short tracks, operator pins and GPS interference hits along your route.
Raw GNSS recording. You can now record your phone's raw GPS measurements around an interference hit and share the latest recording. It's off until you turn it on.
A simpler app overall. Settings now opens as a list that shows each current choice. Path, Debrief, Sit export, Compare, Log export and Signature candidates each get their own page in Reports. Filters are simpler, and Live's view menu now has icons for each view.
Free, open source and offline, with no account and no backend. github.com/OffGridPete/Field…
5
984
Just found this awesome Open Source repository with a list of 10 AI prompts you can integrate into your bug bounty hunting workflow
Web2 Security focused, still very valuable - scoping, triaging, validation & more. 10x for the great resource @mdp_sec 🫡
github.com/mdpsec/bug-bounty…
6
39
363
14,073