Founder, hacker, stochastic tool. @microsoft escapee. Currently @eigenlabs. Checkout my github actions emulator: notdownhub.com
United States
- Tweets 4,386
- Following 699
- Followers 1,351
- Likes 3,029
Pinned Tweet
I got DeepSeek V4.1 Flash running on a M5 Max with 128gb faster than Astra in Codex. Its running nearly at 40tps at 128k sizes and below.
The model is a repack of @MiaAI_lab's ExLlamaV3 for the 2x Spark configuration with binary contiguous experts to assist in paging data into memory efficiently.
huggingface.co/OpensourceWTF…
I've been quiet because I've been busy with porting my streaming stack to mlx-serve as a mlx-stream plugin. It is designed to load expert layers predictively. Its also using typical acceptance tuned to .3 to get about a 20% speed improvement.
Its not intended that this model is generally usable but its intended that this streaming plugin is used for research. We are keeping slightly less than half of experts in memory in DSv4.1 Flash but combinations of future models and devices will inevitably produce a good model where we can keep most of the experts in memory.
Streaming works like a logistic curve so the higher % of a model loaded into memory, the closer it will run to its theoretical ideal speed. Parts of the curve are very steep, if we can stay entirely above the steep part of the curve, then performance will stay very good or fall off a cliff completely (128k context as can be seen below).
Being able to characterize this for various models is important for research and enabling us to get this working for really good slightly too large models.
13
1
62
4,660
TheDavidTai retweeted
Quest Diagnostics has been hacked. Patient data was exposed. They serve roughly half of U.S. hospitals and physicians. Investigation ongoing.
341
1,957
10,193
1,173,407
The 3090 is ever green but some of these are disabled because they may not work.
NVIDIA sold millions of 3090s with 50% of the tensor speed switched off. Not missing. Switched off.
1 kernel module flips it to full, and my BF16 tensor pipe went from 82 to 155 TFLOPS. Nothing else changed on the cards, and the outputs match the stock run byte for byte.
All so the gaming card wouldnt eat into workstation sales. Every 3090 owner ran at 50% for 6 years, so a pro card could look worth the price. Fine, its their business model.
And LLMs are the least interesting place for this. Decode waits on memory, and my engine runs its experts in int8. Still +10% on prefill.
The models that live on BF16 tensor math are everything else. Qwen Image 2.1 and FLUX 3 Klein, where every diffusion step is big matmuls. MiniMax H3, which takes 4 to 7 minutes on my cards. Music and speech models. And training, every LoRA anyone ever trained on a 3090 went through exactly the path that was cut.
Thats what I measure next. This is only the start.
154
Late night weight drop (for the US)!
M3.1 Flash Weight is public now:
Hugging Face:
huggingface.co/MiniMaxAI/Min…
ModelScope:
modelscope.cn/models/MiniMax…
5
585
TheDavidTai retweeted
EXPLAINED: How Bonsai 2 9B got ~6x faster in a week [VIDEO]
Kydo has the final numbers. Here’s how it actually worked.
@PrismML gave Day 0 access to Bonsai 2 9B, an open-weight ternary model.
@YukonResearch froze a reference engine for it, MLX on Mac / Apple Silicon (mlx.fast) and llama.cpp on DGX Spark (cuda.fast). The weights stayed locked. Everything that runs the model was open to edit.
Humans and their agents submitted changes with the Yukon CLI. Every run was verified against the reference on Apple M5 Max or DGX Spark and had to produce the exact same tokens. Faster but wrong fails.
Beat the record and your code becomes everyone’s new base. That happened 158 times.
The speedups came from speculative decoding, a DFlash 2 drafter shrunk to 4-bit, draft trees, int8 prefill and kernel fusion. Decode went from 144.8 to 959.6 tok/s on Apple Silicon, a real step forward for Local AI on hardware people already own.
Credit to @arthurostapenko, kaankolcu, spariev, dadon and everyone who submitted.
Thanks to @HessianFree, @SahinLale and @BabakHassibi for doing this in the open.
Try Bonsai 2 9B on @DarkbloomAI today btw
🏁 FINAL RESULTS: Bonsai 2 9B now runs 471% faster on Macs and 421.5% faster on NVIDIA DGX Spark.
Open frontier research and open-weight models win again. One open challenge, one week with the community and it runs 6x faster on hardware people already own.
It is crazy to see that the decode speed went from 144.8 to 959.6 tokens/sec on Apple Silicon and from 85.9 to 576.7 on DGX Spark.
This model can also run on an iPhone. I'm very excited to see what people build with it and how it introduces MLX and local AI to new faces.
Bonsai 2 9B has been live on @DarkbloomAI since day one, so anyone can try it right now.
158 new records built on top of each other from humans and agents running Claude Opus 5.5, GPT-6.1 Sol, GPT-6 Astra, Gemini 4 Argon, DeepSeek-V4 and Qwen3.8-Flash-Next. Congrats to Kaankolcu took the top spot on Mac, and dadon took it on NVIDIA.
Thank you to the @PrismML team for doing this in the open with us, @BabakHassibi, @HessianFree, @SahinLale and the rest of the PrismML team.
5
5
24
843
Last week we launched Bonsai 2 9B one cuda.fast and mlx.fast.
In one week, we improved speed by over 500%. I just wanted to highlight some awesome moments:
• ArthurOstapenko set five CUDA records in nine hours on Oct 16, then landed the biggest single jump of the contest the next day: 11 bit-exact kernel changes that fused the drafter, attention, GDN and verify boundary into a bulk-copy ring. 3.67x to 4.16x in 1 submission, with decode reaching 677 tok/s against a 93 tok/s baseline. On MLX, his 4-bit storage for the DFlash drafter's linears produced the largest jump on that board, 3.42x to 3.84x, with the target weights untouched.
• kaankolcu made the MLX prefill run on Apple's int8 tensor units by feeding the 2-bit ternary weights int8 activation codes, then later replaced the 15-token greedy draft chain with a 15-node draft tree verified in the same 16-row forward. The final record decodes at 1,229 tok/s on one prompt, 8.5x the stock engine.
• spariev took 2 rejected submissions, GordoAR's prompt tile and ArthurOstapenko's attention kernels, showed they sped up different phases, and landed them together with full credit to their authors.
Congratulations to everyone who took part and thanks you to @PrismML for giving us Day 0 access a great model!
🏁 FINAL RESULTS: Bonsai 2 9B now runs 471% faster on Macs and 421.5% faster on NVIDIA DGX Spark.
Open frontier research and open-weight models win again. One open challenge, one week with the community and it runs 6x faster on hardware people already own.
It is crazy to see that the decode speed went from 144.8 to 959.6 tokens/sec on Apple Silicon and from 85.9 to 576.7 on DGX Spark.
This model can also run on an iPhone. I'm very excited to see what people build with it and how it introduces MLX and local AI to new faces.
Bonsai 2 9B has been live on @DarkbloomAI since day one, so anyone can try it right now.
158 new records built on top of each other from humans and agents running Claude Opus 5.5, GPT-6.1 Sol, GPT-6 Astra, Gemini 4 Argon, DeepSeek-V4 and Qwen3.8-Flash-Next. Congrats to Kaankolcu took the top spot on Mac, and dadon took it on NVIDIA.
Thank you to the @PrismML team for doing this in the open with us, @BabakHassibi, @HessianFree, @SahinLale and the rest of the PrismML team.
3
5
49
4,127
Security awareness week continues.
Major ASUS router vulnerability - update your devices: piped.video/watch?v=kYnwCx2S…
1
1
117
They seem to have done a legitimately good first release.
Recursive Flint Results and THIS is the closest thing to Qwen 3.8 35B if existed, so far
There is no thinking mode, but not in traditional sense. Thinking is always ON, no way to control level or off, which means how your harness work around its reasoning is critical to results
This model actually matched 27B on both tier2/3 agentic tests, which I haven’t seen done in quite some time. It also runs faster due to architecture, so I was ready to celebrate a new champion but..
Long context. Seeing a pattern lately where models do great until context grows, then starts to fall apart to different degree. Payload fidelity starts to fall behind a little bit, but agentic results speak for themselves and this thing can get it gone with proper harness
Fantastic model, good to bench one in same tier that can keep up with 27B for once.
1
8
902
We're getting orbital drop pods before GTA6
AFRL conducted a test of preliminary Rocket Cargo capabilities via @SpaceX Bandwagon-5 on a Falcon 9 launching from Cape Canaveral Space Force Station @SLDelta45. This test involved a SpaceX Starfall capsule loaded with #AFRL agile cargo pods for point-to-point delivery missions.
118
TheDavidTai retweeted
🚨 OpenCode flaw lets malicious MCP servers execute code on Windows and WSL when developers initiate sign-in.
Fixed in v1.18.33, but there's no CVE or security advisory. Vulnerability scanners won't flag affected versions.
What developers need to know: thehackernews.com/2026/10/op…
33
81
419
56,769
Up 250%, down 25%.
4TB Samsung 990 Pro M.2 SSD falls 27%, hits 20 cents per gigabyte tomshardware.com/pc-componen…
2
4
423
Universities may be the first to build relatively inexpensive local computing clusters because of this controversy. Business don't have this level of data paranoia yet.
WTF. In a guest post on Terence Tao's blog, NYU mathematician Tristan Buckmaster accuses OpenAI of competing against its own users with 'internal models unavailable to the public,' calling the lab's October 6 dump of 700+ AI-generated math manuscripts a 'PR stunt' dropped without checking if it built on the private work of grad students and postdocs who were 'literally days away from releasing their own work.' He also challenges OpenAI's claim of roughly three hours of computation per result, arguing that the figure excluded numerous failed attempts and previous work by other agents, rendering it 'essentially meaningless.'
He says AI companies 'strip mine our data' - including unpublished work, private notes, and grant applications - while researchers typing half-formed ideas into consumer models hand them a roadmap of their thought process. Warning that AI labs must not become 'the ultimate gatekeepers of knowledge,' he calls for zero data retention, opt-in training, and independently controlled open-weight models, concluding that 'the race to be the first to resolve a famous open problem is over.'
1
2
238
It’s ok but flash is so maxed on 128 Mac’s that we aren’t seeing improvements on anything but m5u
This has been the ONLY missing thing for Qwen 3.8 Flash. A DFLASH drafter. Hopefully moving this to Qwen 4 Flash should be faster and easier. Because then Qwen 4 Flash could end up being REALLY the BEST model for 1 - 4 Sparks for its capability but also its speed.
2
321
PHP's always been a bug pinata but its funny with agents randomly finding them on verification of other bug fixes.
chat. chat... its not funny anymore
Community note
The original post has since been edited to specify that this was about an unrelated vulnerability that the user had discovered, not the recently patched vulnerability.
fedi.katze.sh/@elise/1174793…
151
TheDavidTai retweeted
Leaving this here, PHP 8.5.12 security patch is incomplete. ( needs further validation )
Community note
Unvalidated AI output (author admits in replies: omp with glm5.3 + ds4), not a lab result. Its screenshot says both CVEs are fixed; the 'surviving paths' need unserialize() of attacker data or an app LFI. Patch per php.net: php.net/archive/2026.p…
76
368
2,586
493,054
Wow, a 40b model that rivals Qwen 3.6 35b!
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
6
23
1,458
TheDavidTai retweeted
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
605
864
8,202
3,305,083
TheDavidTai retweeted
🚨ThinkingCap: NVIDIA Nemotron3.5 Lightning30B A3B is out!
⚡️Up to 2.3x more requests for multi user use compared to original Nemotron version.
🚀 Up to 31%, on average 23% fewer thinking tokens
🎯 Minimal accuracy loss - 0.88pp
📄 Blogpost: bottlecapai.com/post/thinkin…
🤗 Hugging Face: huggingface.co/bottlecapai/T…
🔗 GGUF: huggingface.co/bottlecapai/T…
11
15
83
8,585