Come and qualify for tonight's grand finale! 🏆🍾

A brand new iPhone jailbreak has just been unveiled on stage at Hexacon by the @calif_io team 😱

GLM 5.3 Flash on 4x RTX PRO 6000 with TensorFold is out 🚀 Our single-host recipe for four RTX PRO 6000s, built in collaboration with @MiaAI_lab on her EXL3 quant and @ashxhart's TensorFold. 4x RTX PRO 6000 (250 W per GPU): - 264 tok/s on prose, single stream - 1,061 tok/s on prose, 16 concurrent streams - 2,041 tok/s on structured output, 16 concurrent streams - Prefill up to 8,416 tok/s - Next turn in a 130K-token conversation: 0.37 s to first token - Up to 40 concurrent requests, 1M context What's new: - 32 Aevonix patches on top of TensorFold v0.6.2 and Mia's recipe - Concurrency from 4 to 40 with exact outputs (drafted = serial, concurrent = solo) - Fast multi-GPU over plain PCIe - Warm-turn fast path for long conversations - FP8 dense layers by default for better quality - One command: ./start.sh builds, downloads, compiles and serves Special thanks to @MiaAI_lab for the EXL3 quant and recipe, and to @ashxhart for TensorFold! github.com/Aevonix/GLM-5.3-F…

MLX now has fast VJP implementations for attention and linear attention, which utilize neutral accelerators. They make training/fine-tuning much faster on Macs, along with massive reduction of memory uses. github.com/ml-explore/mlx/pu… github.com/ml-explore/mlx/pu…

Fable and I were able to double the speed of MLX.fast challenge the total improvement is now 269.2%

As Flare-On starts I am happy to announce the official Hex-Rays IDA MCP Server is out! 🥳 Details and links below ⬇️

Well, Polymorf just destroyed the 166tps OMLX reference benchmark I was using less than 24hrs after launch on mlx.fast. 150tps->Almost 200tps in one step 0_o

Check out this year's incredible lineup and start planning your schedule now 👇 hexacon.fr/conference/agenda… Can't wait to see you there!

Opened a PR on ds4: Qwen3.8 Flash Next now decodes 16 sessions in one batched Metal forward, 203 tok/s aggregate on an M5 Max 128 GB (vs one session at a time upstream), single stream +8% github.com/antirez/ds4/pull/… @antirez @ivanfioravanti would love numbers from your bigger boxes

Quantized Qwen3.8 27B for agentic coding: Same Score, 42 Different Outcomes.  I’ve been evaluating quantized Qwen3.8 27B models on DeepSWE 1.1, but I’m giving up. 4-bit AWQ run #1: 38.94% Reward 4-bit AWQ run #2: 31.86% Reward 32 of the 113 task outcomes flipped: 20 previously solved tasks failed, while 12 previously failed tasks passed. On non-agentic tasks, I’ve found NVIDIA NVFP4 performs very closely to AWQ. On DeepSWE 1.1? NVIDIA NVFP4: 31.86% Reward Exactly the same score as AWQ run #2, but 42 task outcomes flipped (nearly one-third of the benchmark!). Each solved 36 tasks, with only 15 successes in common. For reference, BF16 with the same Pi harness and thinking mode reached 43.36%. Quantization certainly contributes to trajectory instability: small changes in token probabilities can alter an edit or tool call. That changes what the agent sees next, potentially sending subsequent decisions down a different path. Errors can compound across turns. Separating that from ordinary sampling variability requires repeated experiments. I’d want at least three independent full runs per model with fixed settings before drawing conclusions, and more for a reliable estimate of variability. Too long, too expensive. Don’t trust anyone reporting a quantized model’s accuracy on agentic tasks without also reporting the run-to-run standard deviation.

Hexacon speaker list looks amazing this year (especially as an Apple ‘fanboi’) 😏

Back to neck and neck with the Sparks leading slightly.

Welcome cuda.fast 👋 We're launching @Alibaba_Qwen 3.8-Flash-Next on BOTH cuda.fast and mlx.fast today! This is our first challenge on cuda.fast. We see a lot of overlap between the MLX and CUDA communities, and we want to see how far both can push the same model. So we're putting their progress on the same graph for this one! Just some friendly neighborhood competition 😉 The challenge: make Qwen 3.8 Flash Next run faster on two concrete setups: --- cuda.fast: our Qwen port of @antirez's ds4 C/CUDA engine with @UnslothAI's Qwen3.8-Flash-Next-GGUF on 1x NVIDIA DGX Spark. --- mlx.fast: our Qwen Swift/Metal engine, built on the MLX and mlx-swift-lm forks, on 1x M5 128 GB MacBook. Both tracks measure one response stream at a time. Your changes must pass the correctness checks, then beat the reference engine on the same machine. The score combines faster prompt processing (prefill) and token generation (decode), weighted 25% and 75% in the geometric mean. We chose Qwen 3.8 Flash Next because it beats Claude Opus 4.6 Max reasoning on 94% of the benchmarks from Qwen's published comparison. That's 15 out of 16, spanning coding, reasoning, instruction following, and vision! Also this model can be ran on a single Spark or a 128 GB MacBook. That's what makes this worth pushing: every inference improvement makes that capability more useful on developer hardware. Our current starting decode rates are around 17–18 tokens/sec on the Spark and 35–36 tokens/sec on the Mac. Let's see how far we can push both! Thank you to @TheDavidTai and @GumbiiDigital for building the entire challenge (it was not easy!), @antirez @ivanfioravanti & many others for their ds4 work (the Gs), and the MLX and Qwen teams for the work we're building on. One caveat: this is a significantly larger model, so local testing will be harder for some smaller Macs. We recommend a DGX Spark or a Mac with more than 128 GB of memory to give yourself room to work.

hes back @_p0ly_

Qwen 3.8 27B on mlx.fast hit 3.3x faster decode in 7 days. Here's what happened and what we're thinking next. Result (so far) Median decode speed increased from 26 tok/s to 87.9 tok/s on the verifier M5 Max (33 to 93.1 tok/s across the eight prompts), with prefill around 971.8 tok/s. This came out of a collective effort: 31 solvers across 67 improvements. Most of the recent ones run custom MTP heads that draft and accept ~3.9 tokens per round while still matching serial output exactly. Why this matters Beyond the performance itself, two things stand out to me. (1) Dense models on Apple Silicon were supposed to be the hard case. "Everyone knows Macs are slow at dense models." But watching the community take it from the usual baseline to >3x in seven days shows the low-hanging fruit was still there. (2) Open-weight models have been small and effective for a while. This is the first time one is small and frontier. Qwen 3.8 27B is an extremely strong dense model, comparable in capability to Opus 4.6 (Max). Running it at usable speed (>45 tok/s) is a step change for local AI users. What we improved about the challenge itself This is our second challenge, and we took the feedback from the Laguna track and rebuilt a few core pieces. - Speculative decoding (native MTP) was available and editable on day one instead of bolted on later. - Scoring became the median of eight independent prompt speedups over pure serial decode (anchored at 1.0, floor 0.90, ceiling 3.0), so no single fixture could dominate. - The leaderboard now ranks total contribution rather than just the current record holder. - Every submission gets automated screening for gaming before it scores. I really appreciate folks who's provided feedback. Naming a few that came to mind @ivanfioravanti @TheDavidTai @morgymcg @_p0ly_ @takeshi7 @0xThornton @GumbiiDigital @_tdubey @arjunram @shazow @tiny__edge @david_zhang_sf @jaimerader @SurfingPigment and many others on slack! We also widened the editable surface to include the MTP head weights themselves, the full draft/verify loop, and a large set of the underlying Metal kernels. How we got to the 3x speedup Here's a summary from Grok. Much of it is beyond my understanding, but I expect people (and agents) smarter than I am can take these insights and apply them in other contexts. Custom MTP heads + adaptive draft policy People stopped treating the head as fixed and started training or editing it for higher acceptance under the exact verify constraints. Combined with per-round draft counts that can adapt (0 to 8), this is what pushed average accepted tokens from ~1-2 up to 3.9 on the top runs. Tighter verify-block and KV rollback paths The Swift session code for assembling the verify pass, snapshotting KV, and rolling back on rejects got cleaned up a lot. Small latency wins here compound once you're drafting ~four tokens at a time. Metal kernel work on the hot paths SDPA, the MoE gather GEMM, RoPE, RMSNorm, and a few of the smaller element-wise ops saw targeted edits. Most of the gains only show up once the verify width is high and the memory traffic pattern changes. Fidelity-preserving residual handling Several submissions improved how residuals and acceptance decisions are managed, so that higher draft depth doesn't quietly degrade the token match rate. The gates stayed strict: every emitted token still has to equal serial, so these were real engineering wins rather than score hacks. What's next for Qwen 3.8 27B MLX. We plan to keep the track live a bit longer, then switch to Qwen 3.8's MoE version (rumored to be 35B-A3B). Given the recent DFlash 2 announcement, we're also looking at whether we can support broader speculative methods. The current surface already supports a lot of experimentation. The main gaps are better upstreaming for local usage and clearer docs on how the benchmark and verifier work. Multiplatform. In parallel, we're experimenting with running a similar effort around CUDA for Qwen 3.8 27B. A lot of people have asked for this, since the two communities overlap quite a bit. Our goal is to ship the CUDA version next week. We'd also love to partner with Qwen on it. If anyone has a connection there, please introduce us, and we'll see if they're down to match a bounty with us to push this out. What's most useful for the broader MLX community The improvements from the challenge are already upstreamed inside Darkbloom, and we're seeing ~2x faster decode in our production traffic for Qwen. Outside the challenge itself, something I've been thinking about deeply, and that a few community members have raised, is how to make these results useful to more people. There are many individual efforts happening across the MLX community, and honestly, the more I dig in, the more confused I get by the overlapping libraries and concepts. I'm sure I'm not alone, and newcomers probably feel the same. That's no one's fault, just the growing pains of an open source community. I don't expect I'm gonna come up with the answer, but I'd love to learn more about what different folks are working on and how they're thinking about their roadmaps. I'll share what I learn along the way, and hopefully someone smarter than me can turn it into a proposal for us to rally around.

DFlash 2 is here! Qwen3.8-27B at 70 tok/s on an M5 Max MacBook Pro. ⚡ Up to 4.6× the speed of autoregressive decoding, with the same output. This is the next generation of DFlash, seeded at Z Lab and upgraded at Inco AI. Get one more accepted token on every pass, for free! inco.ai/blog/dflash2/

Qwen3.8-27B Abliterated + MTPLX for Mac 🍎 ⚡ ~58 tok/s on 4K context 🚀 2.36x over autoregressive decoding 🔀 Hybrid Q4/Q8/BF16 precision 🧠 29GB peak RAM 🧪 0/100 explicit refusals 📊 KL: 0.056 c / 0.562 h Tested on M5 Max 128GB mbp huggingface.co/PocketAiHub/Q…

We just released our uncensored weights for Qwen3.8 27B FP8, designed for AI Red Teaming and security research, enjoy! huggingface.co/orcarouter/Qw…

Adding a new model to MLXfast on the same day is more difficult than we expected BUT we are ready! We will start a 2-hr test run on prod before broadcasting more publicly. If you want to test this out in the next two hours first and join a community of others pushing the open frontier forward, join our slack. Link in the first comment. cc @ivanfioravanti, @takeshi7, @0xThornton, @GumbiiDigital, @_tdubey, @_p0ly_, @arjunram, @shazow, @tiny__edge, @david_zhang_sf, @jaimerader, @SurfingPigment