i should have posted this a long time ago hey, i'm sudo, and if you own a gpu and want to get your hands dirty with local ai, follow along i run hands on experiments with local models on every tier of hardware, 8gb, 12gb, 16gb, 24gb, 32gb, 64gb, 128gb, all the way up to 256gb and climbing, and i've been in the local ai space for a while now i also drop recommendations for the uncensored models and the weird ones, odd models and odd setups, so you can experiment and actually understand what's happening on your own hardware my goal is to get as many normal people as possible started in local ai, and give them a taste of what owning your thinking feels like on your own metal so if this moves something in you, listen to it, tell me in the comments what you want to see and i'll do my best to get you the answer, and if i can't, someone else will, the people in local ai are the most helpful crowd i've ever seen

if you are not harness engineering you probably not gonna make it anon

when claude suddenly starts reasoning in turkish (or some other random language), that’s Anthropic quietly serving you the nerfed 4bit quant of opus 5.5.

someone smarter than me explain this. why do we need a whole new GPU generation just to get more VRAM? can't manufacturers use the same chip and put more memory next to it?

i might be stupid but if my SSD can store 4TB of data, why can't i use part of it as extra RAM when i run out? i know it's slower, but what exactly stops us from making this seamless?

which friend would build an entire startup just to avoid paying $20/month?

i'm falling for my own agent harness xpress agent is the harness i built for myself, it's rust and i can feel it in the speed, and offrail 27b is the model behind it, running on my laptop right now it pulls images off the web and lay them out right under its answer, i slid through them and opened the one i wanted, no more clicking through folders to find what it downloaded, and this one feature saved me so much time the model does the thinking, the harness decides what it sees and what it remembers, and you spend your whole day inside the harness, so it should fit the way you work > one binary under 10 mb > 14 mb of ram for the whole loop > finds my local model by itself > self learning loop it writes its own skills as it works, and on that same run it rewrote its image skill without me asking i think eventually you all will have to build your own agent harness the way you like it and the future of agents is rust

here are the 1017 weights and the recipe, the launch build stays up next to them > weights: huggingface.co/sudoingx/Deep… > recipe: github.com/sudoingX/dgx-spar… 196 gb of weights plus deepseek's own 1017 engram shards, the launch ones differ, and the heavier dense and head runs sit in the repo's benchmarks with their speed

i rebuilt my deepseek v4.1 flash weights on the 1017 update, v1.2 is live on 2x dgx sparks 🐋 > structured output: 141.3 tok/s > code: 100.7 tok/s > prose: 59.2 tok/s and the quality held, my 2.70 bit weights scored against native 1017 on my own kl panel, the one other 1017 packs now report on > kld 0.0243 from native 1017 > 20% closer than libertaidai's 2.9 bit pack > top-1 agreement 95.47% > humaneval+, mbpp+, ifeval within noise > ifeval replies: theirs wins, 0.1135 vs 0.1199 deepseek shipped 1017 as a minor update, so i diffed it against the checkpoint my build came from, the config, tokenizer and tensor layout match byte for byte and every shard's content is new, in the slices i sampled about 27% of the 4 bit expert codes flipped and the built-in drafter moved the most there was nothing to patch, so i pulled the 510 gb checkpoint again, measured all 15,360 experts, ran a 16 hour encode with one spark down for almost 3 hours while the other carried on solo, then dense layers, head, drafter and every eval, about 25 hours from download to serving it runs about as fast as my launch build, which did 103.0 tok/s on code and 58.0 on prose, a verify round on code still takes 42.8 ms on both, the new weights just let the drafter land 4.31 tokens a round on that reply instead of 4.41 by the time mine finished, henryous and 0xsero had shipped 1017 packs built on my recipe, and henryous reports 0.0201 at 2.75 bits with heavier dense layers and head, closer to native than mine so i tried that mix on my own experts, dense at 5 bits and the head at 6 got me to 0.0219 for about 5% of my speed, i'm shipping the fast build and the heavier numbers are in the repo same engine as v1.1, so you keep the image and swap in the 1017 weights plus deepseek's 1017 engram shards next i'm training my own drafter for it to push more

holy shit, llama.cpp rpc actually works this is 4x dgx sparks hosting ling 3.1 flash, a 560b moe, at q4 across all four gb10s, about $28k of sparks at today's $6,950 a box no one shipped an official fp4 yet, so i had to try the gguf over rpc and it's possible. it's super slow because of the engine, but i really like the model's output, it has weight to its answers i couldn't stop myself from running this one, i ran ling 3.0 flash on a single dgx spark before qwen 3.8 took over, and this one feels like a real improvement, but it doesn't fit on my 2x dgx sparks, so @niklaslenz_ai, a friend here on x, lent me his 4x dgx spark just to run it it took about 12 hours to complete a build and i'll share it next, stay tuned anon, looks like @AntLingAGI just had an ai leap

yesterday this was a tweet asking a grokbot to build it. since last night it's the only agent i open xpress agent v0.1.0. local first, learns as it works, written in rust grokbot prototyped it, i took it into cursor and engineered every inch of it, and it's already the best local-first agent i've ever used. built by me, for me the moment you launch it, it finds the model server on localhost by itself and configures itself. no config file, no setup. my 27b is just there, 128k context, ready to work /model to switch between local and cloud. free openrouter models chained as fallbacks so it never dies mid task. permissions ask by default, shift+tab to change. /resume any past chat. @ any file. web search built in and the part i love most: it learns as it works. hermes agent's self-improving loop is the best idea in agents right now, so i rebuilt it in rust my way one binary. no python waking up. it loads faster and answers faster than hermes agent and openclaw on the same model. and my api keys never leave my machine this era is insane. if you're still waiting for someone to ship the tool you want, you're already behind. build it yourself with @bot, in a day

i owe the new people here an intro hey, i'm sudo, i run local models on my metal every day, my 24gb laptop, a 128gb strix halo and 2x dgx sparks, and i post the numbers and the exact commands so you can run the same thing at home i do most of this for free, the tools too: > a one-line installer, 8gb and up > serve recipes for the big models > my own model builds on huggingface last week that was an uncensored 27b at 50.8 tok/s on a 12gb card, the whole model in a 6gb file anyone can download and i help as many people as i can get started, in the replies and dms, so if you're new here anon, reply with your gpu and how much vram it has and i'll point you at the first model to run on it

can i ask a DUMB tech question? why can't we just add more VRAM to a GPU like we add more RAM to a PC? what exactly makes this so difficult?

30 years ago you upgraded vram with your thumb, then they took it away like the headphone jack

i asked one DUMB question and my replies turned into a hardware lab almost a million views and the thread went deep on vram mods and hbm, so if i got this right > gddr is soldered next to the die > 28 gbps a pin, a socket kills that > every chip owns 32 bits of the bus > 384 bit = 12 chips, clamshell = 24 > 48gb 4090 = clamshell + bios mod > hbm is on the package, no upgrades and then there's the rtx pro 6000, the same gb202 die as the 5090 with 96gb instead of 32 and a workstation price tag if you've modded vram onto a card yourself, drop the board below, i want to see every one

if you have never seen 4x dgx sparks dancing, here they are serving ling 3.1 flash for over 12 hours > about 20 w of gpu power per box > 49 to 55 °C after 12 hours serving > no restarts across a 12 hour build the spark's design is so power efficient for the work it does, and the heat flow and the scaling are just as well designed, nvidia supports clusters officially now i think in 2026 it's the best entry point i know into a desktop ai supercomputer that handles almost every ai workload, and the ecosystem is so much broader than 6 months ago it's basically a subscription canceling machine, from hosting your ci runners to serving your own models, and if you're planning to buy one don't overthink it, it's the best value for a 128gb cuda node even at today's price

i am running a monster model on 4x dgx sparks right now the model is ling 3.1 flash, i tested the older ling versions and enjoyed them, and i wanted to try this big one but it doesn't fit on my 2x dgx sparks, so a friend here on x, @niklaslenz_ai from germany, let me borrow his 4x dgx sparks and now i'm set up and about to run my tests there's no official fp4 build for vllm yet, so i'm on atomic chat's day one q4 gguf with llama.cpp rpc across all 4x dgx sparks i know bandwidth and speed will torture me, but i couldn't stay away from loading this one because i want to know where it lands next to the 124b ling 3.0 flash stay tuned, i'll drop numbers as we go, and if you've already run it, share your thoughts

Deepseek said fuck theo