Insane Frog retweeted
>Multi-Millionaire begging his viewers for donations to sue someone so he can get even more money
Hasan confirms he will probably fundraise soon for his Scott Jennings lawsuit
"It's a very costly endeavor..these are very powerful people..this person is quite literally a prominent figure in the media..I don't think you should be able to get away with lying like this"
"If I don't put a stop to it, they're gonna continue..if these guys will not hold up ethics in media and journalism..someone has to fight back..completely unacceptable the way that media and entertainment has merged..and people use these platforms to say whatever they want about their political opposition"
7
42
645
5,738
Buying one of these is an unbelievably bad deal
You could literally get two 256GB M5 Ultra Mac Studios for less money and connect them over thunderbolt 5 RDMA and get significantly faster AI compute, not to mention twice the storage
0nly kinda makes sense if your targeting a 2 TB+ cluster for $85k~
I imagine this decision was made to exclusively target enterprise customers, but once you're spending near $100k on local AI, I'd imagine most businesses willing to drop this much money would would prioritize the much higher AI compute that's Nvidia alternatives would offer over the energy efficiency of Macs
Even if technically $85k~ is cheap compared to standard Nvidia or AMD alternatives if all you want is capacity, models that large are not going to be fast even in an RDMA cluster of these things
So it only makes sense if you're planning on buying four or more of these things and are okay with frontier models running very slowly, probably 10-20 tokens per second on frontier models that can actually take advantage of the memory
However this is still mostly a bad deal since projects like Strata are becoming more popular, which allow these large MoE models to intelligently stream experts from fast PCIe 5.0 storage into system memory and VRAM and keep them in faster memory based on how frequently they are used
If all you care about is capacity, advancements in software are making if possible to run extremely large models for much cheaper using 2+ TB 14 GB/s SSDs combined with 256-512gb of system memory and more modest amounts of vram on GPUs connected over fast PCIe 5.0 lanes
A system like this would likely cost somewhere around $20k and produce similar speeds to the 512 gb M5 ultra cluster for processing one request at a time
Parallel requests might take a harder hit than the M5, but not as much as you might expect, and at a quarter of the price, you could literally get more overall parallel throughput by building multiple of these hybrid systems and still come in cheaper than if you were to have bought the M5 ultra cluster
The only remaining actual strong suit of the M5 Ultra cluster would be fine tuning large models, which would be really poorly suited for the hybrid system, but would be possible yet slow on the Mac cluster
Fine tuning 1-2 TB models is genuinely the only hyperspecific reason that Apple thinks they can get away with pricing the 512 GB option this ludicrously expensive compared to the 256 GB option
Anyway sorry for geeking out, I promise my next tweet will be something racist 🙏
14
943
RT @Aliathewhite: People say if we cut off aid to Africa they couldn’t feed themselves.I say give it a chance.
326
117
Insane Frog retweeted
leftists portray israelis as blonde haired pale skinned and blue eyed so that you know you're allowed to hate them
Well it finally happened. The Human Shields video has reached 5 MILLION views!!! Despite Google’s shadowbans, throttling, demonetization, and other suppression tactics, thank you to everyone who viewed, shared, referenced, or subscribed. Keep it up! piped.video/watch?v=FwFsvx1K…
122
605
12,293
165,691
