Yeah this drives me nuts. I don't want to drop $40-60k on hardware and then have to TUNE THE CRAP out of it all day/night either.
Right now the subs are still cheaper, at least in a 2-4 year timeframe.
But then, when new models come out that are heavier (though I'd argue, they'll start to trend down into specialty models instead of bulkier LLM models), the hardware can't fit the bill.
It's a weird crap shoot.
I'd love to self-host everything myself... I've done some efforts to do that, but every time I start to, I realize it costs more to keep up.
Kimi K3.1 just dropped, here is my honest review.
1. We will finally be able to run an Astra level model at home, but here is the catch. It goes open weights by Nov 4, but running it yourself is $60K+!!
At 8-bit, 2.8T params is about 6 Mac Studios with 512GB each, before you add any context.
2. The Kimi coding plan has a monthly cap too, so one heavy day can burn the whole month and you're stuck. Kimi's API is cheap, but Claude Max is still way cheaper than paying per token for the same work.
3. Compared to Opus 5.5, I think K3.1 lacks thinking. It's just alright. Maybe my bar is too high after Opus 5.5, but I'd rather pay Claude than Kimi.
It's still the closest we've had to an open weights Astra. Huge congrats to Moonshot for doing this.