Open model research, founder, director @trillium_labs. Prev. co-led Olmo at Ai2. Writes @interconnectsai, wrote posttrainingbook.com
Cambridge, MA
- Tweets 12,377
- Following 958
- Followers 107,091
- Likes 44,765
Pinned Tweet
Interconnects Podcast #20: We got together to debate hard questions and messy realities after a crazy, somewhat intimidating, week in the AI world:
- Why did Kimi K3.1 Pro have a mixed reaction with such incredible benchmarks? Will it be adopted?
- Were we wrong about the risks of open-weight models and cybersecurity?
- Why do cyber experts discuss recent events so differently than AI experts?
- What comes next in the few months? What will change our minds?
Chapters:
00:00 Kimi K3.1 Reaction
08:15 Kimi’s open-weight cyber principles
11:23 You can hack everything era
15:54 PHP2Shell, LiteLLM, and the new internet
24:00 Abliterated APIs, safeguards, and trusted access
31:41 Anthropic not hiring juniors employees
36:16 Florian’s Muse addiction
45:30 Wrap-up
More of these soon!
28
12
171
14,614
Interconnects Podcast #20: We got together to debate hard questions and messy realities after a crazy, somewhat intimidating, week in the AI world:
- Why did Kimi K3.1 Pro have a mixed reaction with such incredible benchmarks? Will it be adopted?
- Were we wrong about the risks of open-weight models and cybersecurity?
- Why do cyber experts discuss recent events so differently than AI experts?
- What comes next in the few months? What will change our minds?
Chapters:
00:00 Kimi K3.1 Reaction
08:15 Kimi’s open-weight cyber principles
11:23 You can hack everything era
15:54 PHP2Shell, LiteLLM, and the new internet
24:00 Abliterated APIs, safeguards, and trusted access
31:41 Anthropic not hiring juniors employees
36:16 Florian’s Muse addiction
45:30 Wrap-up
More of these soon!
28
12
171
14,614
YouTube: piped.video/0xiq7RHVGRE
Transcript / Podcast: interconnects.ai/p/podcast-k…
1
3
2,482
DeepSeek is serving 150T tokens a day on one cluster, more than the open model inference on OpenRouter, Fireworks, Together, and BaseTen combined.
Below is my reconstruction of various inference volume reports across said companies.
right from the horse's mouth
ah, right. That's "a single cluster", who knows how many they have
github.com/deepseek-ai/DeepI…
26
35
428
30,287
At the top end of models, real-world usability matters a lot more than +/- 2 points on AA. This takes a while to understand for a model, where AA is a sort of potential, but some models hit and some miss (e.g. opus 5 vs 5.5). Seems like 3.1 may be closer to the miss side.
52
10
274
20,922
Not a lot of positive Kimi 3.1 reactions on the timeline. If you've had time to try the model, what was your impression? (I haven't yet)
69
5
315
54,608
Another:
K3.1 Pro by @Kimi_Moonshot on KingBench 4.
Leaving openness aside, This is a bad model.
It scores the 6th position with 53.7% and costs $31.88 which is jut $16 cheaper than Opus.
You can get better results than this with Haiku or Sol for like $25 cheaper.
The whole pitch of cheap looks like BS right now.
The model constantly re-reads cache and costs way more than most models and it also under-performs on my bench.
It seems that they found out a way to make a model cost more by basically making it read caches more often to recover their costs (which is also why cache reads cost the same as Opus).
In no way will I tell anyone to use it.. The cost to performance is really bad.
3
21
7,088
This is a mistake, but I suspect they will correct. Chinese labs are VERY EAGER to influence the western ecosystem and X, so they likely took a step too far here. They follow the vibes closely and I bet will understand (even if some cultural differences caused this)
That's how Kimi AI are promoting their new model Kimi K3.1 pro and I know all the influencers working with them and none of them are honest,I have posted about it myself but I didn't join this and never used their templates that they provide.
PROPAGANDA, I have more to share it's f**cked up I'm gonna have follow up for this
33
11
239
31,404
Higher score than Fable 5.1 and Astra 6, wild stuff.
Kimi K3.1 Pro launches at 4th on the Artificial Analysis Intelligence Index scoring 55, making large gains in agentic performance; it will be the leading open-weights model once its weights are published
@Kimi_Moonshot's new reasoning model gains 11 points over Kimi K3 (max, 44). It trails only Claude Opus 5.5 (max with fallback, 58), Claude Sonnet 5.5 (max with fallback, 56), and Muse Wonder Full (max, 56) on the Intelligence Index.
Once the weights for K3.1 Pro are released it will be the highest-scoring open weights model and the gap between the leading open weights and proprietary models on the Intelligence Index will narrow from 12 points to 3.
Key results:
➤ Large gains in agentic performance: Kimi K3.1 Pro scores 3rd on GDPval-AA with 1,846 Elo, up 312 Elo from Kimi K3, behind only Claude Opus 5.5 (1,866 Elo) and Muse Wonder Full (1,857 Elo). Kimi K3.1 Pro also achieves the highest score of any model on Terminal-Bench 4.0. K3.1 Pro scores 67%, up from 13% for Kimi K3 and ahead of Claude Sonnet 5.5 (64%). AutomationBench-AA, which tests agentic SaaS workflows, rises from 58% to 73%.
➤ Gains in other evals are more moderate: Humanity's Last Exam rises from 47% to 52%, CritPt from 23% to 25%, and SciCode from 59% to 61%. AA-Omniscience Index rises from 20 to 30, as the hallucination rate falls from 53% to 44% and accuracy rises from 48% to 51%.
➤ Cost per Task roughly equal to Kimi K3 at $1.98 per Task: K3.1 Pro uses 55K output tokens per Intelligence Index task, 14% more than Kimi K3 at 48K, while its first-party token prices are 33% lower.
Additional model details:
➤ Context window: 1M tokens.
➤ Size: 2.8T total, 104B active parameters
➤ Input modalities: Text and image
➤ Weights and license: Weights currently not published, Moonshot is expected to release the weights soon after launch
➤ Pricing: $2/$10/$0.20 per 1M input/output/cache hit tokens, 33% lower than Kimi K3 on all three rates.
Congratulations to the team at @Kimi_Moonshot on the release!
16
29
607
46,064
Is debatable on the language "A single cluster currently serves 155 trillion tokens per day, with overall revenue of US$38 per GPU-hour (at valley pricing)." it could be a capacity rather than volume.
2
18
2,958
TLDR on the kimi k3.1 pro benchmarks is that it looks like they figured out their RL, where the initial release was a very impressive base model with lighter post-training. China is very much keeping pace with capability gains.
34
40
1,067
48,961
Kimi K3.1 is announced as 3.1 Pro. The benchmark scores look to be clearly the strongest current open weight model, if the weights are released as they claim by November 4th. They are very impressive scores.
The Kimi team makes very clear statements about cyber risks and trying to pace the frontier of open-weight cyber capabilities and moderate it with respect to the closed frontier.
Much to be learned in how the model stands up the vibe tests, but in the last few years the Moonshot team has rarely missed with the kimi models. K3.1 is also cheaper than K3 per token, but some benchmarks in the blog show it uses meaningfully more tokens per answer, so it may be about the same price.
This, with all the cyber news, and rumors of a bigger GLM model coming soon, sets us up for an exciting rest of the year! The pro name makes me think Kimi will be moving to release flash model soon too, similar to DeepSeek and GLM.
38
24
457
26,532
Nathan Lambert retweeted
I don't think "everything is fine," but anyone claiming that php2shell is "the worst day in cybersecurity history" needs to explain why there's more risk here than:
Innumerable celebrity vulns and exploit PoCs,
Serious protocol bugs in TCP/IP/DNS,
Out-of-control worm incidents,
Nation-state attacks on economies/critical infrastructure e.g. Russia->Ukraine,
Kernel packet-parsing bugs in popular operating systems,
The proliferation of AI social engineering tools since 2022,
The proliferation of AI hacking tools.
If you can't explain why this time is different/worse, with reference to clear technical mechanisms and attacker motivations, you don't have a serious argument..
I also think this kind of alarmism detracts from the genuine fact that we have a looming crisis in security due to AI which we need correct risk models for
17
12
120
16,165