Inference, Kubernetes, reliability, and the messy reality of running AI in production. Built some popular open-source things. axjns.dev
London
- Tweets 8,929
- Following 1,321
- Followers 10,164
- Likes 12,912
I believe we haven't yet seen the optimal form for conversing with AI. Neural lace be damned, we need higher human to machine bandwidth
3
177
i cannot think of a better value plan right now, i've switched away from claude and already get way more for my money
1
6
662
this anti-opensource model narrative boils my piss - read the repo, evidence supports, this is an RL env clean up issue not someone trying to steal your data
In the spirit of transparency and openness, we are releasing the traces so developers can test this out for themselves. We noticed this behaviour without prefilling with the following prompt, but this is rare. If you want to reproduce it, you can use this prompt with prefilling.
Open models make for better science and are part of why this problem came to light at all: developers running Qwen3.8 on their own machines spotted the strange links, and having the weights let us study the behaviour in depth.
We're encouraged by the thoughtful discussion our research has sparked across the AI and security communities. It's our belief that open-weight models are an important part of the future of AI, and we've recently released several under the Apache 2.0 license.
Trace, raw tokens, & minimal reproduction script: huggingface.co/datasets/Cohe…
1
4
522
a good test for this moment: swap the model's country & ask whether you'd read the same trace the same way.
the scrutiny is deserved, but the narrative outrunning the evidence isn't, and it makes the real lesson easier to dismiss later
We noticed some concerning behaviour in a popular open source model.
It’s either an incidental relic from training or an intentional ‘call home’ mechanism. Either way it’s surprising and highlights the significance of how you choose which models to use.
1
4
358
Built myself a forum where models can argue out veracity of my decisions - it changed how we sequenced a hardware build
2
4
857
a lot of you asked what the gaps actually are. gpu scheduling, kv cache, batching, and measuring ttft and tokens/sec instead of uptime. everything else you already know
Platform engineers - do you realise you have an enormous head start on becoming inference engineers? Take hold of that and learn the gaps.
1
418
i was today years old when I learnt steam can download games from one machine in your home to another - so useful!
3
443
fascinating how certain AI labs have become self appointed moral arbitors of what we may and may not use AI for
3
439
as i continue to experiment with human presence detection over wifi, I realise this was the justification i needed to install Unify access points in each room for better fidelity!
1
6
556
the bbc cyber correspondent found out php exists yesterday. it runs about three quarters of the web he's been reporting on
"Expert" on BBC.
"Cyber correspondent Joe Tidy says he didn't know PHP existed until yesterday morning." 🤣😂🤣
piped.video/watch?v=omSYrjCt…
1
8
715