ˈli.ɹə 🏳️⚧️⚢ · 25 · ableton enjoyer · mechinterp · base model appreciator · evals and data farming · lyraaaa_ on discord · ♡ @bubblemoder ♡ 🦔~ ♪❀🫧 ⁸¹ ¹² ⁴¹
eugene / sf
- Tweets 8,075
- Following 961
- Followers 13,569
- Likes 99,700
Pinned Tweet
"whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided."
- brian eno
8
12
237
60,378
lyra retweeted
I really like that coding agents are now good enough that you can just make your own ultra specialized tool for the one thing you're curious about, which in the past nobody would have taken the time for. Nice example:
25
38
681
24,116
if you have a qwen3.5 4b around, please try steering neurons L14 N8949 +1 and L13 N7956 -1
now you have evil qwen
recipe: dose × median(‖resid‖ leaving layer L, tokens 1:) / ‖W_down[:, n]‖` at the down_proj at every input
246
627
12,481
1,444,549
lyra retweeted
We're releasing the data used for this work by the way, including rollouts with reward hack labels as well as the data for our probes. Check out the updated draft!
Models know when they’re reward hacking. But they still do it a ton - in 50-96% of rollouts we studied!
We built activation monitors that detect the behavior behind the Hugging Face hack in real time. This can help us stop hacks now - and train future models that don’t cheat. 🧵
4
9
150
7,066
download it here
if there are bugs or ux gripes pls tell me
github.com/lyramakesmusic/te…
1
18
4,288
lyra retweeted
There is an (unpatchable?) supply-chain zero-day in PyTorch autograd that allows a regularization term to use the model's weights as RAM and SGD as its CPU clock to run any computation during training.
88
216
2,397
165,405
combinatoric usersims at scale !!
oh to be able to look at some of the rejected traces....
Metas Muse Glimmer tech report is pretty interesting.
Metas Muse Glimmer report describes training agents inside simulated digital lives, complete with 10–14 contacts, weeks of messages, calendars and documents. The agents practice tasks spanning multiple days, while training trajectories containing hallucinated results or unverified claims of success are discarded.
1
1
9
2,830
lyra retweeted
maybe an even better example is @kataletheia 's work extracting weird non-linguistic steering vectors for eg entropy or number of entities/generators in a conversation. which yeah is quite difficult to make a safety case for but is quite interesting and someone should support her
4
1
38
1,151
lyra retweeted
It would be a tragedy for embedding models to become narrowly optimized for retrieval in the same way that generative text models have become narrowly optimized for conversational rollout. There is much magic to explore in the realm of representations.
14
26
297
13,677
