Assistant Prof @columbia CS. Visiting Researcher @ Google DeepMind. PhD from @stanfordnlp. Language x Neural Nets.
New York, NY
- Tweets 267
- Following 65
- Followers 8,011
- Likes 558
Pinned Tweet
New from the lab: @LorenaYannnnn finds that reward misspecification during RL causes unmonitorable chain-of-thought. Models talk about the goals in their prompt, even as they learn throughout RL to optimize the misspecified reward. We also propose an intuitive fix; see below.
3
15
111
4,402
New from the lab: @LorenaYannnnn finds that reward misspecification during RL causes unmonitorable chain-of-thought. Models talk about the goals in their prompt, even as they learn throughout RL to optimize the misspecified reward. We also propose an intuitive fix; see below.
3
15
111
4,402
This work was led by my PhD student. @LorenaYannnnn. If you're looking to fund rigorous safety research through our lab, please get in touch with me.
Code: github.com/Lorenayannnnn/rew…
Paper: alphaxiv.org/pdf/2610.reward…
Blog: tianyi-lorena-yan-me.web.app…
1
4
333
We also release our datasets and all of our misaligned and control models as model organisms for understanding unmonitorability due to reward misspecification. Can white-box monitors help?
Models: huggingface.co/collections/h…
2
256
My lab today at COLM: @GeorgeMorgulis presenting Subliminal Steering, 11:00-1:00 in Franciscan A #78
If you’re studying subliminal learning in LMs, you should be studying George’s variant, which is induced by steering. It just works way more consistently.
1
3
46
2,497
Come say hi at COLM! I’ll be around, and I’ll make a few spots available for 1:1s. As usual, I’m hiring PhD students. interp/safety/reasoning/understanding/++ !!
8
7
235
35,169
As agents perform increasingly complex tasks for researchers, it's important to develop norms for communicating with other humans. Here's my lab's policy on AI in writing and communication, focusing on helping us think deeply and communicate precisely.
cs.columbia.edu/~johnhew/lab…
6
52
323
26,489
Our policy is to be in charge of technical terms. Recent agents love introducing technical-seeming terms, but strong communication involves developing a small number of precisely defined terms and applying them consistently.
1
12
2,107
Camera-ready for subliminal steering is out! New:
- Subliminal learning happens for standard finetuning (not just LoRA)
- Subliminal learning happens for vanilla SGD (not just Adam)
I think studies of subliminal learning should default to biasing via steering, not prompting.
New paper! Subliminal learning—transferring hidden signals between language models—is more powerful than we thought. By biasing the teacher with a steering vector instead of a prompt, we achieve strong, consistent transfer, which we use to study its mechanisms. w/@GeorgeMorgulis
2
1
70
8,715
See our related work for a discussion of what prior work had found on whether subliminal learning was limited to just LoRAs of certain ranks or just certain optimizers. We started pursuing these questions because our COLM reviewers wanted to know. Cool!
arxiv.org/pdf/2604.25783
8
941
I’m at ICML! I’m hiring PhD students for my lab at Columbia. DM me here on twitter to set up a time to chat.
cs.columbia.edu/~johnhew/lab…
15
32
533
51,615