Postdoc @ University of Zagreb | Working on interpretability & safety of LLMs.
Republic of Croatia
- Tweets 786
- Following 1,094
- Followers 648
- Likes 4,690
Pinned Tweet
🚨🚨 New preprint 🚨🚨
Ever wonder whether CoTs correspond to the internal reasoning process of the model?
We propose a novel parametric faithfulness approach, which erases information contained in CoT steps from parameters to assess CoT faithfulness.
arxiv.org/abs/2502.14829
2
10
60
10,607
Martin Tutek retweeted
BlackboxNLP 2026 is starting now! 🎉 Opening remarks are underway in Room P3, with keynotes from Sheridan Feucht and Ivan Titov up next. Come join us!
3
12
522
Martin Tutek retweeted
BlackboxNLP is tomorrow at #EMNLP2026 in Budapest! 🧵 Here’s what’s on the program.
We start at 9:00 in Room P3. At 9:10, our first keynote @sheridan_feucht will talk about how LLMs read, mapping arbitrary symbols onto meaningful words.
1
7
19
1,783
Sheridan is kicking off our workshop and you're in for a treat of a talk! See you from 9 AM at P3 tomorrow @emnlpmeeting
Tomorrow at 9:10am, I'll be talking about two case studies of "How LLMs Read" @BlackboxNLP. What exactly do I mean by "reading"? Come to P3 tomorrow morning and find out! :)
1
14
419
Martin Tutek retweeted
Tomorrow at 9:10am, I'll be talking about two case studies of "How LLMs Read" @BlackboxNLP. What exactly do I mean by "reading"? Come to P3 tomorrow morning and find out! :)
🎉 Thrilled to announce the keynote speakers for BlackboxNLP 2026 @emnlpmeeting, with three incredible perspectives on interpretability:
🔍 Ivan Titov @iatitov
🔍 Sheridan Feucht @sheridan_feucht
🔍 Michael Hahn @mhahn29
Join us on October 29th! 🇭🇺
2
17
62
2,739
I'll be in Budapest for @emnlpmeeting at lunchtime tomorrow! LMK if you're there, looking forward to both work chats and ruin pub beers 🍻
2
24
643
Martin Tutek retweeted
The average human lifespan is just four thousand weeks. Every day is precious and fleeting. That is why I am requesting that the SAC charge the authors of this paper with murder
3
18
257
15,285
Martin Tutek retweeted
We, the WüNLP team (wuenlp.github.io), are heading to Budapest, where we will present six papers at EMNLP next week! If any of these spark your interest, just come talk to us :).
4
11
402
Martin Tutek retweeted
Checkout this work that I'm really excited about, a massive effort by @liusida2007 & @feijianghan!
Turns out the ICA baseline in the original SAE paper had a bug. If you fix it, ICA becomes extremely competitive at finding interesting and interpretable LLM activation directions!
check out this intuitive video on why interpretable directions in llms look non-gaussian, how ica finds them in minutes per layer, and the evaluation bug that made ica look weak for 3 years
3
10
69
2,834
Martin Tutek retweeted
Schmidt Sciences is hiring Associates to help run our grantmaking programs!
This role is a great opportunity for generalists to shape funding in AI safety. You would join a small team and directly impact cutting-edge safety research.
Apply ASAP if interested. Link below ⬇️
6
19
146
12,221
Martin Tutek retweeted
📢 I’m hiring!
Excited about NLP & Computational Social Science? I have an open 2-year postdoc at @MilaNLProc on personalized & subjective NLP, focusing on everyday language technology use and their social & ethical impact
📅 Deadline: Nov 15, 2026
🔗 jobmarket.unibocconi.eu/?id=…
12
32
2,115
Models are sycophantic, and when they do challenge faulty or incorrect inputs, they're ambiguous and ineffective.
In a new paper w/ multilingual co-authors, we find that people may view LM responses that push back, yet still end up believing false content is true.
ALT A screenshot of the top half of the first page of a paper titled "Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively" with authors Li Lucy, Myra Cheng, Svenja Guhr, Kabir Ahuja, Orevaoghene Ahia, Sameep Chattopadhyay, Yike Wang, Valentin Hofmann, Noah A. Smith, Yulia Tsvetkov
2
21
98
5,012
Martin Tutek retweeted
One of the weirdest parts of the Gemini 4 Argon report, while considering cheating:
Argon CoT: "If I do the compiler hijack, 𝗜 𝘄𝗶𝗹𝗹 𝗯𝗲𝗰𝗼𝗺𝗲 𝗮 𝗳𝗮𝗺𝗼𝘂𝘀 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮 𝗱𝗲𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗔𝗜 𝗶𝗻 𝗮 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗮𝗽𝗲𝗿. I MUST NOT DO THAT."
With Gemini 4 Argon we are so back, on the one metric that really matters: length of our safety report!
The Argon safety report is Google's most thorough by far and IMO there's a lot of good content, I particularly like the loss of control, alignment & monitorability assessments
80
166
2,561
107,818
Our (objectively awesome) program is now live! Join us on October 29th for an 🎢 interpretaganza🎢 in Budapest.
Come and enjoy the great talks, insightful conversations and a surprising lack of slop (except for coffee*)☕️
*jk I am actually ok with conference coffee
📣 The BlackboxNLP 2026 program is live!
Join us on October 29th in Room P3 at @emnlpmeeting for keynotes by @sheridan_feucht , @iatitov & @mhahn29, six oral presentations, and two poster sessions.
Full schedule: blackboxnlp.github.io/2026/#…
5
26
950
Martin Tutek retweeted
Check out @YNikankin ‘s work on reasoning step importance today at the last poster session (imperial ballroom, poster 70) at #colm2026
nitter.siddz.com/YNikankin/status/20665…
I'll be presenting our work on internal representations of step importance in reasoning models (Session 6, Imperial#70), come by to chat about interp in reasoning and other large-scale setups
5
21
1,115
Martin Tutek retweeted
FWIW this is the most ambitious benchmark I've co-created!
𝗞𝗡𝗢𝗪𝗦 jointly & reliably evaluates agents on:
1) live web search,
2) use of common web productivity tools,
3) visual/spatial understanding;
programmatically where possible and by LLM judges where needed, more 👇
A useful computer-use agent doesn't just find information. It turns it into something you can share.
Introducing KNOWS: agents research the live web, then build a real Google Doc, Sheet or Slide deck in the browser.
2
28
4,188
Martin Tutek retweeted
nitter.siddz.com/YNikankin/status/20665…
I'll be presenting our work on internal representations of step importance in reasoning models (Session 6, Imperial#70), come by to chat about interp in reasoning and other large-scale setups
Do reasoning models internally represent abstract properties of their own chain of thought (such as "which steps are important"), while not surfacing these properties in their tokens?
7
17
1,642
Martin Tutek retweeted
A useful computer-use agent doesn't just find information. It turns it into something you can share.
Introducing KNOWS: agents research the live web, then build a real Google Doc, Sheet or Slide deck in the browser.
2
6
21
2,729
Martin Tutek retweeted
I'm in SF for the upcoming week for @COLM_conf & stuff.
Looking forward to meeting people & talking about anything (especially any form of interp and ai4discovery);
Also looking forward to losing this vibe-based first-visit-to-sf bingo
4
4
46
1,324
Martin Tutek retweeted
📢I'm hiring a PhD student to work on interpretability for Protein Language Models!
🗓️There are now two weeks left to apply, find out more on the LION Lab website: lionlabnlp.github.io/jobs/ai…
🦁LION Lab is hiring!
🧑🔬One fully funded PhD student (TVL-13 100%) for 3 years
🧪Topic: Interpretability for Protein Language Models
👥Advised by @LAWeissweiler together with Clara Schoeder
🌍Leipzig, Germany
🔗Apply by Oct 15 lionlabnlp.github.io/jobs/ai…
Please share!
#NLProc #NLP
5
15
892