📣 Excited to announce our oral presentation at #ICLR! LLMs capture rich semantic structure, as evidenced by their strong performance across a wide range of language and reasoning tasks. But Sparse Autoencoders (SAEs), a popular interpretability tool, mostly learn local, noisy, token-level features when applied to LLMs (e.g., hundreds of features for the word “the”). So why aren’t SAEs finding that rich semantic structure? 👉 Because they ignore the sequential nature of language. We introduce Temporal SAEs to bridge this gap. arxiv.org/abs/2511.05541 🧵 [1/N]

Can GenAI agents🤖 manage a supply chain? Lessons from the classical beer game🍺 (1/n) @davidsimchilevi @FlavioCalmon @AndreCalmon

‼️🕚New paper alert with @ushabhalla_: Leveraging the Sequential Nature of Language for Interpretability (openreview.net/pdf?id=hgPf1k…)! 1/n

How can we improve LLMs without any additional training? 🤔 The standard playbook is using Best-of-N: generate N responses ➡️ use a reward model to score them ➡️ pick the best 🏆 More responses = better results... right? Well, not exactly. You might be reward hacking! Instead, you should hedge! 🎯

Can we use coding-theory, heavy-tailed distributions, and optimal-transport to create 𝘇𝗲𝗿𝗼-𝗱𝗶𝘀𝘁𝗼𝗿𝘁𝗶𝗼𝗻, 𝗲𝗮𝘀𝘆 𝘁𝗼 𝘂𝘀𝗲, 𝘄𝗮𝘁𝗲𝗿𝗺𝗮𝗿𝗸𝘀 𝗳𝗼𝗿 𝗟𝗟𝗠𝘀? We show they can — and the result is pretty exciting! 🎉 🧵 (1/n)

Without its international students, Harvard is not Harvard. hrvd.me/IntStudents25t

Happy to share we received best paper at NENLP workshop at Yale 🥳🥳! tldr: Current alignment methods give excessive discretion to annotators in defining what good behavior means. This means we don't know what we are aligning to ‼️ We formalize discretion in alignment and propose mechanisms for data curators & model developers to monitor for it. Paper link below ⬇️

[1/x] 🚀 We're excited to share our latest work on improving inference-time efficiency for LLMs through KV cache quantization---a key step toward making long-context reasoning more scalable and memory-efficient.

9/n Full paper here: 🔗 arxiv.org/abs/2502.10441. Huge thanks to my amazing team of co-authors: @maartenbuyl, @hskhalaf, @claudiomverdun, @caiocvm, and @FlavioCalmon. Work done @hseas!

AI is built to “be helpful” or “avoid harm”, but which principles should it prioritize and when? We call this alignment discretion. As Asimov's stories show: balancing principles for AI behavior is tricky. In fact, we find that AI has its own set of priorities (comic @xkcd)👇

The standard practice in differential privacy of targeting ε at small δ is extremely lossy for interpreting the level of privacy protection. In practice (e.g., for DP-SGD), we can do much better! We show how in the #NeurIPS2024 paper: arxiv.org/abs/2407.02191 Short summary👇

Imagine an all-powerful AI with any ideology you don't agree with! Super proud of this work, where we show that every LLM reflects a different ideological worldview, which should worry everyone.

Finally, I am pleased to announce 🪢Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)🪢 Joint work with Usha Bhalla, as well as @Suuraj, @FlavioCalmon, and @hima_lakkaraju, which was just accepted to NeurIPS 2024! Check out the paper here: arxiv.org/abs/2402.10376

Part 2 of my 2024 publication tweets! Please welcome Multi-group Proportional Representation, a novel metric for measuring representation in image generation and retrieval. This work was recently accepted at @NeurIPSConf 2024. (1/n)

First up, how do various aspects of trustworthy machine learning interact? Can we expect a production ML system to satisfy all regulatory requirements of fairness, privacy, and interpretability simultaneously when past research generally focuses on one component at a time? (1/n)

This week, I spoke on the panel “AI, Rights, and Democracy” at the Brazilian Supreme Court. Thank you @STF_oficial for the invitation. It was an incredible experience! See my talk (in pt-br) here: piped.video/mNkZ_Aw2tFs&t=4002s

Back home from FAccT - I thankful for the work our community is doing & the values it stands for. Serving it has been a labor of love for me & I am beyond grateful to have done so this year along my truly wonderful program co-chairs & human beings @mikarv @RDBinns @FlavioCalmon