Postdoc at @ETH_en. Formerly, PhD student at @Cambridge_Uni.
Brasília, Brazil
- Tweets 1,176
- Following 327
- Followers 1,864
- Likes 2,428
Pinned Tweet
checkout our new paper about the superficial alignment hypothesis :) we use algorithmic information theory to formalise this hypothesis, we unify prior work on the topic, and show how post-training affects it!
follow @tvergarabrowne for more great work like this in the future!
first paper of the phd 🥳
the Superficial Alignment Hypothesis (SAH) argues that pre-training adds most of the knowledge to a model, and post-training merely surfaces it.
however, this hypothesis has lacked a precise definition. we fix this.
3
25
4,242
Tiago Pimentel retweeted
🚨 New paper!
What if LLMs could reason like we do when solving Sudoku: keep a few candidates per cell and iteratively narrow them down?
Introducing Binary Discrete Diffusion for language and reasoning tasks: it denoises sets of candidate tokens using binary representations! 🧵
12
39
189
10,042
Happening soon :) tokenization-workshop.github…
Announcing our first invited speaker at TokShop!
We're thrilled to welcome Tiago Pimentel @tpimentelms (ETH Zürich) for a talk on:
"How much does tokenisation impact language models?"
🧵
3
17
797
Tiago Pimentel retweeted
Check out the paper on @askalphaxiv !
alphaxiv.org/abs/2609.tokeni…
1
5
16
1,243
If you've been meaning to dive into tokenisation, we have a beefy new survey for you :) Led by @marco_computers and put together by an amazing team. Check it out 👇
🚨 [Token][ization] Paper Alert 🚨
Tokenization is a wildly understudied area of language modeling despite it having effects across all of NLP. Over the past ~8 months, 32 (!) tokenizer researchers put together the most comprehensive survey of the field. Check it out!
2
30
1,051
Tiago Pimentel retweeted
Please support our call asking frontier LLM providers to share what they’re doing for AI safety, what works and what doesn’t. Many sharp people want to help close the capability–safety gap. Transparency will help them see where their work is needed most.
make-safety-open.github.io
1
29
130
7,809
Tiago Pimentel retweeted
🎙️ Excited to announce our second invited speaker: Amir Zamir @zamir_ar, Assistant Professor at EPFL!
Talk: "(Multimodal) Flexible-Length Tokenization"
Looking forward to a talk that pushes tokenization beyond text 👀
7
12
424
Why is BPE so good: the compression objective or the step-by-step merging? 🤔 Standard tokeniser comparisons (eg, BPE ⚔️ UnigramLM) conflate the two. Our #emnlp2026 paper separates tokenisation objective from search procedure and shows BPE's merging strategy is what matters 🧵
ALT Paper title, diagram with 2x2 diagram with tokenisation objectives and search procedure, and a plot showing that bottom-up methods achieve better bits-per-byte than top-down.
3
3
31
1,518
When measuring grammaticality performance (in BLiMP), though, no design choice performs consistently better. Better compression thus does not seem to buy you better grammaticality.
1
1
123
We hope future work in tokenisation will ask two questions: what should we optimise, and how should we search? Our results suggest the second also deserves some love!
Paper: arxiv.org/abs/2609.19145
Code: github.com/Ahmetcanyvz/comp-…
2
125
First rule of academic writing: don't assume readers can think. Spell everything out #iclr2027
15
688
Tiago Pimentel retweeted
Why does the induction head emerge abruptly? It's because of the geometry of the loss landscape: an almost flat plateau around initialization, followed by a sharp drop.
We study this in more detail in this paper you might find interesting:
arxiv.org/abs/2511.01033
1
1
28
1,274
Tiago Pimentel retweeted
Announcing our first invited speaker at TokShop!
We're thrilled to welcome Tiago Pimentel @tpimentelms (ETH Zürich) for a talk on:
"How much does tokenisation impact language models?"
🧵
1
8
34
2,324
Tiago Pimentel retweeted
🐝New preprint🐝
If you want open source speech technology that's actually optimized for your target language, BuzzASR might be your best option! All models on Huggingface 🤗
Congrats to @realshivamsingh on an amazing first paper!!
Appearing in EMNLP Findings.
1/
Every language is unique, but the big AI companies release a single model to cover dozens of languages at once. BuzzASR 🐝 takes a different approach: We release a swarm of 102 monolingual speech recognition models, beating Whisper on 77 languages and achieving SOTA on 27.
1
2
21
1,745
Tiago Pimentel retweeted
This kind of ongoing accountability is going to be open up a lot of really valuable possibilities for safety, and I'd love to see similar things elsewhere:
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
13
18
219
38,315
Definitely go work with Edoardo if you're searching for a position :)
I am looking for 1 postdoc and 2 fully funded PhDs at Imperial @ICComputing to join my @ERC_Research project AToM ⚛
We aim to build models that learn to compress what they perceive and remember, for permanent memory, longer horizons, and >10x speedups
ducdauge.github.io/atom/
3
490
Tiago Pimentel retweeted
You can train your own Parity-aware BPE tokenizer in the HuggingFace library! See the docs for an example :) huggingface.co/docs/tokenize…
Congrats to @clara__meister for getting the Parity-Aware BPE tokenizer merged into Hugging Face tokenizers!
Learn more about it here: alphaxiv.org/abs/2508.04796
Use it like this:
from tokenizers.pre_tokenizers import ByteLevel
from tokenizers.trainers import ParityBpeTrainer
tokenizer = Tokenizer(BPE())
tokenizer.pre_tokenizer = ByteLevel()
# Plain text files: one iterator per language
def lines(path):
with open(path) as f:
yield from f
trainer = ParityBpeTrainer(num_merges=32000, variant="base")
trainer.train_from_iterator(
tokenizer,
train_iterators=[lines("en.txt"), lines("de.txt"), lines("zh.txt")],
dev_iterators=[lines("en_dev.txt"), lines("de_dev.txt"), lines("zh_dev.txt")],
)
tokenizer.save("parity_bpe_tokenizer.json")
4
20
1,683
Tiago Pimentel retweeted
I am looking for 1 postdoc and 2 fully funded PhDs at Imperial @ICComputing to join my @ERC_Research project AToM ⚛
We aim to build models that learn to compress what they perceive and remember, for permanent memory, longer horizons, and >10x speedups
ducdauge.github.io/atom/
5
54
202
14,852
Tiago Pimentel retweeted
A new, improved version of our language identification system "UniLID" is out! It's more accurate (see paper) and much easier to use: training your own model or adding a language to the existing one now takes a few lines. Try it out; feedback welcome :)
github.com/Ahmetcanyvz/UNILI…
Happy to announce that UniLID was accepted to ICML 2026! Code and model are publically available, if you'd like to try them out :)
1
1
9
1,165