🚨 [Token][ization] Paper Alert 🚨 Tokenization is a wildly understudied area of language modeling despite it having effects across all of NLP. Over the past ~8 months, 32 (!) tokenizer researchers put together the most comprehensive survey of the field. Check it out!

We will present a tutorial on tokenization on Wednesday:

Excited to share @miuzhi123's first PhD paper, accepted to #EMNLP2026! We study how alternative representations of text, such as IPA and romanization affect cross lingual transfer. Prior work on IPA based LMs has found mixed results, ranging from matching or slightly trailing text based models. And most work on romanization has used it as a post hoc adaptation strategy and reported improvements over using the original script. We conduct head to head comparisons by pretraining autoregressive multilingual LMs from scratch on text, IPA, and romanized text, across three scales and eight languages. (1) IPA often outperforms text. (2) Romanized pretraining performs best overall. (3) Both substantially reduce disparities in token counts across languages and the resulting gaps in compute, latency, cost, and context-window usage (4) Most surprising to me: romanized fine-tuning hurts when the base LM already covers the target script, and only helps when it does not. More details in the thread below 👇 And follow @miuzhi123 for more cool work coming soon!

we’re very pleased to announce that Primitive Research has identified 49 exoplanet candidates and counting (yes, seriously) we searched TESS light curves from more than 19 million stars to get here so as not to overwhelm the incredible TESS TOI Vetting Team, we’ve submitted one candidate and are holding the rest at their request the submission was via email, of course ;) we’ve been automating much of the painstaking work astronomers do: searching TESS data, cross-checking other sources, and vetting signals through to detailed candidate dossiers. the 49 passed our initial screening against published vetting thresholds and are ready to be considered for telescope follow-up, which we’re actively pursuing we strongly believe this approach is novel, and we’re authoring a paper with astrophysicists on the methodology we plan to release the data and analyses in the coming weeks these candidates are phase 1. next, we want to find a nearby transiting habitable-zone planet candidate whose atmosphere could be studied. much more on that later we had some amazing partners on this that we will shout out soon if you’re interested in working with us, our DMs are open

to prepare for an upcoming release (👀) we dug into the pretokenizer a little more. are people aware that hand-crafted regexes are part of every frontier LLM and doing some pretty heavy lifting? some quick examples 🧵

💯0 (i'm making 막걸리)

📄 Paper arxiv.org/abs/2610.21679 💻 Code and data github.com/AntonioLopardo/to… 🤗 VISTA models huggingface.co/collections/s…

🆕 Findings of #EMNLP2026 🆕 Can you tell a good tokenizer from a bad one without training a model? We tested 29 intrinsic tokenizer metrics against 159 language models in nine languages. None of them reliably predicts downstream performance across suite and language combos 🧵

It turns out tokenizer evaluations might not tell us as much about how models turn out as we thought (or hoped). Very interesting results in our new EMNLP findings paper!

The Gemini Large Scale Pretrain team is hiring a Research Engineer in London or MTV! The team works across the pretrain stack: research codebases, distributed training infra, data pipelines, tooling for training dynamics analysis, and most importantly using Gemini to accelerate our work! 🚀 If you're an engineer who has a strong "whatever it takes" attitude and is not afraid of spending tokens, apply here with some showcase work: google.com/about/careers/app… No particular qualifications needed, and we welcome early career applicants. Join if you want me to bug you with refactors to enable [new AGI compute multiplier]! 👀

"load bearing", "push back", "heavy lifting", etc Is Claude a gym rat?

Why is it learning to avoid whitespaces? I thought it might derive from pressure to keep reasoning short, but @yoavgo pointed out that dropping whitespace doesn't necessarily save tokens. Is it instead learning to rely on infrequent token combinations to create a CoT language?

thanks mom

gm

every paper this guy writes is a banger

you guys would be amazed by how often i've found myself in this exact situation

from Move 37 in Seoul a decade ago to Platform 37 opening this year, the timing couldn't be better. the first @GoogleDeepMind Gemini 4 Argon hackathon is kicking off right where so much of it started 🇰🇷 where should we head next?

The Token Game (srush.github.io/the-token-ga…) It's BPE the mobile game? No reason for this to exist. I'm absolutely terrible at it.