New in HF tokenizers this week: Special tokens take 1,000× less memory! gpt-oss has 21 added tokens, ids 199,998 to 200,018 We used to keep them in a list with one slot per token id: 200,019 slots, 21 used 🙃 Next v1 rc soon. Follow me to be the first to pip install it 🚀

I hear more and more about the growing need for scientific data to be AI-ready. But what is it really about? Most available scientific data is stored as piles of files. This works well when you know what you are looking for. But it does not when you don't, and an agent has to search through all the files across multiple datasets. I think this is one of the biggest blockers for AI in science today. I'd love to hear how scientific data not being AI ready affects your field and your research, and we could improve it 🚀 🤗 I learned a lot reading this blog post from @EarthmoverHQ : earthmover.io/blog/ai-ready-…

🌍 🧬 🌤️ AI for Science releases seem so technical and out of reach sometimes. I wrote a summary of this month’s releases on the Hub to make them understandable to everyone. I go over the 7 main open-source models and datasets released on @huggingface, from gene annotation to climate simulation. If you find it useful, let me know and I’ll make it a monthly thing 🤗

RFD4-Proteina on HuggingFace, both in Diffusers and interactive spaces to work with, try it out and let us know what you think!

Thousands of sequenced genomes have never been annotated. When a new organism is sequenced, some parts of it encode genes and others do not, and we need to find which is which. Thanks to @cgeorgiaw’s amazing work, the Carbon Annotation Database now covers 22,617 species, including many that had no annotation before. It comes from Carbon-A, an open AI model that reads the DNA directly and annotates the genes in it. You can explore the database here: huggingface.co/spaces/Huggin…

If you want to learn more about AI for weather forecasting and how to train a model to improve a forecast's resolution in an hour using only 1 GPU, read my new blog post: huggingface.co/blog/EmmaScha… Looking forward to hearing some feedback on it 🤗 🌍

If you’re trying to keep up with what we're doing at HF, this is a really good starting list! and of course, don't forget @ariG23498 🤗 🤩

AI weather models see the world on a grid. One cell over the Alps says 24°C, but inside it, the peaks are 12°C and the valleys 36°C. Big labs fix this by building bigger, higher-resolution models. We tried something cheaper. Keep the global model, and train a small model on top that learns how mountains, valleys and coasts shape the local weather. We trained it in 1 hour on a single GPU. It takes forecasts over France from ~28 km to 5.5 km, and gets closer to what weather stations actually measure, especially in the mountains. You can read more about how we trained the model on our blog post: huggingface.co/blog/EmmaScha… and try the model yourself on our demo space: huggingface.co/spaces/EmmaSc…

OlmoEarth v1.3 is now on @huggingface 🤗 🌎 It can create embeddings for the whole planet at 10-meter resolution, which can be used for: 🔸 Mapping crops and land cover 🔸 Tracking change over time (mining, deforestation) 🔸 Exploring places on Earth Congrats to the @allen_ai team on this super exciting new release!

Today we're releasing Carbon-X, a 14B model that brings DNA and natural language together for the first time 🧬 1. Beats every open language model we tested across DNA benchmarks. Within a point of GPT-6.1 Sol at >1,000× lower inference cost. 2. Mid-trained on 300B tokens of interleaved DNA and text, followed by SFT (without distillation!) and RL. 3. A new hybrid tokenizer encodes DNA using ~3× fewer tokens than BPE. 4. Gives other models a DNA specialist subagent: Kimi K3 gains 11.1 points when it consults Carbon-X. 5. Runs locally on a laptop with llama.cpp. Apache 2.0. Your DNA stays on your hardware. Model: huggingface.co/HuggingFaceBi… Blog: huggingface.co/spaces/Huggin…

🧬 🤗 Carbon-X, a new model for DNA understanding and generation developed in collaboration with @huggingface, is now available! It combines DNA and language, and can tell you what a sequence is, what a mutation does, or write new DNA for you. And it’s fully open source! Congrats to the team for making such a contribution to AI for science 🤩

This semester I’m teaching "Fundamentals of LLMs" at the MVA master in Paris 🎓 I’ve turned my 1st lecture, "NLP before LLMs", into a blog post that goes from simple word counts all the way to attention. The approach is learning by doing 💪 : each step comes with hands-on exercises inspired by key research papers. Why practice by hand when an AI agent can write the code for you ? Precisely because it can ! When code becomes easy and cheap, understanding is what matters, and you will build it by doing the work yourself. Have fun learning by doing ! 👉 huggingface.co/spaces/dlouap… (Next up: Tokenizers. Stay tuned !)

Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30

🧬 🤗 Nvidia just released a new protein design model, RFD4-Proteina, on @huggingface! RFD4-Proteina has a thinking setting (like an LLM). At high thinking, it produces 76.40 usable designs per GPU-hour on a benchmark of 24 binding targets, where the next best method managed 13.2. You can design protein sequences and all-atom structures together, conditioned on the molecular context you want to preserve: a binding target, an enzyme motif, a ligand, or selected structural features. Thanks @nvidia and University of Washington's Institute for Protein Design for making the model open source and available on the Hub! 🧬 Try the model on the demo space: huggingface.co/spaces/nvidia… 🤗 Model: huggingface.co/nvidia/RFD4-P… 📖 Blog post: huggingface.co/blog/nvidia/r…

Antibiotic resistance killed over a million people in 2021. Phages could be part of the answer, but matching the right phage to the right strain has always been slow, manual work. Phagos just released CAPHARD on @huggingface! You can upload a bacterial genome and the model ranks which phages are most likely to eliminate it. 🤗 Demo space: huggingface.co/spaces/phagos…