PhD Student @MIT_CSAIL RS Intern @SakanaAILabs RL, Meta-Learning, Emergence, Open-Endedness, ALife

We never really knew how to train nonlinear RNNs well… BPTT struggled with vanishing grads (no long-range memory) and sequential rollout (hard to parallelizable). What if instead an oracle told us the optimal memory state m_t at each step? Then the RNN could do one-step supervised learning on (m_t, x_{t+1}) → m_{t+1} labels. We call this Supervised Memory Training (SMT): a replacement for BPTT that trains RNNs without unrolling them. SMT is time-parallelizable and solves vanishing gradients. Website: akarshkumar.com/smt/ arXiv: arxiv.org/abs/2606.06479
19
120
812
189,240
Akarsh Kumar retweeted
Stephen's Ruliobiology is a great and minimalistic way to show the beauty of cellular automata and evolution. Gave a talk about it to my family this weekend. Interactive slides to play with: znah.net/ruliobio
Can we finally make a theory for Darwin's "Endless Forms Most Beautiful and Most Wonderful"? Introducing ruliobiology as a path to general theory in biology, starting with foundations for a theory of biological form writings.stephenwolfram.com/…
5
25
237
8,963
Akarsh Kumar retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains? We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer. In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans. 📝 xingjianbai.com/reactive-opt… 💻 Try it on your task: github.com/xingjian-bai/opal 🧵
73
200
1,521
210,367
Akarsh Kumar retweeted
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine. • Best-in-class fast local coding • Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark • Built from scratch in under 4 months Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
636
894
8,388
3,431,733
Akarsh Kumar retweeted
Had a great time chatting with @yacinelearning about VPO! Looking forward to hearing people’s thoughts on the discussion we had
one elegant post-training method that caught my attention in 2026 is VPO who in a nutshell post-train the models to output diverse set of good answers. in this 2h odyssey we cover: - why diversity is important - a magic trick - reward shaping for exploration - a top tier figure
1
6
46
2,173
Akarsh Kumar retweeted
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair. It even works when the images and the captions come from different datasets. Project page: dominik-schnaus.github.io/un… ⬇️
85
359
2,411
244,602
Akarsh Kumar retweeted
With agentic AIs now becoming more life-like (e.g. including self-replication, self-improvement, open-endedness and collective intelligence), the field of Artificial Life will become one of the most important research directions in the years to come. If you want to learn more about it, good place to start is our 2024 paper "From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models"
7
23
122
5,336
Akarsh Kumar retweeted
I wrote a blog post detailing the writing approach I took while working on my recent paper: sophielwang.com/blog/paper-w… I describe the behind the scenes for how this paper came to be, why and how I storyboarded figures before running experiments, and some figure design tips! :)
Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog: sophielwang.com/cues
34
157
1,509
115,936
Akarsh Kumar retweeted
“The field of artificial life is a very long-term bet on intelligence.” 🐙 Our former intern Akarsh Kumar recently joined @MLStreetTalk to discuss the intersection of evolution and automated search for artificial life. He dives into our Digital Red Queen research, exploring open-ended evolutionary arms races in Core War using LLMs. This perspective on intelligence through artificial life is central to how we think about the future of agentic AI.
"Evolution is anything but random." This is INSANE... @akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.
4
27
163
18,246
Check out this conversation on the field of artificial life and the underappreciated power of evolutionary algorithms! With all these new autonomous AIs running around, ideas from ALife are going to become more and more important. Thanks @MLStreetTalk @ecsquendor for having me!
"Evolution is anything but random." This is INSANE... @akarshkumar0101 (@MIT_CSAIL, @SakanaAILabs) on artificial life: searching the space of possible universes with foundation models, why evolved networks find structure SGD misses (with @kenneth0stanley), and LLMs evolving Core War programs.
5
14
119
3,774
Akarsh Kumar retweeted
AI agents have advanced so much in the last few years that they have started to resemble a new kind of lifeform. This project takes that prospect seriously, and begins our study of the “ethology of AI agents.”
Is this machine playing? 🏀🪁🛝 We left Eko, an embodied coding agent, on an island for 30 hours with no task or goal, studying it like ethologists study a new species Eko explored, built, climbed, made games, and learned completely new techniques (1/9) machineplaying.org
7
20
223
20,644
Everyone is worried about what research to do in the age of AGI. I think this ethology of agents perspective is one of the most fascinating topics right now... More and more AI research is going to resemble natural science rather than engineering new systems. Great work by @nacloos and @noranta4 !!
Is this machine playing? 🏀🪁🛝 We left Eko, an embodied coding agent, on an island for 30 hours with no task or goal, studying it like ethologists study a new species Eko explored, built, climbed, made games, and learned completely new techniques (1/9) machineplaying.org
3
2
56
3,643
Akarsh Kumar retweeted
Can “chicken” make a base model reason? 🐔 Yes! Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too. blog: sophielwang.com/cues
48
80
616
195,429
Akarsh Kumar retweeted
Linear attention is arguably the most naive RNN, but still massively outperforms traditional RNNs by maintaining a matrix state. So... what if we use a (triadic) outer product of three vectors and maintain a three-dimensional state? Introducing: Triadic Linear Attention 🧵
32
103
862
66,099
Akarsh Kumar retweeted
Replying to @Pontifex
Really? 2008: "Driven by Compression Progress: A Simple Principle Explains Essential Aspects of Subjective Beauty, Novelty, Surprise, Interestingness, Attention, Curiosity, Creativity, Art, Science, Music, Jokes" arxiv.org/abs/0812.4360 Tweet:
81
87
1,090
149,648
In this work, we formulate forgetting as a geometric distribution shift on a layer's inputs. The core insight is to update the function behavior only locally at the new data points. Since gradient learning can often perform unpredictable global function changes, we gate the update using classical non-gradient techniques to explicitly enforce locality. Great work led by @abk_tau!
(1/7) Humans can acquire many capabilities without (catastrophically) forgetting previous ones. Why can't neural networks do the same? Local Support Learning (LSL) takes us a step closer towards this goal. In LLMs of up to 7B parameters, LSL augments gradient-based training to retain prior capabilities while learning new tasks at full capacity. Notably, it does so without access to prior data. Key idea: casting forgetting as a geometric problem, and optimizing for the worst case. Paper: arxiv.org/abs/2610.02126 Website and code: assafbk.github.io/lsl
1
3
72
5,279
Akarsh Kumar retweeted
There’s lots of buzz around agentic harnesses for robots, particularly for long horizon tasks that require complex reasoning and memory. But what will it really take to turn a reasoning agent into a reactive, reliable and low-latency robot policy? In our new paper, workspace models, we design a new memory architecture that acts as a latent harness for stronger reasoning models. Here’s why we think this might be the way forward: (1/9)
8
58
214
35,491
Akarsh Kumar retweeted
Our Neuroevolution textbook is finally in print! Free online edition: neuroevolutionbook.com/ Pre-order: a.co/d/06DpkH9h I am incredibly grateful to my co-authors Sebastian Risi, Yujin Tang, and Risto Miikkulainen for making this happen. Neuroevolution is a subject very dear to my heart. It is the field that convinced me that nature has already figured out how to build intelligence: through evolution, collective behavior, and adaptation under constraints. This idea, that intelligence emerges from evolution operating under constraints rather than unlimited resources, is what eventually led me to founding Sakana AI here in Japan. The concepts in this book about open-ended creativity and self-organizing systems are exactly what we build at @SakanaAILabs. Our name and logo are inspired by schools of fish moving together, adapting as one. It is the core philosophy of what we build. This book captures the theoretical foundations of that belief.
36
119
721
72,507
Extremely cool algorithm by @ion_barrel to get x-ray segmentation supervision!
X-rays are everywhere in medicine, but extracting general, quantitative anatomical information from them remains remarkably difficult. Announcing FleXray: an open-source, open-weight model for zero-shot anatomical segmentation across the human body—from different regions, viewing angles, and acquisition settings. FleXray learns from large-scale synthetic supervision generated from 3D anatomy, then transfers directly to real clinical X-rays. Try it now on your own X-rays in our in-browser demo! 🧵 1/N — project page, demo, weights, code & paper below ↓
1
7
1,123
Akarsh Kumar retweeted
I continue to put my money where my mouth is when it comes to research freedom, this is me doubling down. Come join us and you'll have the time, resources and talented collaboratiors to work on that mad idea that's being rattling around the back of your head for years!!!
Introducing the Sakana AI Frontier Intelligence Group 🪷 sakana.ai/frontier-intellige… Current AI systems are incredibly capable, but is intelligence “solved”? And if not, what’s missing? At Sakana AI’s Frontier Intelligence Group (FIG), we believe that there are still breakthroughs to be made in AI. The Transformer and language modeling may be incredibly powerful, but it doesn’t mean that better alternatives don’t exist. Natural intelligence still beats artificial intelligence across many dimensions. Agents lack the deep insights and creativity of humans. Individual models require far more data than the brain to learn robustly, and require far more energy to run. If we set these as targets, what kinds of AI systems could we develop? Research at FIG has sought to address the gaps between natural and artificial intelligence. Here are some of our works, and the fundamental research questions that motivated them: • Continuous Thought Machines: How can we improve information processing by leveraging temporal dynamics? • Augmented Lagrangian Predictive Coding: How can local learning solve multilayer credit assignment? • Sparser, Faster, Lighter Transformer Models: How can we massively increase data efficiency and generalization? • The AI Picbreeder Experiment: How can we make artificial open-ended systems? • Smart Cellular Bricks: How can physical systems achieve collective intelligence and self-repair without a central brain? We hope that this encourages other researchers to also explore different paradigms, and take a leap of faith with us. After all, in the words of a dear friend of ours, “greatness cannot be planned”.
1
13
175
13,601