OpenAI just announced a breakthrough in matrix multiplication - the critical building block of countless algorithms, including how all of the AI works. Like, if you were wondering what
@nvidia's chips are doing - this is what they're doing.
We could argue that slashing the theoretical efficiency exponent from 2.371339 to 2.25 for square matrices represents a tectonic shift in both computer science and the economics of artificial intelligence.
Over lunch, my colleague at
@pathway_com just extended the result to rectangular matrices, slashing the exponent to ω=2 for the other most used matrix shape (n × √n matrix by a √n × n matrix). Here is the paper (in replies). You're welcome.
What beautiful times we live in, when you can get results like these while chewing on your burrito. Cheers to Przemysław Uznański (
@PrzemekUzn25621)! 🍻
Human mathematicians can still be somewhat useful ;-)
*The remarkable part is the equality: two is the smallest possible exponent, matching the lower bound imposed by the n^2-entry output. For this matrix shape, this result settles the exponent exactly, once and for all.