Philosopher and Engineer. Truth and Beauty. Building the future of computing at @uscompco. Algorithmic dark matter, other things
🇺🇸
- Tweets 15,280
- Following 606
- Followers 5,547
- Likes 44,076
Charles Rosenbauer retweeted
When I was 14, I started learning kanji and immediately complained about stroke order.
context: Japanese uses adapted Chinese characters called kanji. Kids in Japan are taught to write each one in an exact stroke order, not just to make it look right.
"Why does it matter if it looks the same at the end?"
My mom was a math teacher, so she was used to me complaining about things like this and usually had an answer. She isn't Japanese, though, so this one stumped her.
I'd started Japanese at 11 but only knew hiragana and katakana. My public high school in Michigan offered Japanese, and I wanted to test into Japanese 2. To do that, I had to memorize 50 kanji by the end of the week. I had about two days. 40 would've gotten me a B, which was fine, but I wanted all 50.
So my mom stayed up with me until midnight with flashcards. I got 100% on Friday's test. Then it came time to actually learn to write them, and I froze.
I told her stroke order was stupid and a waste of time. "This is America. Surely we can do this more efficiently." She said I was on my own.
I tried everything to get around it. I drilled radicals on my own, looked for some cool calligraphy style to copy. Eventually I realized everything was just easier once I learned the stroke order.
This came up again recently! I was complaining to a friend who studied math at a high level that we're taught to memorize formulas without learning what math is actually doing. He said he felt the same way until grad school, when he realized you have to learn the formulas first because everything builds on them. I immediately thought of kanji. He's a native English speaker who learned Chinese characters for Mandarin, so he knew exactly what I meant.
It reminds me of what the neuroscientist Nikolai Bernstein called the degrees of freedom problem. Your body has too many possible ways to make any movement, so beginners lock most of them down. Once the movement becomes automatic, you can loosen up and get creative. I think rules like stroke order or memorized formulas do something similar for your mind: they narrow your options until the basics are automatic, and then you have room to actually think.
These days I barely handwrite kanji. Most of my Japanese is digital. But when I do write by hand, it's the kanji I learned back then, in the exact stroke order I learned at 14. A lot of processes seem inefficient or silly until you understand why they're there.
Memorization has an unfairly negative reputation. I recently spent a few hours memorizing the names of all the countries in the world and it's such a clear improvement in how well I can understand and reason about other topics. Memorization is scaffolding for more complex ideas
4
15
718
When we look back to the ancient world, we often have a distaste for the degree of chimeric fantasy involved. Sphynxes? Dragons? Centaurs? Clearly nonsense with no value but confusion!
Except that if you look at memorization tricks, one thing that stands out is that strange imagery is one of the most potent methods. If something is so strange it violates your brain's learned latent space, it has to dedicate new neurons to encoding it. Attaching a memory to vivid and strange imagery makes it stick better than almost any other advice.
Poetry is also looked down upon somewhat. Merely pretty words? Goofy rhymes? Alliteration? Meter? Whimsical nonsense!
Except no, these too make things stick. There's a reason why so much music continues rhyme to this day. This extra repetition and structure is effectively a form of error correction that makes large volumes of text much easier to memorize. Think about how many songs you know by heart versus how few pages of your favorite book or chapters of essays you can do anything but paraphrase.
When we look back to ancient cultures which would memorize the entire Iliad or Odyssey and pass them down for centuries before writing them down, we should consider that perhaps they were onto something.
I suspect that what ancient people were doing with all of this was deliberately making the knowledge of their world much easier to memorize! We likely would benefit from working to revive and reimagine such approaches. It may not make the world fundamentally easier to comprehend, but it would make it much easier and faster to fill one's mind more of its details.
Memorization has an unfairly negative reputation. I recently spent a few hours memorizing the names of all the countries in the world and it's such a clear improvement in how well I can understand and reason about other topics. Memorization is scaffolding for more complex ideas
2
1
8
431
I'm surprised that people don't talk more about the "Guess 2/3 of the average" game. One of the places in game theory where more rationality actually loses the game.
en.wikipedia.org/wiki/Guess_…
6
3
328
59,342
Charles Rosenbauer retweeted
So, what came of all the recent cyber exploits (PHP, etc.)? Did the world end? Did the internet go away?
No, a lot of people did a lot of patching, and that was that.
(There will still be exploits. And the world will go on.)
3
2
8
643
So some elaboration on this -
ETH is a much stronger statement than P≠NP. It's a bit surprising that it's not true but it doesn't really break much.
The bounds are technically sub-exponential, but "2^n / complicated log sqrt expression" is still very close to exponential.
Given how well SAT solvers work in practice, and have for decades, some slightly tighter bounds aren't extremely surprising. There are SAT solvers which have been proven to be <O(2^n) for a long time, just with exponents smaller than 2.
What this amounts to is that for any function O(k^n), there's some algorithm that can do better than any value k, if you make n absurdly big. That's a cute result, but we already knew that many NP problems are easier to solve in practice than in theory, and n really doesn't need to be very big for this to be a pretty meaningless statement in practice.
Ultimately though, and this is something I have some unpublished work on myself that I need to get out there soon, the difficulty of NP problems really has dramatically more to do with the structure of the problem than it does the number of variables. You can have small NP problems that are almost as hard as brute force, and you can have others with millions of variables that can be solved quickly. With NP problems in particular, it's clear that the tools we're applying are simply dramatically too primitive to understand what's actually going on.
The computer science collaboration will understand and explain a result concerning the Exponential Time Hypothesis (ETH), a longstanding conjecture about the computational complexity of Boolean formula satisfiability, a problem deeply related to the P vs. NP problem.
Surprisingly, our system has recently proved that the ETH is false, a result that was subsequently verified in Lean.
The refutation of the ETH reopens questions about how much faster these tasks could become, and may help guide the search for better algorithms. Finding such algorithms, and turning theoretical advances into practical improvements, will require significant further research.
Our system found its natural language disproof for the ETH we reached out to to @rrwilliams. Ryan will form a collaboration with Russell Impagliazzo and Ramamohan Paturi to verify and shepherd this result. Impagliazzo and Paturi, along with Francis Zane, are the original proposers of the Exponential Time Hypothesis, and Williams has refuted strong versions of the ETH, used ETH for hardness results, and connected algorithms and lower bounds.
This group asked us to make all raw data of the results public to facilitate independent verification by the community, which we have done by posting the natural language proof and Lean formalization to our GitHub repo. The group also invites the community to help organize an open examination of the ETH claim and to discuss the broader questions that AI-generated breakthroughs raise about verification, access, credit, and the future of research.
github.com/openai/math/blob/…
20
39
425
58,657
Ultimately though I think the whole field is confused. P and NP are really about how the expressivity of FOL changes if you add an existential quantifier. Bolting polynomial and exponent bounds onto this is kind of a mathematical non-sequitur and I think people will increasingly find that it's not a sound way to address these questions at all.
In the early days we had the Automata Theory hierarchy, from Combinatorial Logic → Finite State Machines → Pushdown Automata → Turing Machines. You can prove some stuff about this, there's legitimate work that was done, but eventually in the 70s people switched to an entirely different framework, especially after Karp's work on NP completeness. The field is only a few decades old, the first ideas we have aren't likely to be the best, I expect Complexity Theory is eventually going to need to switch to a completely new foundation once again.
3
2
32
2,946
Trying to put Linux on a new laptop I got and the hoops I have to jump through to deal with all the "BIOS security" nonsense makes me really wish that a bunch of huge security vulnerabilities completely bankrupts Microsoft in the next couple years. They deserve it.
2
1
39
1,499
Charles Rosenbauer retweeted
My mathematician daughter was browsing in a used bookstore this weekend ... and found this:
writings.stephenwolfram.com/…
100
165
2,537
189,158
It'll certainly be a few years at least before I'm having any children but I'm definitely very interested in understanding what the future of education tech might look like and finding ways to train mentat kids.
LLMs are shifting a lot of SW econ, and lots of compute is and hardware is changing right now. But what people expect that will look like and what it will actually look like a few years from now will probably be very different.
I'd definitely be very interested in having the CPUs I'm building be a part of a new computing ecosystem rebuilt from scratch, and doing so in a way designed to be better for the humans using it.
E-ink and stylus stuff is neat. I've written before about how we need more multisensory computing, including richer audio interactions. I really don't like the idea of computers and AI being simply an excuse for people to shut their brains off.
I think a very serious project to understand what a better child-friendly computer look like, a genuine bicycle for the mind that produces Mentat kids rather than ADHD iPad kids, would be a very valuable project that would yield huge returns in time for a lot of computing and human-computer interaction.
4
1
16
781
I find a lot of wisdom in the story in I Kings of Solomon's wish.
Solomon finds that he is about to become king. He makes a wish to God, he asks not to become a particularly wealthy or long-lived king, he just wants to become a king who is wise and knowledgeable of good and evil.
God comes to Solomon in a dream and tells him that He is very happy that Solomon would ask for such a thing, so much so that in addition to granting his wish and making him the wisest of all kings, He will also in time give Solomon the things he explicitly did not ask for; unparalleled wealth and a long life.
Short-sighted maxxing of obvious signals like status and money may have short term benefits, but in the long term it's ultimately inferior to seeking the deeper principles beneath the surface and doing them well.
people need to status max so much more
high status people saying "chasing status is bad" is such an effective way of keeping everyone below them
literally the only reason why you care about a person you have not met is because they've done something publicly approved
rant:
1
1
20
931
I still think LiFi is a good idea; transmitting data through MHz/GHz strobing of visible range LEDs.
Being visible light, it does not propagate through walls, with obvious security benefits.
It also would be very amenable to very local digital interactions. Passing files between two devices in the same room, for example.
Honestly, I suspect there's a lot of room for building information technology with increasingly localized properties. While it's hard to say exactly what it may look like, I would not be surprised if many resolutions to tensions between technology and social issues may lie somewhere in that direction.
6
23
1,341
Charles Rosenbauer retweeted
I have two polygons that tile the plane iff ZFC is consistent.
One has 5 edges and the other has 378,485,108,569,489,476,845,190 edges.
20
65
938
49,323