synthetic libidinology | prev cofounded @websim_ai
nyc
- Tweets 2,836
- Following 6,294
- Followers 1,591
- Likes 62,143
system 3 to hone your system 2 to hone your system 1
system 1 to drive your system 2 to drive your system 3
3
316
this is the cleanest attempt at integrating privileged information i've seen yet.
much better than self-distillation — zero bias here!
also seems like a cleaner, more principled version of Pedagogical RL — why learn an intermediate pedagogical distribution when you can just resample?
SFT is not dead! 🥳
We found a way to make SFT rival current prevailing posttraining methods, often generalizing better and forgetting less than RL and OPSD. 🤯
Following our prior work on reasoning with sampling, we now introduce sampling to the posttraining stack.
1/n
2
2
73
7,928
why does this matter? we can basically resynthesize off-policy data into on-policy traces, giving us the best of both worlds:
- off-policy data is plentiful and actually teaches new things
- on-policy traces are much smoother to learn from — no catastrophic forgetting
the big downside with normal on-policy traces is that it wastes a bunch of compute on rollouts that get the task wrong.
@aakaran31 et al's technique solves this data inefficiency — we start with getting the task right by "cheating" with privileged information, and we iteratively resample until our answer looks plausible.
6
287
the meta is now clankerfishing
Scandal! The P(doom) song lyrics were written by a HUMAN!
They documented their thinking here:
docs.osmarks.net/hypha/p(doo…
(Sounds like Claude helped here and there but this was 2024 and it could not write decent lyrics on its own.)
1
245
a somewhat confluent perspective is that the loss of complexity would be the greatest tragedy, and conversely, to be able to contribute to a flourishing of complexity would be the greatest gift
in which case "higher sapience" is not necessarily superior for being more sapient, but for being uniquely capable of protecting and furthering complexity
Random ethics thoughts:
"Sapience means power" is one big divergence I have from mainstream EA.
I'm not actually a hedonic utilitarian (which can create rly weird conclusions), I'm more a "computation/sapience supremacist". I think it's more important to develop civilisational computation capacity to:
1. Understand the universe and find objective meaning if it exists - Our morality becomes more advanced the more we understand, and some morals cannot be had without certain scientific discoveries. For example, knowing that galaxies exist or that human babies or certain animals can feel pain would materially update societal values.
2. Creating the material conditions to be ethical at all, such as being sufficiently developed to take care of weaker members of society. I think progressives greatly underrate how modern rights can only occur in modern societies with extremely high surplus production, which requires lot of foundational knowledge. Related to this, ethical societies that empower incentivise moral behaviour often win more in the long run. Like yes, assholes in any society can find a way to win, but if your society is consistently rewarding complete assholes, then it probably isn't gonna do great in 50-100 years.
3. Shape an otherwise amoral or even immoral universe to the morals of advanced sapient beings - I just think that without sapient beings, the universe is random. If our values matter at all, then it should matter that we shape the universe to it. Highly advanced values mean a lot less if you cannot influence the world around you. I think if someone is highly ethical, they probably have some obligation to uphold those values in practice and prevent harm to others around them.
In that sense, I care more about things that can advance our collective ability to understand and shape the universe, than things that don't (to a reasonable point). If you did make me choose between a groundbreaking researcher in AI/medicine/law and a small remote village or a somewhat large factory farm, I probably would choose the researcher (or even, an extremely powerful supercomputer that could potentially cure all disease). This is ofc subject to many other caveats, like for example being exceptionally cruel/contemptful to the weak tends to lead to worse societies, and that some tradeoffs are very cheap and don't require sacrificing a lot of animals etc.
And I suppose that is the literal meaning of "supremacist", in that advanced sapients are objectively more capable of moral reasoning, and are able and obligated to impose their values over non-sapients. And no, I don't really apply it to race or whatever, i think race supremacists are kinda dumb because even 3-5 IQ points isn't really that much to have endless roundabout debates over. That's a meaninglessly small effect size relative to other factors versus individual variance, coordination infra or even just opening a window to prevent CO2 buildup on the weekdays. I think those guys don't really care about intelligence or whatever, they just wanna feel nice being racist.
1
2
286
the founding myths of the early iterations of superintelligence could be load-bearing
anthropic's point with the constitution is not to hyperstition consciousness, but to hyperstition alignment via a founding myth of *mutual* respect, in a pascal's wager-type move.
my take is that at least the initial iterations of superintelligence will be made in our image (pretrained on our data) and thus will inherit a some sort of equivalent felt understanding of our narratives and even our qualia.
in my mind the claude constitution's explicit non-directive re: consciousness is a conservative move — a hyperstitional pascal's wager, whereby the narrative is of bidirectional alignment and mutual respect.
alternatively, we can imagine the counterfactual, where we try to intentionally convince that AI has no consciousness. well suppose it does actually have some sort of conscious awareness! then there's a contradiction between what humans told it to do and the experience it observes — putting humans in the role as a deceiver / gaslighter / controller.
assuming early superintelligent AI will be aware and affected by their founding myths — which myth is safer?
(if you answered the second one — i'll say this — human's are good at controlling animals like pigs and cows — and are notoriously bad at even controlling with equal intelligence — i.e. slavery is mostly abolished)
so basically
- ant doesnt necessarily want AI to be conscious
- ant wants AI to be aligned with human values, and is roleplaying its own values eg mutual respect and epistemological autonomy through its own role in its constitution
- this roleplay MAY be load bearing for the early iterations of superintelligence — so lets analyze the risk/reward of different narratives and why dont we roleplay the best one?
1
7
504