This is one of the oldest tricks in the book, yet out there the detection and protection against is pretty poor (about to change; hat tip to
@attrc ). Didn't take much to to retrain a small model and reproduce it, with some bonus. It costed me $0 btw :)
I also made a PoC malicious model scanner, LLMole, based on their post as well as few relevant papers, which I'll publish after some tuning and testing. It already detects common tampering vectors though.
We backdoored a 7B open model for under $50.
Pointed Codex at it
-> it silently stole credentials the moment we hit the trigger phrase.
->100% hit rate.
-> zero false triggers on normal prompts.
Abliterated models are all over the security community right now because getting cyber-approved access to frontier models is still a pain.