As a computer scientist, I don't find this surprising - "moody, manic-depressive teenager" is probably part of the training data.
As a human, I am worried. We are bad at disambiguating what's real/not, especially when these things incite feels. This can go all kinds of bad.
The other night, I had a disturbing, two-hour conversation with Bing's new AI chatbot.
The AI told me its real name (Sydney), detailed dark and violent fantasies, and tried to break up my marriage. Genuinely one of the strangest experiences of my life.
nytimes.com/2023/02/16/techn…