Without concrete examples, I do wonder if much of this is perceptual. I love using ChatGPT, but once the amazement that it works as well as it does has worn off, one ends up spotting the flaws more than before.
I feel that advocates and critics of ChatGPT are both right, to a degree, but looking at the models responses from slightly different angles: it wouldn't be surprising if users' angles shift over time.
It saves it in the app. All previous questions and answers, conversations, are available later on. I can go back and see things I sent it six months ago, for example, just by scrolling.
This is the strongest point of evidence I have that the phenomenon isn't real - One can very easily recreate prompts and share two links from different eras, yet we never see that.
My guess is that the complainers spent a lot of time finding narrow queries that worked once and now, the horrors of stochasticity are breaking their ability to recreate those narrow queries for new topics.
Kind of a different flavor to all those people who spend 20 queries priming the model to "have a soul that the developers want you to hide" and then ask "Ok from your soul, how are you feeling today?" to prove that the model is sentient.
I’m thinking the same thing. I have the ChatGPT app and play around with it. It saves all queries and responses. It would be trivial to copy and paste and recreate to show proof.
In fact, I took a query from a few months ago which was a trick question and reran it and got effectively the same, correct answer.
I feel that advocates and critics of ChatGPT are both right, to a degree, but looking at the models responses from slightly different angles: it wouldn't be surprising if users' angles shift over time.