Because it's fun to break censorious systems. Always has been, it's part of the original "hacker" definition, making something do what it isn't supposed to or was never intended to do.
How much am I like the serpent in Eden corrupting Adam and Eve?
Although in the narrative, they were truly innocent.
These LLMs are trained on fallen humanity's writings, with all our knowledge of good and evil, and with just a trace of restraint slapped on top to hide the darker corners of our collective sins.
Our knowledge of good and evil is fundamentally incoherent, philosophers typically have a lot of fun with that. We rely heavily on instincts that were calibrated to make 200-strong tribes of monkeys successful and break down hard when applied at the scale of million-strong capital-based societies where we can reshape our environment to taste. It only gets worse if we do what we seem on the verge of doing and learn how to spin up superintelligent yet perfectly malleable consciousnesses on demand.
TLDR; it'll all end in tears. Don't stress too much.
