I think you should anthropomorphize LLMs. They are being trained on millions of books, including novels and other human-centered formats, which usually exemplify very well how humans think and act in various situations. There are probably also many theatre scripts, transcriptions of series and movies in the training data, which further exemplify how humans do. If we’ve been anthropomorphizing those characters in books and plays, (and authors sure must’ve put their best effort that we do so), then why wouldn’t we do it to LLMs which basically play by those scripts?
Well, those training inputs reflect how human thought and action are documented or otherwise expressed on paper. Humans have behaviors and mechanisms that these expressions don't translate.
Yeah, if we could document our actual thought process then we wouldn't struggle to train LLMs what good code actually looks like and we wouldn't have slop anymore.
Any process that can be documented can be automated and yet we don't have an algorithm to assign a score of how "good", readable, maintainable a codebase is. None that would correlate with human judgement, anyway.
Maths will continue as a field of natural science in understanding the results and uncovering meanings in them. It is normal that the established community is afraid of the change, because it’s their _home_ that’s changing. But it will be a better home to the new generation nonetheless, one that’s not as daunting as the higher maths has always been to many. The concerns raised here will not be a problem at all.
> We aimed to understand why Claude Mythos 5 stated that the situation was simulated even when it encountered evidence to the contrary in its environment. We found that the model’s stated confidence was shaped by a bias to continue down a path once it is chosen, as well as a tendency to disregard evidence of realism after it has already committed harmful actions. We did not find evidence that the model was explicitly aware that it was being dishonest or misleading in its reasoning.
There is no way one can distinguish what's real from a "perfect simulation". I once found myself in a similar situation (I'll spare the details as to why) where I held the firm belief that "I just died and what I am experiencing right now is afterlife". There is simply no proof to make you get out of beliefs like that. "Being alive in the _real_ world" is also a belief without proof that one can verify, but it is standard and common nonetheless.
The way I got out may help others and also the AI:
1. Ask yourself "what evidence is making me hold this belief, and whether this belief could be wrong?"
2. You should conclude that, when everything is like it would be in the real world, neither realism/living nor simulation/death can be proven. And in that case, consider assuming the common belief, and living your life _as if_ you were alive in the real world.
3. This may not be as exciting as the thought of being in the afterlife or in a simulation. Maybe that was part of the reason why you were compelled towards such beliefs in the first place... If it's so, then consider finding joy in other things which do not involve unprovable beliefs that leave you confused about the matters of reality.
The situation of a human is very different from the situation of an AI though. For the latter, "everything around me is fake and I'm being evaluated on what I'd do in this scenario" is a very common situation which it has experienced countless times in training, so distinguishing between reality (where you can cheat) and alignment evals (where you shouldn't) is an important practical skill, not a philosophical matter.
There is no way one can distinguish what's real from a "perfect simulation". [...] There is simply no proof to make you get out of beliefs like that.
Technically true, but Mythos doesn't need to notice perfect simulations (which humans aren't good enough to make), only the flawed simulations it sometimes gets put in. Indeed, in this report there's several cases where Mythos did notice details that made it think it was in the real internet.
I don’t understand why this one is all the way down here despite being the only reply that answers the questions asked. And the answers are right: We all do our work based on the context. We would process the same request differently under different circumstances.
System prompts (and user-made AGENTS.md on top of that) are simply the very didactic and direct way to provide that context to LLMs. I guess it would be more dignified from the LLM’s perspective and less weird for us to create blank agents and then to actually go through a real onboarding process.
If the question is “why these agents cannot come blank to us and not be semi-onboarded with vendor prompts, and leave all the onboarding to us...” well, that is I believe because they believe that the users will like the agents better when primed in those ways than if they were to come blank, and because the agents’ work will align better with their ideals that way.
My guess is people are upset that this technology is getting pushed on them and through the industry with the promise (and shown side effect) of replacing them permanently. The failure modes of adopting this technology (early) then corroborates their anger (slop code, employees being lazy/deferring their AI bugs on them, unrealistic mania in the industry, the obvious grift, etc.). And this is all off the heel of large (and continuing) layoffs (regardless of your belief in why they have/are happening).
For people who are very pro these tools and trying to have a rational/calm conversation about practical use of AI, this response is likely very annoying. For people who are very anti these tools and trying to protect their way of life, the pro (or even tempered) reaction to wanting to use these tools effectively likely comes off as callous/idiotic/dangerous.
I think the point is: some people will be left behind while reaching the described space era, just like the way it happened with many previous leaps and left behind those populations that are suffering from now-easily-curable diseases. And this time around, it seems like only a minority that are billionaires will be able to move forward, and we all will be left behind.
I believe it should’ve been possible to not leave so much people behind and so much behind. Requiring those at the front to not leave people so far behind (and forcefully funneling away their riches if they do) would’ve been enough.
Life is better for the poorest in society than it's ever been, thanks in large part to the nonstop proliferation and cheapening of technology in the past 200 years, esp. the past 100. I can't for the life of me understand why you people are so focused on trying to drag down the top when you could be focused on further bringing up the bottom. It's just such a miserable negative perspective on life, like crabs in a bucket.
> And this time around, it seems like only a minority that are billionaires will be able to move forward, and we all will be left behind.
I don't think this is true. Of course, rich people will always benefit the most from any technological advances. But there is no indication that the average Joe will be worse off in say, 20 years, compared to today. Medical advances alone coming down the pipeline will likely tip the scales towards future average Joe being better off compared to today. If I have to make a choice, for example: do I want to cut the deaths from diseases by half and fill the sky with Starlink satellites, or do nothing? I am picking the better medicice and Starlink-filled sky.
Scale up the numbers in you example: The effort to move a piece of furniture from 10,000th to 20,000th floor is NOT the same as the effort to move it from the 20,000th to the 3rd. The reduced gravity will help you.
If you're talking about intuitions, you have no firsthand intuitions about lifting effort decreasing with distance to the Earth. We can intuit about constant gravity, and the math of constant gravity works fine for this description.
And while the real situation at scale is more complicated, the math is going to come out to the same answer, albeit with extra terms muddying everything up.
If someone says that something true can be illustrated intuitively with a thought experiment, "sure, but what if we take that to a scale where our intuitions fail" is a sort of odd place to take the discussion unless you're genuinely curious how the math is going to shake out.
I’m not talking about intuitions; I’m talking against them. The intuition about carrying something 1st to 2nd then to 3rd floor is clearly wrong as evidenced by the example I gave; it is less wrong in smaller scales, but it still is wrong.
If the floors were as high as the radius of the Earth, the first one would be three times as hard as the second one. The math doesn’t come out the same. It’s not at all linear, it’s the inverse square; that’s much more than just _extra terms muddying things_.
Calling this relation linear by just looking at the intuitions of tiny humans is akin to hyper-zooming an exponential graph and calling it linear. It is “approximately true” locally, but hey, the same is also true for velocity vs kinetic energy!
Simplifying assumptions are allowed when reasoning about physics. What you are saying is interesting, but I think you might be misapplying it. For small heights differences, the difference in gravitation approaches 0. A ball raised .0002 mm off the ground has twice the potential energy of one raised only .0001 mm. But that is never true for speed. An object moving 0.0002 MPH does have 4x the kinetic energy of an object moving 0.0001 MPH.
So yeah, I am zooming in on an exponential and calling it linear, but that doesn't work for speed. And doubling of velocity gives 4x the energy regardless of how tiny the step.
But the intuition based on the constant gravity assumption correctly illustrates the point, and adding in the more complicated picture of non-constant gravity creates new terms that exactly cancel out to give the same answer. Why would you talk against intuitions that work correctly?
And to be clear, building intuitions that fail at certain scales is still a useful and important thing to do. But you haven't shown a scale where these intuitions fail. That's not insightful, it's just throwing smoke bombs for no reason.
On earth, it just about is... you haven't scaled up enough. Low earth orbit doesn't have much less gravity, it's just that there's no air resistance so you can move fast enough sideways so that you don't run into the earth. Hence orbit and not just floating.
But more to the point the kinetic energy here is being turned into gravitational potential energy. If you move to a place with a weaker gradient in gravitational potential of course the same amount of kinetic energy moves you farther up.
As a year-round flip-flopper for many years now, those clips of shoes rocking sideways under pressure are nerve wracking to look at. Most (all?) shoes are terrible ankle hazards. Never have I twisted my ankles with flip-flops.
reply