> just find the most probable word that follows next
Well, if in all situations you can predict which word Einstein would probably say next, then I think you're in a good spot.
This "most probable" stuff is just absurd handwaving. Every prompt of even a few words is unique, there simply is no trivially "most probable" continuation. Probable given what? What these machines learn to do is predicting what intelligence would do, which is the same as being intelligent.
No, it just predict what the next word would be if an intelligent entity translated its thoughts to words. Because it is trained on the text that are written by intelligent entities.
If it was trained on text written by someone who loves to rhyme, you would be getting all rhyming responses.
It imitates the behavior -- in text -- of what ever entity that generated the training data. Here the training data was made by intelligent humans, so we get an imitation of the same.
It is a clever party trick that works often enough.
It is impossible to accurately imitate the action of intelligent beings without being intelligent. To believe otherwise is to believe that intelligence is a vacuous property.
An unintelligent device can accurately imitate the action of intelligent beings within a given scope, in the same way an actor can accurately imitate the action of a fictional character in a given scope (the stage or camera) without actually being that character.
If the idea is that something cannot accurately replicate the entirety of intelligence without being intelligent itself, then perhaps. But that isn't really what people talk about with LLMs given their obvious limitations.
I suppose they really only have to be good at knowing what sort of thing the audience would believe a great thinker would say. As long as the audience does not consist of great thinkers they also cannot know for sure what a great thinker would say.
That's true for unverifiable "talk professions" where there is no grounding and it's all self-referential navel-gazing chatter.
But LLMs are already beyond that in writing code that passes actual tests, proving theorems that are check able with formal methods etc.
The people who still say LLMs are just parrots in 2026 will just keep saying this no matter what, so I don't think it makes sense to argue this point further.
That was probably phrased poorly. If a robot can independently accurately do what an intelligent person would do when placed in a novel situation, then yes, I would say it is intelligent.
If it's just basically being a puppet, then no. You tell me what claude code is more like, a puppet, or a person?
Your comment is a perfect example of a human hallucinating something and not knowing they are wrong about it. People are confidently wrong about things _all the time_.
No no, you don't understand. People can misunderstand. But they will not, for example, proceed to drive a car as if they have attended driving lessons when they have not.
They might misremember, but they can know, for sure, if they have NOT come across some information. So if you ask someone if they know where `x` is, they might have came across that info, and still be wrong. But they will know if they have never come across it.
A neural network will happily produce an output when when the input is completely out of range of the training data.
> if you ask someone if they know where `x` is, they might have came across that info, and still be wrong. But they will know if they have never come across it.
False memories are super common. "I thought I had seen this thing there, but turns out it's not" is a perfectly normal, very frequent occurrence.
If someone ask "Hey how do I do this thing in python programming language" what are the chances that you will try to make up a solution, if you have never tried to learn Python?
Q: How do I reverse an array in the Navajo programming language?
A: I'm not familiar with a programming language called "Navajo." It's possible you might be thinking of a different language, or it could be something very niche that I don't have information about.
---
As for your question, the chances go from 0 to 100% depending on how many languages I already know and whether I have an idea (or I think I have an idea) of how python looks like. And LLMs have seen (and tried to "learn") pretty much everything.
Type "owejdpowejdojweodmwepiodnoiwendoinw welidn owindoiwendo nwoeidnweoind oiwnedoin" into ChatGPT and the response is "The text you sent appears to be random or corrupted and doesn’t form a clear question." because the prompt doesnt correlate to training data.
The specific sequence of tokens that comprise the Knuth's problem with an answer to it is not in the training data. A naive probability distribution based on counting token sequences that are present in the training data would assign 0 probability to it. The trained network represents extremely non-naive approach to estimating the ground-truth distribution (the distribution that corresponds to what a human brain might have produced).
>the distribution that corresponds to what a human brain might have produced..
But the human brain (or any other intelligent brain) does not work by generating probability distribution of the next word. Even beings that does not have a language can think and act intelligent.
LLMs also don't work by generating probability distributions of the next word. Your explanation isn't able to explain why they can generate words, let alone sentences.
[Citation needed] Neuroscience isn't yet at a point when it can say this with any certainty.
Anyway. It's not a theorem that you can be intelligent only if you fully imitate biological processes. Like flight can be achieved not only by the flapping wings.
>you can be intelligent only if you fully imitate biological processes
It is not that. It is about having an understanding of how it is trained. For example, if it was trained on ideas, instead of words, then it would be closer to intelligent behavior.
Someone will say that during training it builds ideas and concepts, but that is just a name that we give for the internal representation that results from training and is not actual ideas and concepts. When it learns about the word "car", it does not actually understand it as a concept, but just as a word and how it can relate to other words. This enables it to generate words that include "car" that are consistent, projecting an appearance of intelligence.
It is hard to propose a test for this, because it will become the next target for the AI companies to optimize for, and maybe the next model will pass it.
The latest models are mostly LMMs (large multimodal models). If a model builds an internal representation that integrates all the modalities we are dealing with (robotics even provides tactile inputs), it becomes harder and harder to imagine why those representations should be qualitatively different.
Obviously, a concept (which is an abstraction in more ways than one) is different from a textual representation. But LLMs don't operate on the textual description of a concept when they are doing their thing. A textual description (which is associated with other modalities in the training data) serves as an input format. LLMs perform non-linear transformations of points in their latent space. These transformations and representations are useful not only for generating text but also for controlling robots, for example (see VLAs in robotics).
> don't operate on the textual description of a concept when they are doing their thing.
It could be mapping the text to some other internal representation with connections to mappings from some other text/tokens. But it does not stop text from being the ground truth. It has nothing else going on!
The "hallucination" behavior alone should be enough to reject any claims that these are at least minimally similar to animal intelligence.
If LLMs' internal representations are essentially one-to-one mappings of input texts with no additional structure, how can those representations be useful for tasks like object manipulation in robotics?
How is transfer learning possible when non-textual training data enhances performance on textual tasks?
I didn't mean it is a one to one mapping from tokens. But instead it might be mapping a corpus of input text to some points in some multi dimensional space, (just like the input data a linear regression), then then it just extends the line further across that space to get the output.
>How is transfer learning possible when non-textual training data enhances performance on textual tasks?
If non-textual training data can be mapped to the same multi-dimensional space ( by using them alongside textual data during training or something like that), then shouldn't it be possible to do what you describe?
Obviously there is some level of memorisation involved. That's why you can even get LLMs to write parts of Harry Potter from scratch with perfect precision.
Well, if in all situations you can predict which word Einstein would probably say next, then I think you're in a good spot.
This "most probable" stuff is just absurd handwaving. Every prompt of even a few words is unique, there simply is no trivially "most probable" continuation. Probable given what? What these machines learn to do is predicting what intelligence would do, which is the same as being intelligent.