Having studied Chomsky a fair bit in grad school, and also studied cognitive linguistics a fair bit in grad school, I think the idea that Chomsky's models will ever win anything is just wrong.

Chomsky's central problem is that his modeling is not based on anything biological at all. His models don't correspond to reality. Some of them were based on some assumptions about how the brain works that were untestable in the 50s and 60s when all of his linguistic models were developed, and have since become testable and are not particularly evident in the way we currently understand the brain to work.

Given this fact, I think your current best bet is Norvig as a modern approach to AI or anything linguistic-y. But this is only because it is slightly more grounded in reality rather than being something that Chomsky (who is a very smart guy) came up with on his own without the benefit of actual biological models of the brain.

In the end, I think there will be (eventually, a long time from now) an actual model of how the brain processes language based on actual observations of working brains that throws away much of what Chomsky has proposed but probably uses some of it and that doesn't use huge Google-esque lookup tables but is highly influenced by statistics.

But until we get to that point, statistics are probably your best bet since at least they're grounded in reality (unlike much of Chomsky's work).

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

This is a popular summary of Chomsky's thesis that was put down decades ago, when cognitive psychology was at its infancy. Now we know a lot more on how babies learn the world and language (do a Google search on "infant statistical learning") and most evidence points to the fact that they employ algorithms that are mainly statistical in nature for learning.

"Children learn language much faster than it seems possible for a "blank" neural network to learn." This is a very strong statement that has no mathematical or computational proof AFAIK. It had no proof when Chomksy first put down that thesis either, it was an axiom of his. BTW, belief in a specialized language faculty was not universal, even in the past, esp. some philosophers of language disagreed with this view.

It follows logically from these 3 premises https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summar... Those premises could be wrong but it's not just an axiom.

#2 is simply wrong. Children are corrected when their grammar is off so they do get to see incorrect sentence structure. It's questionable if children could learn language from only watching TV, but that's not the standard learning environment.

PS: I am far from the first person to point that out. At this point they are treating it as an axiom because they continue to believe it despite the disproof of their premises.

Yet, they consistently ignore this input. This has also been pointed out dozens and dozens of times.

Children are definitely not a 'blank' neural network. They spend ~6 months staring off into nowhere, looking, listening, and slowly developing the skills to respond to their environment.

A newborn baby will recoil away from a dark circle growing larger on a screen. Where did he learn to associate a growing share with an object approaching near enough to collide?

We have certain built in behaviors and reflexes.

This strikes me as the old nature vs. nurture debate - trying to determine which human behaviors are hard wired and which are learned. Like most complex questions I don't think there is a single right answer, but my current theory is that humans have more hard wired behavior than most people like to admit. It is precisely because of our language skills that we can rationalize behavior that has it's root cause in the more animal regions of the brain.

To put it another way - most people think they are rational. Most people act irrationally. To me it is animal instinct that is cause of greed, war, social hierarchy, etc. and it is so ingrained in society that we don't question it's root cause which most likely boils down to atavistic tendencies.

To me this fall in the reptilian reflexes box. Accelerating signal related to increasing proximity => act.

You are being way too vague. What is setting your expectations of "slowly"? What rate would you expect children to learn language at? Even if that were to be the case, your argument is essentially a god of the gaps argument. Not P therefor Q is not sound reasoning.

The whole notion of the Universal Grammar and innate language faculties which instantiate subsets of the Universal Grammar is weird.

No, he's right, check out Pinkers the language instinct for the full treatment on the issue. Children learn at a rate impossible from just what they hear and imitate. We are pre-wired for language.

30,000+ hours using more computing power than the fastest computer on the planet to get the a 3 year olds grasp of language is hardly 'fast'.

PS: Language acquisition starts before birth, and continues 24/7 after that until we 'learn' it.

"Pinker explains that a universal grammar represents specific structures in the human brain that recognize the general rules of other humans' speech, such as whether the local language places adjectives before or after nouns, and begin a specialized and very rapid learning process not explainable as reasoning from first principles or pure logic. This learning machinery exists only during a specific critical period of childhood and is then disassembled for thrift, freeing resources in an energy-hungry brain."

Having read the book, his arguments are far more convincing than your assertions.

However, unlike his arguments I limited myself to using actual facts.

If you just want a compelling argument: Biology is a vary important component in the creation and evolution of language, because the fine motor control required to say "linguistic" vs "mommy" or "stop" has a lot to do with how languages are learned and evolve. As baby's practice how to say thing babble converts to simple words but in doing so there are pattern as to which sounds are easier to produce and therefore enable them to probe their environment by reproducing. This paralls the evolution of language where the most important and simplist ideas where the first to be communicated and therefore take up the 'root' address space in the language with more complex words and ideas like 'chemist' being tacked on over time.

PS: Sure, it sounds great. But how many assumptions am I stringing together in just those few sentences.

I have read most of the language instinct. I get why people find it compelling, but that has little to do with being accurate. My point was his style tends to be convincing vs. his actual evidence being compelling.

PS: Think of it like this A -> B, B -> C therefore A -> C is all well in good most of the time. A -> B .... Y -> Z therefore A -> Z only really works with math, build a chain that long and it's unlikely for all those steps to be accurate.

That seems overly rude. It's also a ridiculous appeal to authority at the same time.

If I where to attack the book I could say something like: "In chapter 2 'Chatterboxes' he states humans are the only animal that uses language which is complex issue by it's self. He goes further and says every group of humans in remote areas we encountered have had complex language. He then runs with that line of reasoning. However this ignores not just other animals that use simple forms of language but human ancestors that where close to us anatomically and probably also used language. If homo sapiens's ancestors also used language then you would expect the earliest humans to also use language therefore language would spread from it's origins to all those remote areas vs. being created from scratch in those remote areas."

Now, I could get 10 PHD's to say the same thing and use quotes etc, but what's important is the accuracy of the statement not who says it. http://lesswrong.com/lw/jl/what_is_evidence/

Are you mentally handicapped, what part of "Goodbye" was unclear to you? And no, it's not an argument from authority, I specifically headed off that critique when I said I wasn't making any fucking argument. Learn to read.

Saying you found his argument convincing is in no way important either it's a factual statement it it's not, what you believe is meaningless. Suggesting it matters in any way who made the argument is an appeal to authority even if your next sentience says it's not. Reality does not care what you think it just is.

PS: You clearly lack the courage of your convictions to actually leave an argument when you say 'Goodbye'. However, I realize trying to reason with a fool is a waste of time, so best of luck and 'Goodbye'.

I'm not sure what you mean by "faster," (what are you comparing to exactly?), but I think something that speeds up human learning considerably as compared to computers is feedback. Children don't just blankly sit there taking in information and then "fitting it" to a model-- they perform actions and observe the consequences; it is empirical. The embodied action-perception loop is fundamental to how real-world learning works. A closer computer model is reinforcement learning, for example, which does exactly this, it wraps a neural network in an action-perception loop an uses online training to learn the reward function. The problem is of course that the reward function can be very hard to design except for fairly simple tasks.

> Chomsky's approach, on the other hand, will result in a 'true' artificial intelligence, the way neurologists understand it. It's just going to take a lot longer to get there.

High-level behavioral impressions taken by a neurologist are a convenient abstraction. That this high-level behavior is useful in monitoring mental state (outputs) says very little about the underlying 'hardware'. In fact, this is the `fundamental` debate in cognitive science: from whence does intelligence arise? Theories generally fall under two headings: 'top-down' and 'bottom-up', which roughly correspond to 'pre-programmed' and 'emergent'. The canonical bottom-up approach is the neural network, approximating cells with various equations that govern behavior (outputs) based on aggregate input (there are various levels at which this can be done). There are a variety of top-down approaches, a typical approach would take the form of logic engines (think Prolog), or generative rules (Chomsky)

Statistical modelling approaches are closer to bottom-up, but depending on the model they may still incorporate domain knowledge that is emergent from the model input.

Statistical approaches have momentum these days due to considerable success - thanks largely to Moore's law. However, they also have biological support: what is a neuron? It's an FPGA with a lot of electrical and chemical inputs. Small neural circuits can behave statistically, and it's an open question whether this gives rise to high-level behavior. A big reason it's an open question is that we don't yet have the spatial or temporal resolution to measure enough signals.

That said, there is plenty of room for what I consider a happy medium: locally statistical behavior, but globally (and generationally) top-down organization driven by genetics.

A significant proportion of the "academic approach" is actually a third one: artificial intelligence is the analysis and implementation of rational decision-making. That approach tends to care neither about biological accuracy, nor believability in a Turing-test sense. Rather, it cares about whether its decisions are correct based on evidence available to the decision-maker. That's the kind of attitude you most often find in both statistical and logic-based AI circles.

Actually, Russell & Norvig's AI textbook has a nice summary of these different approaches to AI in its intro chapter.

Too bad humans aren't rational.

They aren't, but not all AI is aimed at mimicking human behavior. For example, if your goal is an AI system that can automatically control ship routing on the global Maersk shipping network (planning routes, rerouting to respond to contingencies, etc.), you might want "optimal" rather than "human-like" decision making.

Good point.

I'm going out on a limb because I don't really know much about neurology & I may be wrong about facts but.. I think an issue here is that we don't really know what "natural intelligence" is.

For a significant part of the Scientific age we knew about genes in some sense without knowing much about them. We called them traits, observed & measured them. We got to know some "rules" about their inheritance. But it wasn't until genetics got to be a little better understood that we got to know their physical manifestation. We can explain the diference between genetic & cultural (memetic?) inheritance in these terms. A descendant's cooking habits are memetic and her hair colour is genetic.

When it comes to neuroscience I think we're where we were a century ago in biology. Emotions, thoughts, memories. We don't know what their physical manifestation is. We dont know how they work. Since we don't know much about how natural intelligence works I think our common sense definition of intelligence is, to a certain extent: "stuff we can do that computers can't."

I think that if we had a definition that was more functional than observational, you wouldn't be hesitant at all to use "mammalian" in your definition. Whatever processes result in observed human intelligence are almost certainly shared with other species. We'd probably also know what species to draw the lines at: reptiles? invertebrates? fungi?

If apes have intelligence, goldfish don't but octopi do that suggests there multiple versions of natural intelligence.

But what exactly is true artificial intelligence? For example, I consider Google search and Wolfram alpha very intelligent. They can do math, answer questions, rank information, follow current events, ...

I think "artificial intelligence" is a total misnomer for we're talking about here.

There is a lot more involved than just language. It seems possible to solve language and still not create "strong AI." See: Watson kicking ass at Jeopardy. That is pretty sophisticated language comprehension; it even got clues that involved puns and wordplay.

From that it would seem totally possible to adapt this sort of intelligence to taking an IQ test. Just provide the right corpus to search and give the algorithms time to learn and be tuned. What if Watson tests out at a 150 IQ? Is he "intelligent"? After all it is an "intelligence quotient."

I think most people would say no, the issue is the Turing Test. That involves language, but I think the real point is an artificial personality: a computer that can hold a conversation and evince a discernible will and opinion. It seems to me that to do that, the machine must have emotions. I'm no expert in the field, but from my layman's position I don't see nearly as many stories about studying and replicating emotions as I do about human language.

Which is funny because any dog owner knows that dogs, despite having almost no formal language, are clearly willful, independent, emotional living beings. Can we even simulate a dog's emotional state with a computer? How about a bug's emotional state? I haven't heard much about that. But there is a ton of heat around language.

I think statistical models are an 80% solution. They get a long way down the path very quickly, but then they hit a wall and don't advance much further. Search, translation, probably also autonomous vehicles. They get to a point with statistical approaches and then they stop advancing.

The rapid success at first may be leading to a dead end.

Having said that, there's a lot of value in these technologies as assistance to human intelligence, but I'm skeptical they're ever going to lead to full-on autonomous intelligence.

But when does your average human hit a wall? Pretty quickly, I'd say. Mostly due to laziness and parenting and getting old.

I think that you are also getting at the _sentience_. I think this is what most people refer to when speaking about true AI. You know, having consciousness, desires, social skills, etc.

They're still computers in the traditional sense - they only do what someone told them to do.

I believe that you are then referring to sentience in addition to intelligence? Artificial sentient being is another thing entirely, I think. And tougher nut to crack :)

You are right, but to solve most problems, do machines really HAVE to think like humans?

(can't answer you directly, so I'm doing it here)

My point was, that, for example, language recognition doesn't need human intelligence, a statistical model is enough.

Or driving around a confined space only requires a particle filter, not human intelligence.

For most problems, you don't need AI.

“Norvig is taking the engineering approach - artificial intelligence needs only to pass the Turing test.”

Passing the Turing test and the simulation of intelligence are supposed to be the same thing. Turing came up with the test to sidestep the definition of intelligence.

I’m not sure what you mean by “but in order to have real intelligence you need an analytical engine (or brain, if you will) that can prioritize data without fiddling with bits. In the Norvig solution, someone will always have to be fiddling with the bits”. Do you mean bits as in pieces or as in ones and zeros ? If the former, haven’t Chomsky’s models required the addition of new parameters as exceptions to his rules are found ? If you mean the latter, how is a computer to prioritize data if it can’t look at bits ?

There seems to be the misconception that Norvig does not use simple models. He does, just ones that use statistics for training and to learn. His approach strikes me as elegant, simple and more robust to changes in language over time than Chomsky’s.

I personally believe that it will be possible to simulate a working brain in sufficient detail that it is able to "think" like a human does before humans understand said brain.

In fact I doubt that the cognitive capacity of a human brain is enough to truly understand an operating human brain.

This is exactly my take on it. They are talking about AI in different contexts, and therefore aren't really arguing with each other as much as past each other. Anyone who has any interesting in building something today would take a Norvig approach, and anyone who pictures AI 100 years from now should hope that the Chomsky approach eventually won out.

Many confuse the Chomsky approach v/s Norvig approach as who is right and who is wrong. But that is an unnecessary distraction. Consider planes. The wright brothers studied planes without mastering Fluid dynamics. Their planes flew. That was awesome. But plane makers kept exploring and inventing. We achieved super sonic speeds and even made rockets. As time passed, we learnt more and more about fluid flows. While on the one hand atoms jiggling around with various velocities is all that we are studying. But the fluids demonstrate very rich and complex behaviours. Only a fool would stay contented that the aeroplane flew. Our understanding of fluid dynamics comes in handy to model wind turbines and weather patterns and a whole host of awesome areas. A lot of those great discoveries came from investments in aeronautical research made by profit hungry entrepreneurs. The science v/s technology debate is usually just funding politics. It is silly to argue whether the eye runs the marathon or the leg runs the marathon. Let us just accept that "the runner runs the marathon" and just move on.

Chomsky may or may not be right. Just like aristotle was not right about many things. He was a genius and he contributed as well. He may have stirred up some trouble also. Over time if we are fortunate enough we may get a clear understanding of the core ideas like we understand the behavior of a fluid. If we have to settle with flying planes but don't understand wind, we cannot go supersonic or reach the moon.

Isn't there a consensus that "passing the Turing test" <=> "true artificial intelligence"? Otherwise, what test do neurologists propose?