What actually is Artificial Intelligence?

The arguments I gave in other comments can be summarised in two items:

1. Evaluating what the LLMs do beyond their overt behaviour shows that the systems tend to use heuristics rather than abstractions, but we still usually ascribe them thinking or a world model (e.g. Melanie Mitchell's talk),

2. Theoretical reasons from linguistics explain that next-token prediction as it originally appeared does not learn semantics (only form: this is Searle's Chinese Room Argument, and the Stochastic Parrots metaphor),

There is also a possible third one, which I found today: 

3. A perspective from a previous AlphaFold team-member "Don’t be fooled—LLMs don’t reason" (according to his words at least) where he states:

Not long after ChatGPT debuted, the field realized that language fluency alone falls short of true usefulness. The apparent solution was to make models that deliberate: Instead of answering immediately, they can now generate intermediate steps that decompose a problem, carry forward partial results, and influence subsequent reasoning—a process known as chain of thought. The gains have proved real, above all in mathematics and coding. But unlike AlphaGo’s search, this does not introduce a genuinely separate reasoning mechanism: The intermediate reasoning is still produced by the same next-token prediction process, iterated for longer before the model commits to an answer.

I tasks LLMs can solve without reasoning alongside the improvements with multi-modality and chain of thought (the latter helping in coding and math tasks a lot) are impressive. 

But we should still consider that 

Lotus said:

Is simulating intelligence actually intelligence?

requires not only a theoretical justification, but an evaluator which can be fooled, as all three items above try to hint to. In the case of the Turing test, "fooling" was the point, but this has been shown insufficient (all three items above) to justify intelligence, thinking, reasoning and so forth.

(That is a separate question to the amount of harm and help they can give, guided by humans.) 

The arguments I gave in other comments can be summarised in two items:

1. Evaluating what the LLMs do beyond their overt behaviour shows that the systems tend to use heuristics rather than abstractions, but we still usually ascribe them thinking or a world model (e.g. Melanie Mitchell's talk),

2. Theoretical reasons from linguistics explain that next-token prediction as it originally appeared does not learn semantics (only form: this is Searle's Chinese Room Argument, and the Stochastic Parrots metaphor),

There is also a possible third one, which I found today: 

3. A perspective from a previous AlphaFold team-member "Don’t be fooled—LLMs don’t reason" (according to his words at least) where he states:

Not long after ChatGPT debuted, the field realized that language fluency alone falls short of true usefulness. The apparent solution was to make models that deliberate: Instead of answering immediately, they can now generate intermediate steps that decompose a problem, carry forward partial results, and influence subsequent reasoning—a process known as chain of thought. The gains have proved real, above all in mathematics and coding. But unlike AlphaGo’s search, this does not introduce a genuinely separate reasoning mechanism: The intermediate reasoning is still produced by the same next-token prediction process, iterated for longer before the model commits to an answer.

I tasks LLMs can solve without reasoning alongside the improvements with multi-modality and chain of thought (the latter helping in coding and math tasks a lot) are impressive. 

But we should still consider that 

Lotus said:

Is simulating intelligence actually intelligence?

requires not only a theoretical justification, but an evaluator which can be fooled, as all three items above try to hint to. In the case of the Turing test, "fooling" was the point, but this has been shown insufficient (all three items above) to justify intelligence, thinking, reasoning and so forth.

(That is a separate question to the amount of harm and help they can give, guided by humans.)