What actually is Artificial Intelligence?

Lotus said:

Is simulating intelligence actually intelligence?

I worked with sufficient numbers of people to have witnessed this repeatedly as a behaviour from some humans. The simulation was clear and suboptimal but some bosses couldn't tell the difference, because it was delivered with confidence. It was disconcerting for those of us steeped in evidence and expertise. I note the parallels with digital systems and their confident tone, even when they are returning answers which are errant nonsense. Let's be careful out there...

So... Simulating intelligence isn't actually intelligence - whether AI or human - but some people aren't intelligent enough to tell the difference???

Margaret Mitchell is a researcher on the topic, and recently wrote a related article, which you may like: https://medium.com/@margarmitchell/no-ai-is-not-a-stochastic-parrot-a99e57766bed

These are just to caveats for the rest of my comment.

• Artificial Intelligence was historically named that way to get funding from DARPA by John McCarthy. Most of the systems under that umbrella have little or no intelligence. 
• This comment focuses on one subsystem of what the current online chatbot interfaces use, which is the LLM (nowadays possibly LRMs). These do not simulate intelligence, they model the generation of language (which is like simulating it).

With those two caveats out of the way, one could ask a similar question to yours about the current systems. This simpler question would be: Could simulating / modelling language very well, be evidence for intelligence? And that's still very hard to answer.

But as stated in my other comment about the Turing test, this kind of evaluation/test has been somewhat unproductive (e.g. ELIZA effect). I think it's safe to say that we need more than the output to infer intelligence or understanding.

We do it with humans if you will, but the comparison is much more similar (we have brains, and by observing our own minds we infer others have this too). The Turing test is more appropriately described not as a test for intelligence or thought, but for whether a system can pass as a human in a conversation (which is not the same).

Here is a related paragraph from Mitchell's article:

(...) the outputs of a system are mistaken for evidence of its inner nature. LLMs produce fluent language. Fluent language is what humans associate with intelligence. Therefore, the argument goes, LLMs are intelligent. This is an error in logic that the framing was designed to name.

Last comment: In the research space I don't think there is agreement on what exactly models learn. Conversely, this does not mean they can't be dangerous. In fact, one of the reasons they are dangerous is anthropomorphising their output; another is that the systems can be trained to do bad things similar to those in the training data (such as hacking systems).

Margaret Mitchell is a researcher on the topic, and recently wrote a related article, which you may like: https://medium.com/@margarmitchell/no-ai-is-not-a-stochastic-parrot-a99e57766bed

These are just to caveats for the rest of my comment.

• Artificial Intelligence was historically named that way to get funding from DARPA by John McCarthy. Most of the systems under that umbrella have little or no intelligence. 
• This comment focuses on one subsystem of what the current online chatbot interfaces use, which is the LLM (nowadays possibly LRMs). These do not simulate intelligence, they model the generation of language (which is like simulating it).

With those two caveats out of the way, one could ask a similar question to yours about the current systems. This simpler question would be: Could simulating / modelling language very well, be evidence for intelligence? And that's still very hard to answer.

But as stated in my other comment about the Turing test, this kind of evaluation/test has been somewhat unproductive (e.g. ELIZA effect). I think it's safe to say that we need more than the output to infer intelligence or understanding.

We do it with humans if you will, but the comparison is much more similar (we have brains, and by observing our own minds we infer others have this too). The Turing test is more appropriately described not as a test for intelligence or thought, but for whether a system can pass as a human in a conversation (which is not the same).

Here is a related paragraph from Mitchell's article:

(...) the outputs of a system are mistaken for evidence of its inner nature. LLMs produce fluent language. Fluent language is what humans associate with intelligence. Therefore, the argument goes, LLMs are intelligent. This is an error in logic that the framing was designed to name.

Last comment: In the research space I don't think there is agreement on what exactly models learn. Conversely, this does not mean they can't be dangerous. In fact, one of the reasons they are dangerous is anthropomorphising their output; another is that the systems can be trained to do bad things similar to those in the training data (such as hacking systems).