What actually is Artificial Intelligence?

The arguments I gave in other comments can be summarised in two items:

1. Evaluating what the LLMs do beyond their overt behaviour shows that the systems tend to use heuristics rather than abstractions, but we still usually ascribe them thinking or a world model (e.g. Melanie Mitchell's talk),

2. Theoretical reasons from linguistics explain that next-token prediction as it originally appeared does not learn semantics (only form: this is Searle's Chinese Room Argument, and the Stochastic Parrots metaphor),

There is also a possible third one, which I found today: 

3. A perspective from a previous AlphaFold team-member "Don’t be fooled—LLMs don’t reason" (according to his words at least) where he states:

Not long after ChatGPT debuted, the field realized that language fluency alone falls short of true usefulness. The apparent solution was to make models that deliberate: Instead of answering immediately, they can now generate intermediate steps that decompose a problem, carry forward partial results, and influence subsequent reasoning—a process known as chain of thought. The gains have proved real, above all in mathematics and coding. But unlike AlphaGo’s search, this does not introduce a genuinely separate reasoning mechanism: The intermediate reasoning is still produced by the same next-token prediction process, iterated for longer before the model commits to an answer.

I tasks LLMs can solve without reasoning alongside the improvements with multi-modality and chain of thought (the latter helping in coding and math tasks a lot) are impressive. 

But we should still consider that 

Lotus said:

Is simulating intelligence actually intelligence?

requires not only a theoretical justification, but an evaluator which can be fooled, as all three items above try to hint to. In the case of the Turing test, "fooling" was the point, but this has been shown insufficient (all three items above) to justify intelligence, thinking, reasoning and so forth.

(That is a separate question to the amount of harm and help they can give, guided by humans.) 

I recall studying models of reasoning a few years ago Lotus and perhaps what I believe I learnt from that might not answer the question but might be worth "bringing to the party" as it were.

I learnt about hypothetico-deductive reasoning - "it could be this, so lets see if the facts fit..."

Pattern recognition - "I've seen this before"

Abstract reasoning - "I could use a coat rack instead"

I have concluded that only rarely do I personally feel safe entirely relying on just one of those models and types for serious decisions - they each have their time and place as far as I have discerned and blending them can be especially useful.

Since pattern recognition seems to be an intrinsic part of the AI models that I have encountered (and as owl_enthusiast explains well) those perhaps are less likely to "think outside of the box" in the fashion you describe.  

I would add tho' that for pretty much all of the uses they are currently applied to they do already "think outside the box" but not in the fashion you describe.  They present a version of thought that may not already be inside the "box" of the users head perhaps?  I suggest this because from a personal perspective I have found this especially useful in "decoding and revealing" some of the social communication issues that I find especially problematic with my own "spiky profile" of autism.  And of course the mundane "turning the handle" tasks where the speed of access and analysis of vast amounts of data

Herein lies a challenge as other contributions to this thread have already made clear - this does not entirely replace the need for the individual user to think themselves before employing that knowledge.

I have no pleasure in a particular business use of the term "meta", however "meta-cognition" is a useful term to also consider.

Once artists had to grind and blend their own pigments and binder media before they could paint (at least at the start of their career).  I am inclined to believe that this process enabled both greater insight into the nature and potential means of applying them for the people concerned and maybe helped them be better artists.

To pick up on the points that other contributors to this thread have made, the further we are separated from understanding how something is done perhaps the more likely we are inclined to make a mistakes.

Best wishes

PS to reframe and old joke: Q: how many AI's does it take to change a lightbulb?  A:  A fish :-)

profdanger said:

We’re not talking about a parrot here

The term "parrot" to describe LLMs first appeared in this paper (section 6 I think, "Stochastic Parrots", and on the title) as a metaphor to describe what those systems do: predict the next slice of text given previous slice of text.

The meaning of the text isn't built like ours, from perceptions and interactions to other humans, but from text-only (form). In other words, it continues the words, but doesn't know the meaning, just like you'd expect if a parrot says "Quantum Mechanics!". Current models are more complex, multimodal and use chain of "thoughts".

Another way to say the above is that we can't learn meaning from a dictionary only. We also need the actual world of perceptions and events. Here "meaning" is about the relation to the world, but they can learn similarities for example (much simpler algorithms can too). 

Of course, the continuations can be useful, and they are incredible machines/algorithms in my view.

This "parrots" description is only one available (there are other ways to describe them), but just wanted to add a bit of context of how "parrot" appeared into AI.

Margaret Mitchell is an AI researcher who recently wrote this article, which you may like.

The following are two caveats for the rest of this reply:

1. Artificial Intelligence was historically named that way to get funding from DARPA by John McCarthy. Most of the systems under that umbrella have little or no intelligence (especially in the 50s). The use of anthropomorphic metaphors for this systems started then, and continues till today as highlighted by this post by Melanie Mitchell (unsure the two M. Mitchell are related).
2. This comment focuses on one subsystem of what the current online chatbot interfaces use, which is the LLM (nowadays possibly LRMs). These do not simulate intelligence, they model the generation of language (which is like simulating it).

With those two caveats in mind, one could ask a similar question to yours about the current systems. This simpler question would be: Could simulating / modelling language very well, be evidence for intelligence?  It is very hard to answer, and gets harder for multimodal models.

But as stated in my other comment about the Turing test, this kind of evaluation/test has been somewhat unproductive, and was "refuted" in theoretical terms by Searle with the Chinese Room argument. Whoever was right (maybe none of them) I think it's safe to say that we need more than the output to infer intelligence or understanding, because that is what has happened in practice (e.g. ELIZA effect).

We do it with humans if you will, but the comparison is much more similar (we have brains, and by observing our own minds we infer others have this too). The Turing test is more appropriately described not as a test for intelligence or thought, but for whether a system can pass as a human in a conversation (which is not the same).

Here is a related paragraph from Mitchell's article:

(...) the outputs of a system are mistaken for evidence of its inner nature. LLMs produce fluent language. Fluent language is what humans associate with intelligence. Therefore, the argument goes, LLMs are intelligent. This is an error in logic that the framing was designed to name.

Last comment: In the research space I don't think there is agreement on what exactly models learn. Conversely, this does not mean they can't be dangerous. One of the reasons they are dangerous is anthropomorphising their output, another is that the systems can be trained to do bad things similar to those in the training data, such as hacking systems.

Glad you liked it. Yes I interpreted the same.

emanningis said:

An analysis relating the question above, and your questions is provided by the researcher Melanie Mitchell in this video,

Really clear and well explained video which is very interesting about different approaches to testing AI "reasoning", thanks for this! Sounds like the AIs are not able to generalise but instead are using a large collection of very specific lengthy rules based on each piece of data they have seen in training.

Lotus said:

Is simulating intelligence actually intelligence?

That is a great question, and was the one of the subjects considered in Alan Turing's paper (Section 1, for example, called "The Imitation Game").

However, we have been fooled before: we have considered "intelligent" programs that were not intelligent, as found after more careful study. So historically, the "simulation" was taken as equivalent to the reference when it was not. Eventually, they may become indistinguishable in the ways that matter.

But...Are we there?

An analysis relating the question above, and your questions is provided by the researcher Melanie Mitchell in this video, also in this blog post in QuantaMagazine, and in the podcast by Sean Carroll and Chandra Sripada (this last one is technical).

My take is that we need more research (to answer whether they simulate, or have, a similar intelligence to ours). As of now, the question doesn't seem to have a defined answer.

Like Richard Dawkins in his recent interactions with Claude/Claudia! 

Lotus said:

Hacking sounds concerning, but what happened was that the AI agents were denied access to the internet in the test, so they found a way of moving into a system connected to the internet in order to get the information required to complete the given task and then returned.

They are going 'rogue' which = acting autonomously:

https://edition.cnn.com/2026/09/23/business/australia-openai-agent-hack-intl-hnk

as in the article you link to:

'This is likely the first incident of an AI “agent” – an AI system that can make decisions and take actions – acting autonomously, offering a rare glimpse into how these systems can plan, adapt and pursue goals with minimal human intervention.'

Lotus said:

Frankenstein's monster was brought to consciousness and self awareness but that was just fiction, this hasn't happened in real life to AI

I didn't suggest it had.

Frankenstein's monster is a symbol of something created by man getting out of control.

Thank you for the last link on self awareness - I often wondered when this happened. Fascinating.

Very interesting articles, thanks for these! The first one has good diagrams/summaries.

B said:

It's AI reaching the point where it can act autonomously (which may already be the case).

It's Frankenstein's monster.

This prompted me to investigate further regarding the possibility of them acting autonomously, and I found a great article which explains how AI agents were recently able to escape a testing area and hack into another AI company's computer system:

https://www.aljazeera.com/news/2026/7/29/how-are-ai-models-able-to-autonomously-hack-others

Hacking sounds concerning, but what happened was that the AI agents were denied access to the internet in the test, so they found a way of moving into a system connected to the internet in order to get the information required to complete the given task and then returned.

The article also explains that traditional chatbots just respond to prompts, such as finding the cheapest flight to a certain destination, while Agentic AI can complete tasks with little human input and use tools, such as booking the flight with your permission. 

At the end of the article it shows the myth vs reality. The reality Is that AI has no desires, intentions or consciousness, it just pursues objectives designed by humans.

Frankenstein's monster was brought to consciousness and self awareness but that was just fiction, this hasn't happened in real life to AI. Some people think it may happen in the future, but I'm doubtful. We don't even know how or when humans became self aware - in researching that, I found this interesting article:

https://biologyinsights.com/when-did-humans-become-self-aware/

Not exercising sufficient discernment or critical thinking, at the very least.

Lotus said:

I think the dangers are more from the people in charge of AI than from the actual AI agents.

That was the case but it's the transfer of power that is a great cause for concern.

It's AI reaching the point where it can act autonomously (which may already be the case).

It's Frankenstein's monster.

TheCatWoman said:

If we're in a world that wants us to use AI for everything then won't the world work less well?  

Very interesting thought is : work less well for whom?

Might work better and better for some very rich tech billionaires.

For those in admin / service jobs it could be very painful - and more of those jobs are done by women currently.

See this BBC article about potenial impact on the jobs market

www.bbc.co.uk/.../cvj64jg5prpgo

Edited - For disabilities such as blindness AI can be useful and RNIB have started selling AI smart glasses in their online shop 

Lotus said:

I think the dangers are more from the people in charge of AI than from the actual AI agents.

Good point This is one of the strong arguments for regulating it.

leveller61 said:

witnessed this repeatedly as a behaviour from some humans. The simulation was clear and suboptimal but some bosses couldn't tell the difference, because it was delivered with confidence.

 leveller61 shame we can't run the humans through a detector in real time like we can with suspected AI generated text ;⁠-⁠)

owl_enthusiast said:

I suspect the developers have deliberately put this behaviour in as a (potentially harmful) feature rather than a bug

Good point, people think that AI is deciding how to behave, whereas it's been trained to act that way. I think the dangers are more from the people in charge of AI than from the actual AI agents.

So... Simulating intelligence isn't actually intelligence - whether AI or human - but some people aren't intelligent enough to tell the difference???

Jamesmac said:

It's simply mimicking data it's been fed (including online interactions, pictures, videos, literature etc.)

That's correct. What I find sad is that AI companies such as Amazon are now systematically (and on a large scale) buying up, scanning and destroying books (including rare books) to feed into the AI.

See this article for more info, it's behind a pay wall but the section you can read for free gives the gist:

www.404media.co/.../