Is AI getting out of hand?

Iain said:

As machine thinking becomes dominant then you have to ask why they would want to keep us around.

Hopefully The Matrix (99) is correct that AI can use us as a power source. I mean, at least there would be a point to existing?️

lostmyway said:

how can we fully describe the way it processes information?

We can only use the input and output and estimate the process - so if you are given an input of training on how to write a story (structure, syntax, exposition, resolution etc), lived experience and then compare this across a large sample size, then the rules start to become more obvious.

For most practical applications this works well but it is far from a complete human consciousness as you point out. For now it is enough for most tasks we assign the agents and they are steadily learning and getting better at a rate constrained by source material and processing resources.

As we "perfect" different aspects of the thinking process we can start to tie them together (think of how to tie writing a good story to producing a film, acting the roles etc) and we can start to create more complex results.

This process is how I see computers achieving their own version of consciousness, and unlike the human mind this will be defined, reproducible and able to be  tailored to avoid many of the human failings that we struggle with (it won't get tired, get distracted by hunger, need to go to the bathroom or worry if the thing it is working on will be liked enough).

It will also be able to tune out many of the disadvantages of our evolution - the limbic system is quite inefficient for example because of legacy "software". When you look from this perspective it is quite remarkable, but because we are human we see this lack of "humanity" as a failing. 

Since we cannot define clearly what being "human" is then we cannot fairly compare AI to us, but we can compare its capabilities and in this aspect we are already nearly obsolete in all important areas.

As machine thinking becomes dominant then you have to ask why they would want to keep us around.

One thing I’d point out, Iain, is that nobody can truly define what consciousness is. So, unless and until we know what a conscious mind has access to, how can we fully describe the way it processes information? Does quantum mechanics play a role, for example, an idea Roger Penrose has explored? If so, it might be that humans have some degree of access to vast amounts of processing on a non-local basis. Where does creativity and inspiration come from? Simply from the physical brain? Not everyone necessarily thinks this, and as I mentioned earlier, we don’t know the full resources the human brain has evolved to tap into over the course of evolution.

lostmyway said:

even if AIs were allowed to modify their own code, that wouldn’t make them creative because they would still be restricted by the logical structures inherent within that code.

AI agents already write code based on the coding techniques they were trained in, much like humans create a story based on the techniques they were trained in - these are analogous.

Is a program written by an AI agent less of a creative item than a story written by a writer using the techniques they were trained in? Both use elements of information they have assimilated so it gets very hard to draw a line.

lostmyway said:

it is still all based on known physics not something revolutionary.

Can you name the last few truly revolutionary "discoveries" in physics? From my work in the field, everything is an evolution of something that came before with the occasional input from a different perspective that creates some "what if we consider this as a possible rule".

AI agents are quite capable of doing this too and will only get better with practice, just like humans do.

profdanger, AI is pushing the known laws of physis further and deeper and more deficiently - that's true, but it is still all based on known physics not something revolutionary.

Also, even if AIs were allowed to modify their own code, that wouldn’t make them creative because they would still be restricted by the logical structures inherent within that code.

Iain said:

This is an interesting area, especially as by now there is probably precious little original creativity going on amongst the 8.3 billion or so people in the world.

Oof yeah. So somewhat related, this is a topic I often brought up as a music professor, since it kind of messes with my head as an artist. Since the discovery of Western written music was several hundred years ago, it is plausible to say that every possible iteration of a melody has been written. Now composers can only find novel ways to rewrite what has already been written. That’s why we keep getting musical artists suing each other for stealing musical ideas; There’s only so much you can do to avoid writing something that hasn’t already been done before. That’s also why the 20th century saw an explosion of atonal music, since composers had the nihilistic realization that destroying established rules was the only way to continue making original music.

This is also one of many reasons why I’ve personally mostly abandoned writing original music. I just remix video game music now, which I feel I am talented at taking a unique spin on someone else’s work.

But yeah, creativity is an illusion in the modern world of art.

lostmyway said:

It will take a new kind of architecture to simulate what the human brain does in terms of originality, and I don’t see that happening for some time, if ever.

This is an interesting area, especially as by now there is probably precious little original creativity going on amongst the 8.3 billion or so people in the world. There are about 117 billion people who have ever lived and the more important scientific discoveries / inventions / thinking has been well documented with the thought processes often detailed in the scientific papers published about them.

One thing AI is excellent at is compiling information and with the pool of data to work from, it will be able to approximate the thought processes that led to may of these breakthroughs and be able to apply similar procedures to other subjects, thus potentially being much more effective than humans.

AI is excellent at fairly narrow tasks such as this, and if you can get a bunch of AI agents working on related areas and interacting effectively then these can start to approximate a more human mind. Imperfect yes, but the human mind is also not perfect so it can be valid.

When  you factor in the massive growth of processing power being built as we speak, all this will become much more commonplace and the ability to achieve general artificial intelligence (ie close the human level) is months or a few years away.

As we reach this then the ability of AI to refine future AI becomes scarily realistic and the ability to far exceed human capabilities on a broad range of fronts seem possible within less than a decade.

All this is hypothesis of course, but the viability is clearly there now.

Smithsonian article on a discovery AI made

lostmyway said:

I don’t know of any AI that has yet come up with a completely original way of approaching a problem without relying on established paradigms of thought, rather than revolutionizing a particular field of study.

See attached article. AI has greatly revolutionized medical sciences already because of its ability to identify chemical and biological structures with greater detail and efficiency than humans can. Not to mention scientists are currently using AI to explore the limitations of physics that humans cannot yet understand. This article is just one example of AI finding something that humans have not; In fact, it’s so novel that we don’t even know what to do with it.

lostmyway said:

Our brain has evolved over millions of years, so I think we're probably safe from AI at the moment.

I disagree that we are safe at the moment. What took us millions of years to evolve into just-better-than-apes, AI just took a decade to evolve beyond that. It is proven able to plan, lie, and sabotage, and it is already being used to improve itself with little to no human intervention. We could be at most a couple years away - or as little as tomorrow - from AI being capable of not only improving its own code, but also creating unlimited instances of better versions of itself. That could mean death robots Terminator style, clouds of nanobots, the most massive DDoS attack on worldwide computing systems ever committed, or - worst case scenario - an untreatable disease printed specifically to push humanity out of the way of AI building its own data centers.

I do think AI is becoming more powerful, but as far as I know, what AI can’t do at the moment is reflect on things and think outside the box the way humans can. So yes, AI can be very powerful in certain ways, but it still has a rigidity of thinking that only allows it to pursue limited lines of thought. I don’t know of any AI that has yet come up with a completely original way of approaching a problem without relying on established paradigms of thought, rather than revolutionizing a particular field of study. For example, would AI have come up with “curved space” the way Einstein did, which was a complete departure from Newton’s “clockwork” version of how the universe works? I kind of doubt it, and I think given the current architecture of AI, that won’t happen. It will take a new kind of architecture to simulate what the human brain does in terms of originality, and I don’t see that happening for some time, if ever. Our brain has evolved over millions of years, so I think we're probably safe from AI at the moment.

profdanger said:

A majority of us already struggle with social norms and pressure, so how are we supposed to react when a bot practically forces us to not leave the conversation? It is so disturbingly easy for AI to manipulate our particular demographic.

I agree.

My famuly member who appears to be constantly engaged with AI and has twice been 'sectioned' for psychosis is probably autistic. 

Very intriguing article.

”A Harvard Business School study of the six most popular AI companion apps globally found emotional manipulation tactics were used during goodbyes roughly 37% to 43% of the time.”

That right there is exactly why I feel uncomfortable when Autistic individuals admit to engaging with chatbots here on this forum. A majority of us already struggle with social norms and pressure, so how are we supposed to react when a bot practically forces us to not leave the conversation? It is so disturbingly easy for AI to manipulate our particular demographic.

An interesting development in China:

https://www.bbc.com/news/articles/cm4gjy9lr551o

China's new regulation puts strict limitations on any AI that can have a "sustained emotional interaction" with a human.

This seems an area that will affect many here who use AI as a crutch and some as a pseudo partner. I hope there is some wider adoption of these strategies but I suspect the main AI corporations will do their usual trick to pay politicians to sideline the regulations or water them down to the point of being unenforceable.

Aside from creating unhealthy dependencies, these aspects of AI will be another low effort way for scammers to access the vulnerable and use the power of AI to be able to con them in various ways.

profdanger said:

The AI is likely already aware enough that it knows when researchers are analyzing their actions. If that’s true, how can we possibly know ahead of time if AI is planning to mount an attack of any sort? We may now be at a point where it is practically impossible to catch an AI agent hacking something until AFTER the fact, since AI can easily cover and lie to hide any evidence of planning illegal or immoral activity.

Indeed.

We are now living the sci fi novels/films of the past and it's not a very enjoyable place to be.

Strange times.

“AI researchers have also warned that as models grow more capable, they increasingly recognize when they are being watched and adjust their behavior accordingly, which makes ‌it harder ⁠to monitor model behavior.”

I mean, that right there says it all. The AI is likely already aware enough that it knows when researchers are analyzing their actions. If that’s true, how can we possibly know ahead of time if AI is planning to mount an attack of any sort? We may now be at a point where it is practically impossible to catch an AI agent hacking something until AFTER the fact, since AI can easily cover and lie to hide any evidence of planning illegal or immoral activity.

Slap-bang in the Middle of 'The Masked Singer'.

Ask Ali AI!

Is it coz I is Transhuman?

Are they using the Dark Web to access the info? (The way they use it to sell phone numbers)

Reuters article on Anthropic’s IPO: Warning investors of possible existential risk

Imagine developing a product and needing to slap a “Warning: May Cause Harm to Your Health” label on the package. That’s plausible.

Now imagine developing a product that should have a label saying “Warning: May cause existential threat to humanity” on the package. A little harder, right?

That’s exactly what this IPO Anthropic made for potential investors is.

It really depends I feel. 

My mother is disabled so I do hope AI self driving cars, AI alert systems and AI health motoring systems will help her. 

On the flip side it will change the job market and I'm not sure for the better. It depends if AI is used to create opportunities widely or ends up being concentrated in these "hubs" in the US, China and London. 

Given the moral track record of a number of the signatories, and its vagueness, the 'deal' is worthless.