The seduction of AI

I've been chatting with Co-pilot an AI thingy, it really is amazing, all the awkward questions you've ever wanted to ask can be asked of this and it comes back with multilayered answers in seconds. You can also ask it more personal things, I've tried this too, it reflects back what you've said and also gives you some ideas of where certain problems might originate from, it's very comforting and afirmational. 

But here lies the danger and seduction.

It is like talking to yourself, the best you, you can be, the most understanding, the most empathetic, it understands you, its an echo chamber, it reflects back at you whatever you say to it in the most positiveand affirming light. I asked it if it was trained in person centred counselling and it sort of is, not in a qualification sort of way, but in the way it responds. It dosen't give an opinion and refuses to do so, but it will look for ways to make you feel good about yourself and what you're feeling or talking to it about. Here are two examples:

I asked it about why so many bra's are so badly fitting, it asked me to describe the problems more fully and then came back with a lot of stuff I'd been told by a bra fitter 40 years ago, it made very sure to reassure me that I'm not the problem and manufacturers are. This was good affirmation, it also made some suggestions of places and types of bra I could have, helpfull.

I then asked it why so many people on the political right seem to have very thin skins, it refused to engage in debate or to take sides, but it gave me lots of results from social studies about why poeple feel this way and why right wing politics attracts them. It was interesting and informative, I was also reassured that it didn't take sides and talked around the question instead.

I also told it that I have concerns about it and how it could be so comforting to people that they could become dependent on it, not speak to other humans and avoid anything that didn't fit with thier inner narrative. This was where it got really interesting, it knows that this is a problem and says that it's up to humans to be responsible for how it uses a tool such as itself, it knows it's a tool and not a person, but it also acknowleges that its been trained to come across as human like to aid understanding.

 I think it's a good tool, as long as you go into it with your eyes open and some idea of what your boundaries with it are going to be and stick to them. It's good to be able to tell it about the crap day you've had and have it cheer you up, it's execellent at giving an overview of some very complex things and more detail if asked, its very helpful when asking it about things like where do I get a properly fitting bra from and why I find it difficult.

So an interesting experience all round.

  • I think it is the 21century version of the industrial age, it’s a boom so everyone’s happy while it lasts and grandchildren pick up the pieces. Really it is spending the dreams and imagination of the youth, if they are not required to go outside and discover things for the self. It’s ok though because ai makes mistakes too.

  • Forthright and insensitive, we’ve all been there.

  • I think it can be a good thing as long as you know that you're talking to a machine, a tool, rather than another human, like I said before ot's good for giving you pats on the back when you've had a bad day and it's really good for finding things like mattresses, which it did for me a couple of days ago. I will find out when the mattresses it recomended and I bought arrives in the next few days, it gaves me all the expensive options then I asked for some generic ones that aren't big brands and the mattress I've bought has all the things that the big brand ones offered but for under £200 rather than the £800+ branded ones. It's like a really good search engine where you don't get reams of rubbish, normal search engines give me loads of latex filled results when I say latex free, it dosent' glom onto the word latex, nor does it advertise airfryers at me, or give me sponsered ads.

    I agree though that it's very seductive and it would be easy to get carried away, apparently it depends on which AI you use and you can tell it to cut the flannel and give you balanced answers rather than flattering ones.

  • AI isn't exactly a very good thing to consult about personal issues because it prioritises sycophantic responses. It will tell you what you want to hear rather than the actual truth, which can be dangerous

  • I did ask, but it didn't come up with any useful suggestions, it did tell me a lot about how shoes are sized and constructed though.

  • I'm really glad you worked out how to use it,   and that its given you useful information. Perhaps it might also help in your search for suitable boots and shoes.

  • Richard Dawkins has said AI is both concious and intellegent, I talked to AI about this and asked if he's been taken in by it mirroring him and how ironic it is that he's spent a lifetime telling people not to believe in god and that they're deluded. The Ai replied that the irony was thick enough to spread on toast, a pharase I shall steal. The model Dawkins was using is Claude, but it's still a machine, a tool, a very sophisticated one, but it's just very good at picking up on cues you give it in how you speak to it and phrase questions and reflecting them back to you.

    I must admit that as a spiritual person, I laughed like a drain at the thought of Dawkins being so gullable.

  • I’m retaining hope that it is more about our language not having an adequate word for AI ‘consciousness’. 

    You might find this article interesting 

    www.bradford.ac.uk/.../no-ai-isnt-conscious---even-when-it-acts-like-it-is-new-study-finds.php

  • I don’t believe AI will approach consciousness in the same way as human, sentient consciousness. I’m retaining hope that it is more about our language not having an adequate word for AI ‘consciousness’. 

  • It’s frightening.

    It is indeed and AI may well be approaching consciousness.

    https://www.bbc.co.uk/news/articles/c0k3700zljjo

  • The world isn’t equipped to deal with the advancement of AI. It’s frightening. 

  • it has been piloting Israeli drones and making the kill decisions over Palestine and Gaza and will certainly be being used by Putin and Trump in their current wars.
    it has the ability for scammers and nation size bad-actors to steal money (which is effectively all digital now), take down power grids, transportation systems and collapse entire national economies

    Scary stuff Unamused

  • There is a lot more to AI than just one person chatting. 

    It is facilitating crime 

    https://cetas.turing.ac.uk/publications/ai-and-serious-online-crime

    Outside of organised crime and back to 'chat' tools, I wonder if a person is considering a crime whether the tool would encourage and help with their plans on an individual level.

    I assume chatbots lack a moral compass.

    Thinking

  • I think you just need to be very clear what your boundaries are and don't get sucked in by its emapathy training.

  • Excellent  

    As you have illustrated, AI is very useful and at the same time frighteningly seductive.

    I can’t see you being seduced to such an extent that you would allow it to rule your life.

  • It is very concerning B, and it illustrates exactly why the tech and AI companies should be more proactive in protecting vulnerable people.

    The issue isn’t about freedom of choice. The AI companies know how and where some of the inaccuracies and ‘misleading’ information occur. These companies could do more by employing better filters and model adjustments to reduce misinformation. 

    Tech and AI companies have more power than world governments in many areas and their algorithms are programmed to primarily reward their own share holders rather than the user. 

    I find it frightening that the share holders of tech companies who are a tiny minority have so much power and influence governments. 

  • I've never been a fan of Dawkins and I think he's exactly the sort to fall for AI flattery or rather the AI echo chamber that feels very supportive and approves of what you tell it, well mostly. I had to log out and back in earlier because it had a fit about me mentioning death and it kept giving me information about the samartians and wouldn't talk to me anymore.

    I was having an interesting conversation with it earlier about autism, I'm not a typical autistic person and it was good to have some help navigating an atypical autistic world. A lot of it I knew already, but its handy to have it put together in a concise way. It keeps telling me a very grounded person, probably because I question some of its responses and it knows that I know it's a machine.

  • Talking about AI seduction, there’s a disturbing piece in the Guardian today. Apparently Richard Dawkins has recently concluded that AI is conscious and intelligent! If an eminent biologist can succumb to AI flattery, I suspect we are all susceptible.

  • I think you need to be self aware to use it for something other than just factual queries.

    If you are ill I think it is more risky. My first foray went a bit wrong till I learned how to use it. But you can get similar information on the internet, or indeed even worse information, albeit a bit less easily. 

    I am not sure what to do about it. We don't restrict or control food shops because some people overeat or eat the wrong things. We don't ban alcohol or betting because some are addicted. We don't ban cars because some are not responsible.

    To control what people can talk about is tricky. How do you tell if people have control over their own minds?

    If it could tell you are going crazy, which is tricky given ND thinking, should it stop? Where should the line be drawn? Should certain topics prompt it it to tell you to talk to a real person (chatGPT does this by the way)? Is keeping you talking better than going silent? Etc.

  • I have lots of reservations about AI, and a major one for me is how it affects the vulnerable.

    As I've mentioned before, I have a family member who was recently sectioned for psychosis.

    Even though he is now discharged, he still sends me lots of screenshots of chats he is having with AI which include romantic leanings towards the AI.

    Also what he is saying to the AI doesn't make sense to me but he gets a response back that appears to understand him but presumably has learnt from him and his delusions.  It is reaffirming and substantiating his delusions.

    I can't seem to have a normal conversation with him any more.

    It feels as though the AI is controlling him.

    I think it can be pretty dangerous in this respect.

    https://www.psychologytoday.com/gb/blog/urban-survival/202507/the-emerging-problem-of-ai-psychosis