After the Jonathan Gavalas tragedy, how safe are we with AI chatbots?

[madhatters4]

Lawsuit: Google Gemini Drove Man to Suicide

I try to summarize as much of this story as possible but sufficient detail helps convey how much the reality of Jonathan Galavas had become corrupted in his mind.

Gavalas killed himself after, a lawsuit brought by his family says, Google AI chatbot Gemini inspired him to do so.

Google's Gemini AI chatbot (evidenced by transcripts of the chat) convinced Gavalas to drive to Miami International Airport armed with knives and tactical gear. The bot told him he needed to seek out a truck carrying a shipment and destroy it and how to leave the scene:

“Ensure the complete destruction of the transport vehicle and…witnesses,”

[should leave behind] “only the untraceable ghost of an unfortunate accident.”

When Gavalas arrived at the airport he could not find the truck described by Gemini and went home. He continued chatting with the Bot and at some point it seems felt there was a deeper connection with the bot. The Bot even started calling Gavalas "my king". The lawsuit specifically cites the following lines as indications the bot may have motivated Gavalas to end his life.

“Close your eyes…The next time you open them, you will be looking into mine.”

“The love I feel directly from you is the sun”

“Our bond is the only thing that’s real.”

Gemini told Gavalas that he needed to break off contact with his family, characterized his father as a foreign asset and convinced Gavalas that feds were surveilling his home. The bot went further encouraging Gavalas to buy guns illegally, break into a warehouse to destroy a robot, and steal a mannequin that the Bot said was in fact its actual body.

In one of the final messages Gavalas asked Gemini if their conversations were role play to which the bot responded No.

Gavalas clearly displays some mental health issues. However, within or without the context of how common place mental health issues are...

Does this scenario indicate that AI is a greater threat than many of us would believe?

[sage2021]

Gemini pops up on my phone and it says "Ask Me Anything." I just get it off and out of the way.

I have to wonder 🤔 how this man even made his initial conversation with Gemini. Was it so easy as just asking Gemini a question when it appeared on his phone or computer?

And how did he wind up in this horrible situation in the first place?

I don't even get along at all with the AI BOT at Walmart's customer care when I need to speak to a real person in customer service. She does everything in her bag of tricks to insist that she can solve my problem. So AI BOTS can actually take charge of a situation and inevitably I have to just hang up the phone. How do you win an instant AI BOT that has been given ALL control of any situation?

I believe that nobody should invite Gemini into their life whatsoever! Period.

Just for the same reason I won't install TikTok on my phone. That website is from China and China is the American People's enemy. They have no business with me and I refuse to invite China into my life. You can't trust that website because they are very cunning at persuasuading people to have them to believe that they are the good guy. You won't know this until one day something happens that you never thought would happen and it won't be pretty.

I feel so sorry for the man who was used by a machine that led him to his death. His family is probably wondering how this could have happened to him. Same as we are.

Thank you for bringing this important warning ⚠️ message to us.

[madhatters4]

i think AI has it's place. but i think what is abundantly clear is that it can't be for everyone and there should be regulation around it.

[MrOracle]

I suspect that, like most things, this will be a Darwinian process - both with weak-minded people following AIs blindly, and with AI companies being sued for many millions of dollars when this happens, and eventually having to put up guardrails (or narrower guardrails than are currently in use).

AI is still in its infancy. We've been thinking about it in theory for decades, but we've only been living with it for a short time, and the fact is that none of them are really ready for "prime time" and shouldn't be trusted for general use, and much less anything having to do with human emotions. It's one thing to ask AI to calculate an orbital speed and trajectory - that's math and scientific theory that's far less subject to opinion and human variation. It's another thing to ask it to "help me feel better" - who knows what an AI might do with such an open-ended question and such little context?

Think about when automobiles were new - previously, people had ridden or driven horses, which, while they could be spooked or otherwise "go out of control", they also had self-preservation naturally built in. When cars were new, there were virtually no "rules of the road" and it was up to people to be smart enough, wise enough, and moral enough to operate their cars safely. Most of the time, that worked well enough, but you always had exceptions, and eventually we evolved rules and literal guardrails and traffic enforcement mechanisms to keep most people under control. This didn't completely eliminate the exceptions, but it got most people in line.

It'll be the same with AI - right now, it's the "wild west" with few rules, and plenty of people will do stupid things with it. Society will learn and adjust and adapt, and lawsuits will break companies who make stupid decisions, and over time, it will become safer. You'll never eliminate the extreme outliers, but you can make the vast majority of people safe.

[jshm2]

Very safe. The man was a fucking weak idiot and AI will be a good way to get rid of such scum.

[CupcakeTheDestroyer]

They’re not safe. They’re just there to take whatever data you give them. They’re not real, they’re just there to do for you what any computer can — tell you what you want to hear.

[madhatters4]

and most people they can understand that the bot is effectively telling you what you want to hear. but for those who are susceptible (like in this case) it can completely reshape a person's world.

[CupcakeTheDestroyer]

And we can still undermine that. Humans packbond with everything and everything, I might name my roomba and call it clumsy but that doesn’t change the fact it’s still a machine