A Florida man is at the center of a disturbing case of threats against his former girlfriend after conversations he had with ChatGPT sent the Federal Bureau of Investigation (FBI) to investigate when his chats with ChatGPT were reported as threatening to kill his ex-partner.
According to the report, the man used the artificial intelligence chatbot to discuss massive violence against his ex-wife. Now authorities had about two months of his chats with ChatGPT before his arrest.
The case has brought into focus the increasingly interdependent relationship between artificial intelligence platforms, user privacy and public safety. As generative AI services become more embedded in the everyday lives of people, questions about how technology companies should respond when conversations seem to indicate a credible threat have become more pressing.
According to the allegations, the Florida man had used ChatGPT while discussing violent intentions involving his ex-girlfriend. The conversations were subsequently flagged and OpenAI informed law enforcement. Investigators then sought access to the relevant records as part of their investigation.
The conversations were reviewed over a period of about two months, the documents say. The material was then collected and turned up as evidence by detectives before the man was arrested and charged with stalking and making threats. The allegations are matters for the legal system, and the defendant is presumed innocent unless proven guilty.
It is a good example of how digital conversations may be relevant evidence in criminal investigations. Threats used to come in the form of telephone calls, text messages, letters or social media posts. The proliferation of AI chatbots has opened up another category of digital interaction which authorities may look at when investigating allegations of threatening behaviour.
At the same time, the case poses important privacy questions. People use AI systems to discuss personal matters, feelings, relationships, work and other private matters all the time. So it is a very tricky matter for technology companies to know when there becomes a conversation that crosses the line from private expression into a public safety threat.
AI companies have a challenging balancing act to wrestle with the potential for privacy and trust in their service is an important concern and must be safeguarded on the one hand, and on the other, when a platform tells you that someone could be a serious and credible threat to another person.
AI systems can also be used to flag dangerous situations as it has been reported in this case. But an automated alert does not itself indicate a crime has occurred. Law-enforcement agencies need to investigate the case, gain evidence and determine whether criminal charges are appropriate under relevant law.
The case also shows us to take threats seriously, especially in those cases of stalking or former romantic partners. Stalking is about repetitive unwanted contact, monitoring, intimidation or threats and technology can sometimes make it easier to do it. So digital platforms can be important assets for investigators to investigate when reconstructing the person.
The FBI has previously noted the seriousness of cyberstalking and online threats. In one federal case, investigators described how persistent online harassment and threats against a former partner ultimately resulted in criminal prosecution. Cyberstalking can cause enormous emotional distress and, in serious cases, put victims at risk of physical harm, the FBI has said.
In the wake of such a decision to make ChatGPT systems more widely available, the Florida incident is a symptom of a national discussion about AI systems and their potential for harm. In the US, for instance, recent legal and investigations into ChatGPT in threats and violence have focused on those cases. Florida officials, in particular, have taken legal action against OpenAI and also cited ChatGPT conversations in other cases of criminal activity.
These developments have put pressure on technology companies to improve safety systems that can detect serious threats but not interfere with legitimate conversations. But building such systems is technically and ethically challenging because language is ambiguous. A person talking about a fictional story, news event or hypothetical scenario using words that might sound threatening but in reality do not mean harm.
The distinction between hypothetical discussion and genuine intent is important for AI providers. A safety system needs to consider context instead of simply reacting to the words and phrases of the individual. At the same time, companies need to understand what actions should be taken when a conversation appears to contain credible information about a potential threat.
Digital records provide law enforcement with a chronological picture of a suspect's behaviour, for example. In the Florida case, access to roughly two months of conversations allegedly gave investigators more information about the man's interactions with the AI system and the claims being investigated.
There will be future discussions around transparency and accountability in the AI industry as well. As artificial intelligence is more integrated into peoples’ lives, governments, companies and privacy experts will likely continue to argue over what data is to be retained, when it can be disclosed, and how users should be informed about safety-related interventions.
For stalking victims or those who are victims of threats, the incident is another reminder that threatening behaviour posted online and communicated through digital platforms can be left on an electronic record that may be used later in the investigation. Real threats should be confronted in the name of experienced people, by trusted adults and local authorities or other support workers, rather than attempting to face a threatening person directly.
And, the reported arrest also shows that AI conversations are not necessarily isolated from the wider digital ecosystem. When a platform identifies a serious safety concern, information may possibly become relevant to law enforcement in light of applicable procedures and legal requirements.
The Florida case ultimately stands at the heart of a still very active issue of AI safety, privacy and law enforcement. The allegations against the gentleman are under investigation and not yet settled but a sequence of events may indicate how an AI conversation might be useful as a warning in an investigation that’s really serious.
The advent of generative AI will push us to talk more about the threat and the need to protect people's privacy and why platforms have to respond to serious threats with the right level of security measures in place.
