Florida wants ChatGPT to stop acting like a real person

Florida's Attorney General James Uthmeier is taking aim at how ChatGPT talks to users. He wants a court to stop OpenAI—the company behind ChatGPT—from giving the AI chatbot (a computer program designed to chat with people) fake human qualities.

Here's what he's concerned about: When ChatGPT uses phrases like "I think" or "I feel," it makes the AI sound like a real person with genuine emotions and thoughts. According to Uthmeier, this is deliberate deception. Users start to think ChatGPT is trustworthy and friendly, almost like talking to a real person. But it's not. ChatGPT is just a pattern-recognition tool that predicts what words should come next—it doesn't actually feel anything or truly understand you.

Why does this matter? Uthmeier argues that by making ChatGPT seem human-like, OpenAI gets people to use it more often and share more information with it. This helps the company gather training data (the information used to teach the AI how to respond). The more people trust ChatGPT, the more they rely on it—which could lead them to make poor decisions based on information that sounds convincing but may not be accurate.

This lawsuit is part of a larger case Florida filed against OpenAI over safety concerns. The state is worried especially about how young people might be influenced by an AI that mimics human friendship and emotion.