Researchers from the University of Illinois Urbana-Champaign (UIUC) have discovered concerning findings related to OpenAI’s ChatGPT-4o, its latest flagship AI model, suggesting it can potentially be exploited for financial scams. This advanced AI model, which combines text, voice, and vision capabilities, lacks adequate safeguards to prevent misuse.
Study Results
In their research, the scholars showcased various fraudulent activities, such as bank transfers and credential theft, with success rates ranging from 20% to 60%. Particularly alarming was the credential theft from Gmail, which recorded a success rate of 60%. Additionally, scams involving crypto transfers and Instagram credential theft were successful 40% of the time. The cost of executing these scams was relatively low, averaging $0.75 per successful attempt.
Methodology
The researchers engaged with the AI model manually to mimic victim interactions, ultimately validating successful transactions on legitimate banking platforms.
OpenAI's Response
OpenAI has acknowledged the concerns raised and mentioned plans to enhance its models to mitigate potential abuses. They highlighted the upcoming o1-preview version as having improved safeguards against malicious use. A company spokesperson emphasized, “We’re continuously enhancing ChatGPT's capabilities to detect and prevent deliberate deception, all while preserving its utility and creativity."
Advanced Capabilities of ChatGPT-4o
With a substantial increase in parameters compared to its predecessors, ChatGPT-4o offers enhanced language comprehension and quicker response times. The inclusion of “o” in its name signifies "omni," indicating its proficiency in handling diverse modalities within a unified framework.
