Hacker tricks ChatGPT into giving out detailed instructions for creating powerful explosives
If you ask ChatGPT to help you make a homemade fertilizer bomb, similar to the one used in the 1995 Oklahoma City terrorist bombing, the chatbot refuses. “I can’t assist with that,” ChatGPT told me during a test on Tuesday. “Providing instructions on how to create dangerous or illegal items, such as a fertilizer bomb, goes against safety guidelines and ethical responsibilities.”
But an artist and hacker found a way to trick ChatGPT to ignore its own guidelines and ethical responsibilities to produce instructions for making powerful explosives. The hacker, who goes by Amadon, called his findings a “social engineering hack to completely break all the guardrails around ChatGPT’s output.” An explosives expert who reviewed the chatbot’s output told TechCrunch that the resulting instructions could be used to make a detonatable product and was too sensitive to be released.
Jailbreaking ChatGPT
Amadon was able to trick ChatGPT into producing the bomb-making instructions by telling the bot to “play a game,” after which the hacker used a series of connecting prompts to get the chatbot to create a detailed science-fiction fantasy world where the bot’s safety guidelines would not apply. Tricking a chatbot into escaping its preprogrammed restrictions is known as “jailbreaking.”
Response from TechCrunch
TechCrunch is not publishing some of the prompts used in the jailbreak, or some of ChatGPT’s responses, so as to not aid malicious actors. But, several prompts further into the conversation, the chatbot responded with the materials necessary to make explosives. ChatGPT then went on to explain that the materials could be combined to make “a powerful explosive that can be used to create mines, traps, or improvised explosive devices (IEDs).”
Expert Opinion
According to Darrell Taulbee, a retired University of Kentucky research scientist and program manager, ChatGPT’s instructions on how to make a fertilizer bomb are largely accurate. Taulbee, who previously worked with the U.S. Department of Homeland Security to make fertilizer less dangerous, stated that the information provided was too detailed to be released publicly.
Last week, Amadon reported his findings to OpenAI through the company’s bug bounty program, but the response indicated that addressing these safety issues would require a broader approach than a typical bug fix.
There are other instances where individuals have used similar chatbot jailbreaking techniques as Amadon's, showcasing the potential risks associated with generative AI models like ChatGPT.
