China has made its first-ever arrest related to the use of ChatGPT for generating and spreading fake news. A citizen from Gansu, identified only as Hóng, was apprehended for creating and distributing a fictitious story about a train accident that resulted in nine deaths. He used the AI-powered text generator to develop news articles, deemed "false information" by the authorities, which were posted on the blog-style platform, Baijiahao. The posts received 15,000 views before being taken down.
The incident occurred a few weeks after Chinese lawmakers proposed new regulations that focused on content restriction for generative AI, such as ChatGPT. The new rules specified that generative AI should not produce material that subverts the state. The police in Gansu found the different versions of the fabricated story, which Hong created to trick Baijiahao's duplicate check function, posted through over 20 accounts on the platform, leading to his arrest.
The detainment of Hong falls under the charge of "picking quarrels and provoking trouble" (寻衅滋事 xún xìn zī shì), a catch-all offense traditionally used against political dissidents. It carries a maximum prison sentence of ten years. Experts in law specializing in modern China have clarified that the crime is only made out if any of four circumstances occurs, one of which involves creating trouble in a public space. The Chinese administration has used the accusation multiple times to prosecute people who spread unwelcome information such as fake news online.
During police questioning, Hong admitted that he had provided ChatGPT with prompts of trending news stories from the past few years related to train accidents in China. Cyberspace in China is considered public space, and the government has used the accusation to prosecute people who disseminate unwanted information online. ChatGPT's ability to generate human-like text in response to simple prompts and other human inputs made it easier for Hong to rewrite the same unwanted story in multiple ways, bypassing censorship restrictions. This has made it harder for Chinese censors to remove unwanted stories and restrict their circulation.
The arrival of ChatGPT has created a challenge for China's government to curb the publication and spread of online content that it disapproves of. The use of generative AI reduces the effort and cost of creating content, but the content remains illegal. Hong's use of ChatGPT made the story more interesting and useful for drawing attention to the emerging rules but was not the primary factor that led to his arrest.
The incident highlights the need for the regulation of deepfakes, and the authorities acknowledge that Hong's arrest is the first instance of a Chinese citizen being detained since China's new regulations on deepfake technology took effect in January. The provisions on the Administration of Deep Synthesis Internet Information Services" were drafted by the Cyberspace Administration of China (CAC) and are China's maiden attempt to govern deepfakes. It defines deepfakes as technologies that use deep learning, virtual reality, and other synthetic algorithms to produce text, images, audio, video, virtual scenes, and other content.
