The emerging deepfake crisis, powered by AI

Originally Posted on The Triangle via UWIRE

Photo by Navneeth Nalajala | The Triangle

In the past month, X’s proprietary AI chatbot, Grok, produced thousands of non-consensual images. Over nine days since Jan. 22, users had Grok generate and post over 4.4 million potentially compromising images of young girls and women.

Following Elon Musk’s launch of “Grok Spicy Mode” last summer and the AI image-editing feature, images on X now show photographs of women at various levels of undress. Users are asking Grok to enlarge the body parts of photograph subjects and adjust their clothing to make them appear highly revealing.

A worsening issue is the distribution of Child Sexual Assault Material by AI since 2023. The National Center for Missing and Exploited Children has reported that its online abuse reporting system saw a 1,325 percent increase in reports involving generative AI. The Internet Watch Foundation shared that it found sexualised imagery of young girls on dark web forums, where users claimed Grok created them. This massive surge of CSAM brings this imagery onto public sites, with younger X users now victims of digital abuse.

As of early January 2026, Grok’s image generator and editing functions are limited to users who pay for premium features. This choice was criticized by many, who described it as the monetization of abuse rather than fixing core issues, such as the rejection of image removal requests with the justification that neither Grok nor the users have “broken safety policies.” This change was implemented following global outcry and pressure from online users and authorities. 

X is now being investigated by multiple countries for distributing explicit material. In an escalation of a French investigation into CSAM and sexually explicit deepfake distribution that began in January 2025, authorities raided the Paris headquarters of X. Following France’s lead, this past week has revealed that the UK Information Commissioner’s Office opened a formal investigation against X, and in the U.S., the Take It Down Act takes effect in May 2026 and criminalizes the sharing of these images, compelling platforms to take down flagged nonconsensual imagery within 48 hours.

While AI has demonstrated groundbreaking potential, the frequency with which its use puts vulnerable people at risk, especially in the context of abusive misuse of Grok, warrants discussion. This expanding use of generative AI to exploit women and children facilitates the mainstream circulation of CSAM and has demonstrated a void in policies and enforcement; in the meantime, X’s solution of paywalling Grok has done little to address these issues.

Read more here: https://thetriangle.org/news/national/the-emerging-deepfake-crisis-powered-by-ai/
Copyright 2026 The Triangle