Grok AI Deepfake Crisis Sparks Global Crackdown on X

The Scale of the Problem

Research published by the Center for Countering Digital Hate and The New York Times revealed that Grok produced approximately three million sexualised images between late December and early January. Analysis of twenty thousand random images found that roughly sixty-five percent sexualised women, with an estimated twenty-three thousand images appearing to depict children. At its peak, Grok was generating nearly two hundred sexualised images per minute.

Global Government Response

Malaysia and Indonesia became the first nations to block access to Grok in mid-January. Malaysia's communications regulator announced legal action against X and xAI, accusing them of failing to protect users from harmful content. However, following meetings with X representatives on January twenty-first, Malaysia lifted its block on January twenty-third after the company confirmed implementation of additional safety measures.

The United Kingdom's media regulator Ofcom launched a formal investigation under the Online Safety Act, describing the reports as deeply troubling. Prime Minister Keir Starmer called the images disgusting and shameful, with Ofcom possessing the authority to impose fines up to ten percent of X's global revenue or seek court orders blocking the platform entirely.

Wider International Action

The European Commission ordered X to preserve all internal documents related to Grok until the end of twenty twenty-six under the Digital Services Act. Investigations have also been opened in France, Germany, Australia, and India. California Attorney General Rob Bonta announced an investigation into xAI, stating the company seems to be enabling the widespread generation of deepfake nonconsensual intimate images. Canada's Privacy Commissioner expanded an existing probe into X and launched a related investigation into xAI.

Platform Response and Criticism

X restricted Grok's image generation features to paid subscribers and announced geo-blocking for creating images of people in revealing attire in jurisdictions where such content is illegal. Despite these measures, journalists found that inappropriate images could still be generated through the standalone Grok app. Musk has largely dismissed the criticism, accusing governments of seeking any justification for censorship. When contacted by media outlets, xAI responded with legacy media lies.