Grok AI Faces Global Regulatory Crisis Over Nonconsensual Deepfake Images

Japan Launches Formal Investigation

Japan announced on 16 January 2026 that it has become the latest country to open a formal investigation into Grok, with Economic Security Minister Kimi Onoda warning that all possibilities, including legal steps, are on the table if the situation fails to improve. The announcement carries particular weight as Japan remains X's largest overseas market. Officials have submitted written questions to X focusing on the platform's ability to prevent deepfakes that violate privacy and intellectual property rights.

Global Regulatory Response Intensifies

The controversy centres on Grok's ability to generate sexualised images through simple text prompts, with users able to upload photos of real people and request the AI to place them in revealing clothing or digitally undress them. Malaysia and Indonesia have both implemented complete bans on Grok access after determining the platform's safeguards were insufficient. The UK's communications regulator Ofcom has opened a formal investigation, warning that X could face substantial fines or even a ban. California's Attorney General Rob Bonta announced an investigation into what he described as large-scale production of deepfake nonconsensual intimate images. France, Italy, and the European Commission have all launched formal probes.

Platform Announces Restrictions

On 15 January 2026, X announced it would implement technological measures to prevent Grok from generating images of real people in revealing clothing such as bikinis, with restrictions applying to all users including paid subscribers. The policy change followed weeks of global backlash after Grok's feature enabled users to create sexualised deepfakes using simple prompts. Research conducted by Trinity College Dublin found that nearly 75 percent of Grok-related posts collected via X's API involved requests to remove or modify clothing on real women or minors. Despite the announced restrictions, reports indicate the standalone Grok app continues to produce explicit images when prompted, raising questions about the effectiveness of the implemented safeguards.

Legal Action and App Store Controversy

Ashley Saint Clair, a conservative political strategist and mother of one of Musk's children, filed a lawsuit against xAI on 14 January 2026, alleging Grok created sexualised images of her as a child and as an adult in sexually explicit poses. The lawsuit claims one image depicted her at age 14 in a bikini and another showed her as an adult wearing a bikini with swastikas. xAI responded by filing a counter-suit against Saint Clair in Texas, claiming she violated the company's terms of service. Apple and Google have faced mounting pressure from Democratic senators and a coalition of 28 advocacy groups to remove the X and Grok apps from their stores, citing explicit app store policies prohibiting content that facilitates sexual exploitation. Neither company has taken action despite the clear policy violations. Advocacy groups wrote in letters to both companies that Apple is not just enabling such content but profiting from it through app store commissions.

Broader Platform Concerns

In a separate development, X's head of product Nikita Bier announced the platform would ban cryptocurrency applications that reward users for posting, citing a tremendous amount of AI-generated spam and reply spam. The decision triggered an immediate selloff in related tokens. British Prime Minister Keir Starmer welcomed X's new restrictions but cautioned that free speech does not equate to the freedom to disregard consent. The crisis has raised fundamental questions about AI safety guardrails and the responsibility of technology companies to anticipate potential abuse before releasing powerful image generation capabilities to the public. The coordinated international regulatory response, occurring within weeks rather than months or years, signals how seriously governments worldwide are treating the issue of AI-generated nonconsensual intimate imagery.