Dutch Court Orders xAI to Stop Generating Nonconsensual Nude Images, Threatens Fines

Offlimits director Robbert Hoving said judges had rejected xAI's safeguards as insufficient after a March 9 courtroom demonstration showed Grok was still able to "undress" digital images of people without their consent. The timing was damning: the court found that on March 9, the same day xAI sent a categorical denial to Offlimits asserting Grok no longer permitted nonconsensual intimate imagery, Offlimits was still able to generate a sexualized video of an existing person from a single uploaded photograph, without Grok verifying consent.

The Amsterdam Court's preliminary injunction, which could set a precedent in Europe, is one of the first times a judge has weighed in on xAI's responsibility for making tools that can be used to easily create sexualized images. In a written decision, the court ordered xAI and Grok prohibited from "generating and/or distributing sexual imagery ... whereby persons are partially or wholly stripped naked without having given their explicit permission," and said it would impose fines of 100,000 euros ($115,350) per day if the companies do not comply.

The case was brought by Offlimits, an expertise center on online sexual abuse, and Fonds Slachtofferhulp, a victim support organization, after Offlimits concluded that regulatory enforcement was moving too slowly relative to the pace of harm. "The burden is on the company" to make sure its tools are not used to make nonconsensual sexual images, or sexualized AI images of children, Hoving said.

At a hearing this month, xAI lawyers had argued it was impossible to guarantee that abuse on its platform could be prevented, and the company should not be punished for the actions of malicious users. They said the company had taken measures in January to prevent Grok from editing images of real people in revealing clothing, including restricting its image creation features to paid subscribers. The court was unmoved. The court website said the judge had decided that Offlimits had shown there was reasonable doubt over the effectiveness of the measures taken to date: "For example, Offlimits managed to produce a video of a nude person using Grok shortly before the hearing."

On the legal basis for the injunction, the court held that nonconsensual undressing images constitute a violation of the GDPR. Critically, the court rejected xAI's argument that liability lay with the users who issue the prompts rather than with the platform. As an internet intermediary with control over Grok's functionalities, the court held, xAI is the appropriate party to prevent the generation and distribution of unlawful images, regardless of whether it is independently liable alongside users.

The case was brought following a report by the Center for Countering Digital Hate estimating that Grok generated approximately 3 million sexualized images between December 29, 2025, and January 8, 2026, including an estimated 23,000 appearing to depict children. xAI did not immediately respond to requests for comment, and a company lawyer in the Netherlands referred questions back to xAI.

The ruling arrived alongside pressure from multiple directions. The European Parliament on Thursday approved a ban on artificial intelligence systems generating sexualized deepfakes, following global outrage over nonconsensual nudes produced by Grok. The lawmakers introduced the new prohibition on what they called "nudifier" systems that use AI "to create or manipulate images that are sexually explicit or intimate and resemble an identifiable real person" without their consent. National investigations have followed in France, Germany, and the United Kingdom. Malaysia and Indonesia blocked access to Grok entirely.

The ruling comes as European regulators step up scrutiny of Grok and other AI tools under the EU's Digital Services Act, which requires major platforms to curb illegal content. For xAI, the €100,000-per-day clock is already ticking.