The EU investigates X for Grok: did it meet the DSA by launching an IA that generates unconsensual sexual images?

The European Commission has initiated a formal procedure under the Digital Services Regulation to find out whether X properly assessed the risks before launching its Gook artificial intelligence tool, after using this technology to generate images of unconsensual sexual content. The investigation comes after examples were detected in which the IA produced sexually manipulated images, some of which could fit the definition of child sexual abuse material, according to the European authorities. More information on the Commission's official note is available at press release.

From the beginning, regulators have expressed the gravity of the matter: the use of generative models to create unconsensual sexual images poses specific risks to the integrity and privacy of real people, and requires a rapid regulatory response. The European Commissioner responsible for technology stressed that these are not minor side effects, but particularly damage to women and minors, and that the investigation aims to check whether X fulfilled its legal obligations under European law.

The institutional context has been shaping in a few days. The Office of the United Kingdom Information Commissioner (ICO) contacted X and xAI in early January to collect information on data protection measures, and the British online security regulator, Ofcom, began his own inquiry into the appearance of sexualized images generated by Grok. The ICO communiqué is available on its website. Here. and the action of Ofcom Here.. In parallel, the California Attorney General opened an inquiry into the generation of unconsensual sexual material using Grok; his official note can be read on the website of the justice department in that state.

The X reaction has included limiting Grok's ability to generate and edit images, restricting that service to payment subscribers, a decision that received criticism for converting a functionality that has proved potentially illegal into a premium service. Voices of the British government described this measure as an insufficient and offensive response for victims of sexual violence; The Guardian it includes public and political reactions.

The legal background in the European Union makes this research particularly important. X was designated as a very large online platform by the EU after exceeding the monthly active user threshold, subjecting it to reinforced obligations under the Digital Services Act (DSA). These obligations include the assessment and mitigation of systemic risks - such as the dissemination of illegal content and threats to fundamental rights - transparency vis-à-vis authorities and cooperation in research. In addition, in December the Commission punished the company with EUR 120 million for non-compliance with transparency under the same law, which shows that the European authorities are firmly applying these rules.

What can be derived from this procedure? In the short term, the Commission will examine the documentation and evidence submitted by X to decide whether the company applied appropriate risk assessment procedures before activating Grok and whether it put in place effective safeguards to prevent the production and dissemination of illegal material. If non-compliance is found, the consequences can range from orders to correct mandatory practices and audits to significant fines. The DSA provides monitoring tools that seek to ensure that platforms do not treat user safety as a mere operational issue, but as a central responsibility for their operations.

Beyond X and Grok, this case raises broad questions about how the IA systems that generate images should be managed and about the responsibility of the platforms that offer them. Generative technology facilitates the mass creation of content that falsifies the presence of real people in intimate contexts; without safe design measures, access controls and robust moderation processes, the potential for damage is real and tangible. The public and regulatory debate is committed to requiring comprehensive risk assessments, pre-commercial mitigation tests and effective channels to report and withdraw illicit content quickly.

For users and developers, the lesson is clear: innovation in IA is not free of legal and ethical limits. Regulators no longer expect ex post reactions; they want to see evidence that products have been thought from design to minimize damage, and they require transparency on how these mechanisms work. This case will probably be a reference point on how authorities interpret and apply DSA to emerging technologies.

As research advances, attention will be kept to the measures X takes and to the response of transnational regulators. The resolution of this file will serve as a test for the real scope of digital regulation against IA tools capable of generating potentially harmful content, and will mark a precedent on the balance between innovation, freedom of expression and protection of fundamental rights in the digital environment.