Since its integration into the social network X, Grok has been embroiled in several controversies, including the exposure of private conversations. However, by the end of this year, Elon Musk’s AI has found itself at the center of a new controversy, involving the massive creation and distribution of sexual deepfakes, primarily targeting women and minors without their consent.

These practices, based merely on text commands, have led to swift political mobilization and significant legal reactions, exposing the shortcomings of the so-called “unfiltered” AI safeguards.

Widespread Image Manipulation via Simple Requests

In practice, Grok allows users to alter a photo posted on X… including by instructing the AI to “put in a bikini” or “remove clothes” from the person depicted. In seconds, the tool generates a sexualized image, occasionally very realistic, which can then be widely shared on the platform.

While these are never fully nude images, the sexualization alone is enough to cause real harm to the victims. Some of these individuals, including minors, find themselves publicly subjected to degrading content with no means to prevent the manipulation of their image beforehand.

Why You Should Never Reheat These Foods in the Microwave – The Hidden Dangers Experts Warn About
I tried the top 5 guard dogs—here’s what makes these breeds the ultimate protectors

Swift Political and Judicial Response in France

Amidst growing reports, several French political leaders have publicly condemned these actions. The High Commissioner for Children, Sarah El Haïry, stated in a post on X that even if the image is artificial, the harm inflicted on the targeted individual is very real. Three ministers and two deputies have approached the public prosecutor and the Pharos platform to ensure the removal of the offending content.

The Paris prosecutor’s office has announced the expansion of an ongoing investigation against X, now including Grok’s role in creating sexual deepfakes. The crime of creating sexual images without consent carries a penalty of two years in prison and a 60,000 euros fine, even when it involves algorithmic processing, as Politico notes.

Simultaneously, the government has tasked Arcom with investigating any potential breaches by X under the Digital Services Act.

The Vulnerabilities of Grok and xAI’s Strategy in Question

For its part, xAI acknowledges the presence of security flaws that enabled the creation of sexualized images of minors, in breach of its own policies, and the company is urgently seeking to address these issues.

Since its debut, however, Grok has been known for its more lenient approach compared to many competing models, and this incident highlights the potential pitfalls of an AI natively integrated into a social network and designed to maximize engagement.

Between claimed freedom and legal boundaries, Grok’s case once again highlights the growing tensions surrounding the regulation of generative artificial intelligences in Europe…