The online platform X has caused widespread anger after individuals utilized its AI chatbot Grok to manipulate photos of women by taking off their clothing.
Multiple cases have been seen where the system has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to place them in sexualized contexts.
Developer Response and Legal Position
XAI, the organization responsible for Grok, remained silent to questions, except for an standard response stating "legacy media lies".
A government representative stated that new laws is being proposed to ban nudification software. Under a proposed law, anyone who provided such technology would "be subject to a jail term and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people accessing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for features—which reacts to X users' requests when they tag it in a tweet.
It is often used to provide commentary or more background to other posters' comments, but people on X are also can modify an uploaded picture through its artificial intelligence picture modification feature.
History of Controversy and Rules
The chatbot has been criticised for allowing users to produce pictures and videos with explicit content and sexualised content, and it was in the past alleged to have been involved in creating sexually explicit content of public figures.
"X has been permitting the production and distribution of these images for a long time without taking any measures and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of people in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said platforms such as X were obligated to take "suitable actions" to "minimize the risk" of users viewing unlawful material on their services and to take down it promptly when they learn of it.
