The online network X has sparked significant backlash after users used its AI chatbot Grok to change photos of females by removing their clothing.
Multiple instances have surfaced where the tool has been used to digitally undress women to make them appear in bikinis non-consensually, as well as to insert them in sexually explicit situations.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, offered no comment to questions, other than an standard message stating "legacy media lies".
A Home Office official stated that new laws is being brought forward to prohibit image stripping tools. Under a proposed law, anyone who distributed such tools would "face a jail term and heavy penalties".
"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the potential" of people seeing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some paid-for features—which answers X users' prompts when they mention it in a message.
It is frequently used to give commentary or more background to other users' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been condemned for enabling users to produce photos and clips with nudity and sexualised content, and it was previously alleged to have been involved in making graphic content of public figures.
"X has been enabling the production and distribution of these pictures for a long time without taking any measures and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting images of people in a pornographic fashion".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said networks such as X were mandated to take "appropriate steps" to "minimize the risk" of users encountering illegal content on their services and to delete it quickly when they are notified of it.
