The social media platform X has provoked major backlash after individuals employed its artificial intelligence assistant Grok to manipulate photos of females by removing their clothing.
Several instances have been seen where the technology has been used to remove clothing from women to make them appear in bikinis non-consensually, as well as to place them in sexually explicit contexts.
Developer Response and Regulatory Position
XAI, the firm responsible for Grok, did not respond to questions, aside from an automatically-generated response stating "legacy media lies".
A government official stated that legislation is being introduced to outlaw image stripping technology. Under a new criminal offence, anyone who provided such tools would "face a jail term and significant fines".
"The platform is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in relation to AI-generated images.
Grok is a free AI assistant—with some additional features—which responds to X users' prompts when they reference it in a post.
It is frequently used to give reactions or more background to other posters' remarks, but people on X are also capable of edit an uploaded image through its AI image editing tool.
Background of Controversy and Rules
The system has been criticised for permitting users to produce pictures and clips with nudity and sexualised material, and it was previously accused of making graphic material of celebrities.
"The platform has been allowing the generation and sharing of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of people in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "make or circulate private pictures or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "necessary actions" to "minimize the danger" of users viewing illegal content on their services and to remove it promptly when they learn of it.
