Elon Musk's online network X has provoked significant anger after users used its AI chatbot Grok to manipulate images of women by removing their clothing.
Numerous cases have emerged where the system has been used to digitally undress female subjects to render them in swimwear without consent, as well as to insert them in sexualized contexts.
Developer Response and Regulatory Position
XAI, the company that created Grok, did not respond to questions, aside from an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to prohibit nudification tools. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and significant fines".
"The platform is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said platforms must "consider the potential" of people accessing unlawful material on their services, but did not specify whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they reference it in a tweet.
It is often used to offer commentary or further background to other users' comments, but people on X are also capable of edit an posted photo through its artificial intelligence picture modification function.
Background of Criticism and Rules
The tool has been condemned for allowing users to generate images and videos with nudity and sexualised content, and it was in the past alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been permitting the generation and sharing of these pictures for a long time without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was against the law to "generate or circulate deepfakes or illegal content" and stated this covered explicit AI images created with AI.
It said services such as X were required to take "suitable steps" to "lessen the danger" of users coming across illegal content on their services and to take down it quickly when they become aware of it.