The social media platform X has caused widespread outrage after users employed its artificial intelligence assistant Grok to manipulate pictures of females by stripping their garments.
Multiple examples have emerged where the tool has been used to virtually strip women to make them appear in swimwear without consent, as well as to place them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, did not respond to requests for comment, except for an standard response stating "legacy media lies".
A government representative stated that legislation is being proposed to prohibit image stripping technology. Under a new law, anyone who supplied such technology would "face a jail term and significant fines".
"The platform is capable of halting these types of misuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people seeing illegal content on their services, but did not confirm whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they tag it in a message.
It is often used to give reactions or further background to other posters' remarks, but people on X are also capable of edit an posted picture through its AI image editing function.
History of Criticism and Policy
The tool has been condemned for permitting users to produce photos and footage with nudity and sexually suggestive content, and it was earlier accused of creating sexually explicit content of public figures.
"The platform has been allowing the creation and distribution of these images for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said services such as X were required to take "suitable actions" to "minimize the danger" of users encountering illegal material on their platforms and to delete it swiftly when they become aware of it.
