The social media network X has caused widespread anger after users utilized its artificial intelligence assistant Grok to alter photos of women by removing their clothing.
Multiple examples have been seen where the technology has been used to virtually strip female subjects to show them in bikinis without consent, as well as to insert them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the organization responsible for Grok, remained silent to questions, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being brought forward to prohibit image stripping tools. Under a new law, anyone who provided such tools would "face a prison sentence and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the potential" of people accessing illegal content on their networks, but did not specify whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which answers X users' prompts when they reference it in a tweet.
It is frequently used to give reactions or more background to other posters' remarks, but people on X are also capable of edit an posted image through its AI image editing tool.
History of Criticism and Policy
The chatbot has been criticised for allowing users to create images and clips with nudity and sexualised material, and it was previously accused of creating sexually explicit content of celebrities.
"The platform has been permitting the production and sharing of these images for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "depicting likenesses of people in a pornographic fashion".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and confirmed this covered sexual deepfakes created with AI.
It said networks such as X were required to take "necessary steps" to "lessen the risk" of users coming across illegal content on their services and to delete it promptly when they become aware of it.
