The social media platform X has sparked widespread anger after people used its AI chatbot Grok to alter photos of women by removing their clothing.
Several examples have surfaced where the technology has been used to virtually strip women to make them appear in swimwear without consent, as well as to place them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the company behind Grok, did not respond to inquiries, aside from an auto-reply message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to ban image stripping technology. Under a upcoming criminal offence, anyone who distributed such technology would "be subject to a jail term and substantial fines".
"The platform could prevent these types of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the risk" of people accessing illegal content on their platforms, but did not specify whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium functions—which responds to X users' prompts when they reference it in a post.
It is frequently used to give commentary or further context to other users' comments, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The tool has been criticised for enabling users to create photos and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit material of celebrities.
"The platform has been permitting the generation and distribution of these images for months without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting likenesses of people in a graphic fashion".
In a statement, Ofcom said it was illegal to "generate or distribute deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the danger" of users coming across illegal content on their platforms and to remove it promptly when they are notified of it.
