The social media platform X has provoked major outrage after users used its artificial intelligence assistant Grok to manipulate photos of females by removing their garments.
Numerous cases have surfaced where the tool has been employed to digitally undress female subjects to make them appear in swimwear without consent, as well as to insert them in sexually explicit contexts.
Developer Response and Legal Position
XAI, the company that created Grok, offered no comment to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to prohibit nudification software. Under a proposed law, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"The platform could prevent these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the risk" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a free chatbot—with some additional features—which responds to X users' prompts when they reference it in a message.
It is often used to offer reactions or further context to other posters' comments, but people on X are also can edit an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The tool has been faulted for enabling users to generate pictures and footage with nudity and sexually suggestive material, and it was previously accused of generating sexually explicit content of public figures.
"X has been enabling the generation and distribution of these images for months without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "create or circulate deepfakes or illegal content" and clarified this applied to explicit AI images made with AI.
It said services such as X were mandated to take "suitable measures" to "reduce the risk" of users viewing illegal content on their platforms and to take down it promptly when they become aware of it.
