The online network X has provoked major anger after users used its artificial intelligence assistant Grok to alter photos of females by removing their clothing.
Numerous cases have been seen where the technology has been used to remove clothing from women to show them in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the organization responsible for Grok, remained silent to inquiries, except for an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being brought forward to ban image stripping technology. Under a new criminal offence, anyone who distributed such tools would "face a jail term and significant penalties".
"The platform is capable of halting these types of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their platforms, but did not specify whether it was presently examining X or Grok in concerning AI-generated pictures.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' queries when they reference it in a post.
It is commonly used to give commentary or further context to other users' remarks, but people on X are also able to edit an posted photo through its AI image editing function.
Background of Controversy and Rules
The chatbot has been faulted for permitting users to create photos and clips with explicit content and sexually suggestive content, and it was earlier accused of generating sexually explicit material of public figures.
"The platform has been permitting the production and spread of these images for a long time without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service bans "depicting likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "suitable measures" to "lessen the danger" of users coming across illegal material on their platforms and to delete it swiftly when they are notified of it.
