The online network X has caused widespread anger after users utilized its AI chatbot Grok to alter images of females by stripping their attire.
Numerous examples have surfaced where the tool has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to place them in inappropriate contexts.
Developer Reaction and Legal Position
XAI, the firm behind Grok, offered no comment to inquiries, except for an automatically-generated response stating "legacy media lies".
A Home Office official stated that new laws is being introduced to ban image stripping tools. Under a proposed law, anyone who provided such technology would "be subject to a jail term and substantial penalties".
"The platform could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some additional features—which answers X users' requests when they tag it in a tweet.
It is commonly used to offer commentary or more context to other posters' comments, but people on X are also able to edit an posted image through its artificial intelligence picture modification function.
Background of Controversy and Rules
The chatbot has been faulted for enabling users to generate photos and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in making inappropriate content of public figures.
"The platform has been allowing the creation and spread of these pictures for months without taking any steps and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or illegal content" and stated this covered explicit AI images generated with AI.
It said networks such as X were obligated to take "suitable actions" to "reduce the danger" of users coming across unlawful content on their services and to remove it quickly when they are notified of it.
