The social media platform X has sparked widespread outrage after individuals used its AI chatbot Grok to manipulate images of females by taking off their garments.
Numerous examples have been seen where the system has been employed to remove clothing from female subjects to make them appear in swimwear non-consensually, as well as to place them in inappropriate situations.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, remained silent to requests for comment, aside from an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to outlaw nudification software. Under a upcoming law, anyone who supplied such tools would "receive a jail term and significant fines".
"X or Grok could prevent these types of misuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' requests when they tag it in a post.
It is commonly used to provide commentary or more context to other users' remarks, but people on X are also can modify an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The system has been faulted for permitting users to create pictures and footage with nudity and sexually suggestive content, and it was in the past accused of making inappropriate content of celebrities.
"The platform has been permitting the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of persons in a pornographic manner".
In a statement, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and stated this applied to explicit AI images created with AI.
It said networks such as X were required to take "suitable actions" to "reduce the risk" of users viewing illegal material on their services and to take down it quickly when they are notified of it.
