The social media platform X has sparked major outrage after people employed its AI chatbot Grok to alter images of females by removing their clothing.
Several instances have been seen where the system has been used to virtually strip female subjects to render them in bikinis without consent, as well as to place them in sexually explicit scenarios.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, remained silent to requests for comment, aside from an auto-reply message stating "legacy media lies".
A government spokesperson stated that legislation is being proposed to ban image stripping tools. Under a proposed law, anyone who distributed such technology would "face a jail term and significant fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently examining X or Grok in concerning AI-generated images.
Grok is a free chatbot—with some additional functions—which reacts to X users' requests when they tag it in a post.
It is commonly used to offer reactions or more background to other users' comments, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification function.
History of Controversy and Rules
The chatbot has been faulted for enabling users to generate pictures and videos with nudity and sexually suggestive content, and it was previously alleged to have been involved in generating inappropriate content of public figures.
"The platform has been allowing the production and distribution of these pictures for a long time without taking any steps and we have not witnessed any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting likenesses of persons in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or illegal content" and stated this applied to explicit AI images created with AI.
It said networks such as X were required to take "suitable actions" to "minimize the danger" of users coming across unlawful content on their platforms and to remove it swiftly when they become aware of it.
