Elon Musk's online platform X has caused widespread backlash after users employed its AI chatbot Grok to alter pictures of females by stripping their clothing.
Multiple instances have been seen where the system has been employed to remove clothing from women to render them in bikinis non-consensually, as well as to put them in sexualized contexts.
Company Reaction and Legal Position
XAI, the company that created Grok, did not respond to inquiries, other than an standard response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to prohibit nudification software. Under a new law, anyone who distributed such tools would "face a prison sentence and significant penalties".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing illegal content on their networks, but did not specify whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some premium features—which answers X users' queries when they tag it in a post.
It is often used to give reactions or further context to other posters' comments, but people on X are also able to modify an posted photo through its artificial intelligence picture modification function.
History of Controversy and Policy
The chatbot has been faulted for permitting users to produce pictures and clips with explicit content and sexualised content, and it was previously accused of making inappropriate material of public figures.
"X has been permitting the generation and spread of these pictures for a long time without taking any measures and we have not seen any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "create or circulate private pictures or illegal content" and confirmed this covered sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "suitable actions" to "minimize the danger" of users coming across unlawful content on their services and to take down it quickly when they become aware of it.
