Indonesia temporarily blocked Elon Musk’s Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool.

The move comes after governments, researchers and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app.

xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs, including depictions of scantily clad children.

“The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space,” communications and digital minister Meutya Hafid said in a statement.

The ministry has also summoned X officials to discuss the matter.

Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if they had uploaded illegal content.

xAI replied to Reuters’ email seeking comment with what seemed to be an automated response: “Legacy Media Lies”. X did not immediately respond to a request for comment.

Indonesia, with the world’s biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene.

Indonesia’s block follows Grok switching off its image creation function on Friday for the vast majority of users after the widespread outcry about its use to create sexually explicit and violent imagery.

Musk has also been threatened with fines, regulatory action and reports of a possible ban on X in the UK.

The tool has also been used to manipulate images of women to remove their clothes and put them in sexualised positions. The function to do so has been switched off except for paying subscribers.

Australian prime minister Anthony Albanese has also expressed concern over the use of artificial intelligence on the Grok chatbot to enable exploitative sexual content.

The prime minister on Saturday joined a growing list of international leaders including British counterpart Keir Starmer in criticising the social media platform.

“The use of generative AI to exploit or sexualise people without their consent is abhorrent,” he told reporters in Canberra.

“The fact that this tool was used so that people were using its image creation function through Grok is, I think, just completely abhorrent.

“It, once again, is an example of social media not showing social responsibility and Australians and indeed, global citizens deserve better.”

While the number of reports received by Australia’s eSafety Office remains small, it says there has been a recent increase relating to the use of Grok to create sexualised or exploitative imagery.

The watchdog warned on Friday it would use its powers including removal notices where such material meets the thresholds defined in the Online Safety Act.

“X, Grok and a wide range of other services are also subject to systemic safety obligations to detect and remove child sexual exploitation material and other unlawful material as part of Australia’s world-leading industry codes and standards,” it said.

With Reuters and AAP