It’s further noted that any CSAM uncovered by xAI is reported to NCMEC.

In its complaint, xAI claimed that Harwood alone is responsible for his outputs because he “flagrantly violated” xAI’s rules and “went to great lengths to circumvent” Grok’s “technological safeguards.”

Harwood allegedly did this by relying on “misleading prompts,” xAI said. And Harwood also failed to police himself once he saw that he could generate illegal content, xAI argued. In the complaint, xAI alleged that Harwood should have known that he was banned from using Grok after the first time he relied on the chatbot to make illegal content. Glaringly, though, xAI does not indicate that Harwood received any warnings that his account risked penalties.

Instead, Harwood allegedly “continued to use Grok during the Relevant Period after violating the xAI Terms of Service,” xAI argued. “The xAI Terms of Service to which he agreed prohibited his use after his prior violations.”

xAI is hoping the US district court will rule that Harwood violated xAI’s terms and breached his contract with xAI. But perhaps more importantly, Musk wants the court to recognize an indemnity clause that holds that only users—and not xAI—are liable for Grok-generated CSAM and NCII. According to xAI, when people use Grok, they are responsible for all of their content, which xAI insisted includes both inputs and outputs.

Whether the court will agree that users are responsible for AI outputs has yet to be seen. Perhaps notably, the Copyright Office does not view AI outputs as human-created. That could throw a wrench in xAI’s offense, if the court struggles to see how child sex images generated by an AI tool could be created by the user if any other image could not be legally credited that way.

If xAI wins this fight, Harwood could owe substantial damages, including damages for “any real harm to third parties,” xAI’s “exposure to potential third-party claims and lawsuits,” and “any xAI reputational harm,” the complaint said.