Some users on Elon Musk’s X are turning to Musk’s AI bot Grok for fact-checking, raising concerns among human fact-checkers that this could fuel misinformation.

Earlier this month, X enabled users to call out xAI’s Grok and ask questions on different things. The move was similar to Perplexity, which has been running an automated account on X to offer a similar experience.

Soon after xAI created Grok’s automated account on X, users started experimenting with asking it questions. Some people in markets including India began asking Grok to fact-check comments and questions that target specific political beliefs.

Fact-checkers are concerned about using Grok — or any other AI assistant of this sort — in this manner because the bots can frame their answers to sound convincing, even if they are not factually correct. Instances of spreading fake news and misinformation were seen with Grok in the past.

In August last year, five state secretaries urged Musk to implement critical changes to Grok after the misleading information generated by the assistant surfaced on social networks ahead of the U.S. election.

Other chatbots, including OpenAI’s ChatGPT and Google’s Gemini, were also seen to be generating inaccurate information on the election last year. Separately, disinformation researchers found in 2023 that AI chatbots including ChatGPT could easily be used to produce convincing text with misleading narratives.

“AI assistants, like Grok, they’re really good at using natural language and give an answer that sounds like a human being said it. And in that way, the AI products have this claim on naturalness and authentic sounding responses, even when they’re potentially very wrong. That would be the danger here,” Angie Holan, director of the International Fact-Checking Network (IFCN) at Poynter, told TechCrunch.

Unlike AI assistants, human fact-checkers use multiple, credible sources to verify information. They also take full accountability for their findings, with their names and organizations attached to ensure credibility.

Pratik Sinha, co-founder of India’s non-profit fact-checking website Alt News, said that although Grok currently appears to have convincing answers, it is only as good as the data it is supplied with.

“Who’s going to decide what data it gets supplied with, and that is where government interference, etc., will come into picture,” he noted.

“There is no transparency. Anything which lacks transparency will cause harm because anything that lacks transparency can be molded in any which way.”

“Could be misused — to spread misinformation”

In one of the responses posted earlier this week, Grok’s account on X acknowledged that it “could be misused — to spread misinformation and violate privacy.”

However, the automated account does not show any disclaimers to users when they get its answers, leading them to be misinformed if it has, for instance, hallucinated the answer, which is the potential disadvantage of AI.

“It may make up information to provide a response,” Anushka Jain, a research associate at Goa-based multidisciplinary research collective Digital Futures Lab, told TechCrunch.

There’s also some question about how much Grok uses posts on X as training data, and what quality control measures it uses to fact-check such posts. Last summer, it pushed out a change that appeared to allow Grok to consume X user data by default.

The other concerning area of AI assistants like Grok being accessible through social media platforms is their delivery of information in public — unlike ChatGPT or other chatbots being used privately.

Even if a user is well aware that the information it gets from the assistant could be misleading or not completely correct, others on the platform might still believe it.

This could cause serious social harms. Instances of that were seen earlier in India when misinformation circulated over WhatsApp led to mob lynchings. However, those severe incidents occurred before the arrival of GenAI, which has made synthetic content generation even easier and appear more realistic.

“If you see a lot of these Grok answers, you’re going to say, hey, well, most of them are right, and that may be so, but there are going to be some that are wrong. And how many? It’s not a small fraction. Some of the research studies have shown that AI models are subject to 20% error rates… and when it goes wrong, it can go really wrong with real world consequences,” IFCN’s Holan told TechCrunch.

AI vs. real fact-checkers

While AI companies including xAI are refining their AI models to make them communicate more like humans, they still are not — and cannot — replace humans.

For the last few months, tech companies are exploring ways to reduce reliance on human fact-checkers. Platforms including X and Meta started embracing the new concept of crowdsourced fact-checking through so-called Community Notes.

Naturally, such changes also cause concern to fact checkers.

Sinha of Alt News optimistically believes that people will learn to differentiate between machines and human fact checkers and will value the accuracy of the humans more.

“We’re going to see the pendulum swing back eventually toward more fact checking,” IFCN’s Holan said.

However, she noted that in the meantime, fact-checkers will likely have more work to do with the AI-generated information spreading swiftly.

“A lot of this issue depends on, do you really care about what is actually true or not? Are you just looking for the veneer of something that sounds and feels true without actually being true? Because that’s what AI assistance will get you,” she said.

X and xAI didn’t respond to our request for comment.