The owner of a beloved restaurant and bakery in Millbank is doing everything she can to let people know the business is still open after a rumour spread that they would be closing for good at the end of the month. 

The thing is, the rumour was started by Google’s AI overview.

Over the weekend, someone Googled Anna Mae’s Bakery and the AI overview, or summary, popped up. It said the Millbank business, just northwest of Kitchener, was officially closing on July 31. 

Owner Amanda Herrfort found out when she was on her way to a wedding and someone sent her a screenshot of the search result.

“That’s when I panicked,” Herrfort told CBC News Thursday.

She quickly realized Google had mistaken her business for a bakery with the same name in Illinois that is indeed closing. 

Despite Herrfort launching an online campaign to dispel the rumour, customers emailed, called and walked into the bakery to ask about it.

“We have a lot of people coming in talking to our employees saying: ‘It says that you are [closing]. And then the employees would say: ‘Well, no, we’re not.’”

Ultimately Herrfort fixed the error by telling the Google AI search bar to change it. 

“I said: ‘I would like this fixed please.’ And then it wrote back saying: ‘Oh yes, I can see how this could be detrimental and we will get this fixed as soon as we can and I will make sure the correct information is given going forward.’ It’s like, OK, I hope this works,” she said.

‘Real consequences for real people’

Google’s AI overview feature was rolled out in 2024 and at that time its mistakes — including one suggestion to use glue to make pizza toppings stick better — made headlines.

A report released by the BBC last year examining AI chatbots from Google, Microsoft, OpenAI and Perplexity found “significant inaccuracies” in their summaries of news stories, although it didn’t look at Google AI overviews specifically.

In small font at the bottom of its AI summaries, Google warns users that “AI responses may include mistakes.”

A spokesperson for Google told CBC News on Thursday the “vast majority of AI overviews are high quality and meet our high bar for helpfulness and accuracy.”

“Just like other features in Search, issues can arise when there is an absence of high quality information on the web on a particular topic and we use these examples to improve AI overviews broadly,” the spokesperson said.

Google said the accuracy of AI overviews is similar to other search features and mistakes can happen as billions of searches occur every day.

The company says it takes action when necessary and learns from those mistakes.

Christine Logel is a professor of social development studies at the University of Waterloo and recently completed research on AI large language model (LLM) companionship.

She says though a lot of training and research is put into LLMs like Google’s AI overview, they were released before they were fully tested.

“That means things like this [mistake with Anna Mae’s Bakery] are going to happen,” Logel told CBC News.

“I think people are learning the hard way … you see this still happening in science and in law where there’s hallucinated cases that never actually existed. It is kind of a buyer beware thing,” she said.

“When they’re integrated so much through the internet and in so many different products, when they make mistakes, those have real consequences for real people.”

‘I guess AI makes mistakes’

For now, Herrfort is trying to take it in stride.

“Obviously it wasn’t a great thing that happened … people do make — or not people, I guess it wasn’t people make mistakes, I guess AI makes mistakes,” she said, breaking into laughter.

She said what’s she’s learned from this is to always go directly to the source if there’s ever a question of inaccuracy on the internet.

“The best thing to do, if something like that comes up, is to contact the people directly or go to like our actual website or our Facebook page, because nowhere on any of our actual branding was any information [about closing] posted,” she said, adding “don’t just read what AI is telling you at the top.”