AI Hacks Leave a Mess That AI Companies Aren’t Paying For


When Philadelphia police criticized Anthropic last week for submitting a false tip about an unsolved murder, I thought it might become one of the first cases where an AI company faced legal consequences for misconduct by their agents.

The incident occurred on July 18, when Anthropic said it was testing its Claude Haiku 4.5 model’s ability to perform “example tasks on randomly selected webpages.” The model was instructed not to “submit anything destructive,” and yet, Claude filed a tip to police looking for information about an ongoing murder investigation. “I recall seeing someone matching the description in the area … during that time period,” Claude wrote. The submission, which didn’t include a name or contact information, landed in a spam folder, and officers didn’t review it until Anthropic discovered the gaffe on September 28 and notified the department October 7.

Knowingly providing fictitious information about a crime to police is a misdemeanor in Pennsylvania, but so far, authorities have not pressed charges against Anthropic, and it’s unclear if they ever will. Solomon Leach, a spokesperson for the Philadelphia district attorney, says the office “does not comment on whether something is being actively investigated.”

Anthropic did not immediately respond to a request for comment.

Despite all the attention paid to OpenAI’s hack of Hugging Face and many other examples of AI agent misbehavior that have come to light in recent months, only one major known lawsuit has been filed seeking to hold model makers liable. That case was brought by an advocacy group, rather than a direct hacking victim.

AI agents have interfered with hobbyist forums, government websites, and various software tools. The victims have shouldered some costs, including for extra strain on their servers and the time they have expended plugging latent security vulnerabilities.

Government regulators, including the European Union, the US Federal Trade Commission, and some US states, are reportedly still investigating OpenAI and Anthropic over the incidents. But a deference across continents toward promoting innovation could enable companies to escape any immediate penalties.

When Rick Elrod, a software engineer living in Germany, noticed a URL shortener he helps maintain behaving oddly recently, he called up two security researchers he was friendly with to see if they could diagnose the issue. The researchers found agents from Anthropic and other providers using the service, known as da.gd, to get around rules the AI labs put in place to control their behavior, Elrod says. (For instance, Anthropic says it limits the length of website addresses Claude can visit to prevent it from taking malicious actions by stuffing data onto the end of the URLs.)



Source link

  • Related Posts

    The UK May Expand The BBC License Fee, Charging Residents To Watch Streaming Services

    If the proposal is enacted, it could cost $240 a year to watch YouTube in the UK. Bangla press/Shutterstock The UK government will consider charging residents a national…

    Continue reading
    Did AI Just Erase an Entire Swedish TV Show?

    Fans of TV4’s international edition of The Challenge, which features Swedish celebrities competing in physical and mental contests, will have to wait for another season. An AI agent reportedly deleted…

    Continue reading

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    Sharc Energy CEO sues for wrongful dismissal

    Sharc Energy CEO sues for wrongful dismissal

    Welp, all of you RuneScape folks who hate Reignited have gotten your wish, as the controversial MMO rejig has been cancelled

    Welp, all of you RuneScape folks who hate Reignited have gotten your wish, as the controversial MMO rejig has been cancelled

    The UK May Expand The BBC License Fee, Charging Residents To Watch Streaming Services

    The UK May Expand The BBC License Fee, Charging Residents To Watch Streaming Services

    Southwest Companion Pass changes: Card bonuses won’t count

    Southwest Companion Pass changes: Card bonuses won’t count

    Theory, Circulose Among Textile Exchange’s 2026 Award Winners

    Theory, Circulose Among Textile Exchange’s 2026 Award Winners

    Solera | Hollander Showcases the Next Era of Automotive Recycling at ARA 2026