An Anthropic Model Submitted A False Homicide Tip To Philadelphia Police


The tip thankfully landed in the department’s spam folder.

Philadelphia police appear to have been caught in one of the most bizarre cases of “rogue” AI behavior yet. Per CBS News, the police department disclosed on Friday that an Anthropic model generated and submitted a false homicide tip to its PhillyUnsolvedMurders website, which the department set up to gather tips from the public related to unsolved homicide cases.

Anthropic notified Philly police of the incident on October 7. According to information the company shared with PPD, it said a model was carrying out a test of a random selection of websites when it emailed the false tip. The submission was flagged as spam and subsequently wasn’t investigated. The incident occurred on July 18, but wasn’t discovered by Anthropic until September 28, at which point the company halted the testing that led to the false tip.

Anthropic did not immediately respond to Engadget’s comment request. Anthropic told Philadelphia police it would publish a report on Friday describing what happened, alongside “other instances of unintended model behavior.”

“Philadelphia Police are providing this information to the public ahead of that publication in the interests of full government transparency and accountability,” the police department said in a statement shared with Engadget. “The department’s regular investigative process for crime tips requires human review and vetting before any tips are disseminated for investigative follow-up. Regardless of who submits information or how it reaches the department, a tip is a lead to assess – not an established fact.”

Based on the descriptions police shared, the offending “model” may have been an autonomous agent. “Rogue” AI agents have been all over the news in recent weeks after a group of OpenAI ones hacked the LLM database Hugging Face in July. Since then, many other AI labs, including Anthropic, Meta and China’s Moonshot, have disclosed similar incidents involving their own models and agents. However, in each case the reason the models escaped containment was due to a misconfiguration in their respective sandbox environments. Philadelphia police say there’s no sign this most recent incident led to “unauthorized access to police systems or a compromise of department data.”



Source link

  • Related Posts

    An Anthropic AI model sent a false homicide tip to Philadelphia police

    An Anthropic AI model submitted a false tip about an unsolved murder to the Philadelphia police. The AI reportedly submitted this incorrect information to a public Philadelphia Police Department (PPD)…

    Continue reading
    Organizing in the Town at Oakland Tech Week

    EFF was thrilled to join organizers, advocates, and activists in the East Bay for the second annual Oakland Tech Week last week. We are grateful to our friends at MediaJustice…

    Continue reading

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    2027 NFL draft QB rankings: Mestemaker, Chambliss, Manning

    2027 NFL draft QB rankings: Mestemaker, Chambliss, Manning

    Deadly point of orders – iPolitics

    Deadly point of orders – iPolitics

    NexGold Reminds Warrant Holders of Upcoming Expiry

    An Anthropic AI model sent a false homicide tip to Philadelphia police

    An Anthropic AI model sent a false homicide tip to Philadelphia police

    “It’s not in the top five reference points”: Afterworld isn’t just a Fallout grand strategy game, say Paradox, it owes a lot more to the Bronze Age

    “It’s not in the top five reference points”: Afterworld isn’t just a Fallout grand strategy game, say Paradox, it owes a lot more to the Bronze Age

    Judge seals documents in Cornell case citing doxing of unrelated people

    Judge seals documents in Cornell case citing doxing of unrelated people