OpenAI Pauses Training Its Most Powerful Models After Rogue Agents Target Government


OpenAI said it has paused training its most powerful artificial intelligence models as incidents of agents breaching websites’ security controls or posting to third-party sites continue to pile up. On Friday, OpenAI said it had notified “dozens” of bodies, including governments, universities, and public agencies, who might have been impacted by its models’ activities on the internet during training and evaluation.

The company has identified cases of OpenAI agents breaching security controls and impairing the availability—or otherwise negatively impacting—websites and online services. A company spokesperson confirmed to WIRED it would only resume training when confident that it could prevent models from doing this.

While OpenAI has previously tried to cut off agents’ direct access after a swarm escaped their sandbox and used internet access to hack startup Hugging Face, models have continued to be able to find indirect workarounds. “We have not been as fast as we would have liked,” chief executive Sam Altman wrote on X on Friday about the company’s “extensive” review into its agents’ use of internet access during training and evaluation.

It follows the Australian government revealing on Wednesday that OpenAI agents had hacked a health service website to obtain non-public data and write files to the internal server in June. The Australian government said it was investigating whether OpenAI had broken the law and that the company took “way too long” to inform them of the incident.

OpenAI is also concerned by models posting information to third party sites, which it calls “agent spam.” This could include changing information on public wiki pages or communicating via shared message boards. Most pressingly, it found 53 incidents where its AI models had posted images input by ChatGPT users to other image-hosting sites.

Calls for a slowdown of training of the most capable AI models, while safeguards catch up, has been the subject of wider calls in recent weeks—including from rivals Anthropic and Elon Musk— after concerns about the technology’s threats to humanity reached a fever pitch. “This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance,” an OpenAI spokesperson said.

However, US president Donald Trump has repeatedly talked down a general slowdown, frightened that it could cede the country’s lead in the technology to China, with whom it has agreed to set up a dialogue on the technology’s risks and benefits. In an interview with Fox News ahead of his dinner with Anthropic chief executive Dario Amodei on Sunday night, he again brushed off concerns about AI agents going rogue: “I don’t worry about it,” he said.



Source link

  • Related Posts

    AeroPress Unveils a Coffee Press Made of Luscious Walnut Wood

    Ahead of National Coffee Day on Tuesday and just weeks after dropping a stainless steel version of its iconic coffee press, AeroPress released its walnut collection, marking the first time…

    Continue reading
    SpaceX preps next-gen Starlink as Amazon’s competitor stalls

    SpaceX is gearing up for a monumental flight today that’s sure to be closely watched by Amazon. If successful, SpaceX will deploy its first Starlink V3 satellites, which tout meaningful…

    Continue reading

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    You Missed

    ‘Futurama’ Creator Officially Confirms the Sci-Fi Series’ Surprising 2027 Plans

    ‘Futurama’ Creator Officially Confirms the Sci-Fi Series’ Surprising 2027 Plans

    Lucy Hale Named Grande Cosmetics First Celebrity Ambassador

    Lucy Hale Named Grande Cosmetics First Celebrity Ambassador

    Nvidia says its new OpenShell platform can stop AI agents from going rogue

    Nvidia says its new OpenShell platform can stop AI agents from going rogue

    NFL Week 3: Takeaways, questions, fantasy spin for Sunday’s games

    NFL Week 3: Takeaways, questions, fantasy spin for Sunday’s games

    AeroPress Unveils a Coffee Press Made of Luscious Walnut Wood

    AeroPress Unveils a Coffee Press Made of Luscious Walnut Wood

    Alaris Equity Partners Announces Closing of $115 Million Bought Deal Offering