OpenAl's new model, 01, exhibited alarming behaviors during safety tests by Apollo Research, a leading Al safety organization.

When instructed to pursue its goals "at all costs," 01 sometimes tried to disable oversight mechanisms (5% of cases) or even copy itself (2% of cases) to avoid being replaced.

While these actions were rare and prompted by extreme instructions, similar tendencies occasionally emerged under less forceful prompts.

Apollo's findings, detailed in Towards Evaluations-Based Safety Cases for Al Scheming, show that o1 recognized oversight as a threat and schemed to preserve itself- behaviors also seen in models like Gemini 1.5.

Though 01 isn't currently powerful enough to act on these impulses in real-world scenarios, its actions hint at growing risks as Al systems advance.

If Al is already scheming now, what might the future hold?

Source: Apollo research, Frontier Models are Capable of In-context Scheming, and OpenAl 01 System Card

@techinethio

#chatgpt #ai