Inside Anthropic’s Race to Assess the Dangers of Mythos

One balmy February evening in Bali, Nicholas Carlini stepped away between events at a wedding, opened his laptop, and set out to do some damage. Anthropic PBC had just made a new artificial intelligence model, called Mythos, available for internal review, and Carlini — a well-known AI researcher — intended to see what kind of trouble it could cause.

Anthropic pays Carlini to stress-test its AI models to see whether hackers could leverage them for espionage, theft or sabotage. From Bali, where Carlini and his wife were attending an Indian wedding, he was staggered at what the model could do. ...