{"id":43327,"date":"2024-05-20T14:45:52","date_gmt":"2024-05-20T18:45:52","guid":{"rendered":"https:\/\/mjtsai.com\/blog\/?p=43327"},"modified":"2024-05-24T11:48:43","modified_gmt":"2024-05-24T15:48:43","slug":"sutskever-and-leike-out-at-openai","status":"publish","type":"post","link":"https:\/\/mjtsai.com\/blog\/2024\/05\/20\/sutskever-and-leike-out-at-openai\/","title":{"rendered":"Sutskever and Leike Out at OpenAI"},"content":{"rendered":"

Sigal Samuel<\/a> (tweet<\/a>):<\/p>\n

\n

For months, OpenAI has been losing employees who care deeply about making sure AI is safe. Now, the company is positively hemorrhaging them.<\/p>\n

Ilya Sutskever and Jan Leike announced their departures from OpenAI, the maker of ChatGPT, on Tuesday. They were the leaders of the company’s superalignment team — the team tasked with ensuring that AI stays aligned with the goals of its makers, rather than acting unpredictably and harming humanity.<\/p>\n

[…]<\/p>\n

Altman was fundraising with autocratic regimes like Saudi Arabia so he could spin up a new AI chip-making company, which would give him a huge supply of the coveted resources needed to build cutting-edge AI. That was alarming to safety-minded employees. If Altman truly cared about building and deploying AI in the safest way possible, why did he seem to be in a mad dash to accumulate as many chips as possible, which would only accelerate the technology?<\/p>\n

[…]<\/p>\n

For employees, all this led to a gradual “loss of belief that when OpenAI says it’s going to do something or says that it values something, that that is actually true,” a source with inside knowledge of the company told me.<\/p>\n<\/blockquote>\n

I don’t think wanting access to chips is a bad sign, but it seems clear that the safety folks lost the power struggle within the company.<\/p>\n\n

Greg Brockman and Sam Altman<\/a>:<\/p>\n

We’re really grateful to Jan for everything he’s done for OpenAI, and we know he’ll continue to contribute to the mission from outside. In light of the questions his departure has raised, we wanted to explain a bit about how we think about our overall strategy.<\/p><\/blockquote>\n

As many of the replies note, the words seem rather hollow and don’t really correspond with their actions.<\/p>\n\n

Kelsey Piper<\/a>:<\/p>\n

\n

But there was no stronger sign of OpenAI’s commitment to its mission than the prominent roles of people like Sutskever and Leike, technologists with a long history of commitment to safety and an apparently genuine willingness to ask OpenAI to change course if needed.<\/p>\n

[…]<\/p>\n

And it makes it clear that OpenAI’s concern with external oversight and transparency couldn’t have run all that deep. If you want external oversight and opportunities for the rest of the world to play a role in what you’re doing, making former employees sign extremely restrictive NDAs doesn’t exactly follow.<\/p>\n<\/blockquote>\n

Altman claims<\/a> that they didn’t actually mean to cancel the equity for employees who didn’t sign the exit NDA. It was just a mistake in the paperwork (via Ryan Jones<\/a>, Hacker News<\/a>).<\/p>\n\n

Previously:<\/p>\n