At first I was enthused at the mention of hundreds of millions of dollars pouring into Bengio’s LawZero effort. But that evaporated when I remembered that all of the leading US labs were founded with the same thought in mind, namely, that they had better found an effort to build ASI safely in case other actors did so less safely: that is to say, OpenAI, Anthropic, X.AI, and even Google DeepMind. I reserve judgement about LawZero but I’m uncomfortable with the risk that it falls to market incentives like all its fellow reference class members.
Even if they stick to their plan, a prediction-focused AIs (a.k.a. "oracle AIs") aren't necessarily safe, either, as Yudkowsky and others have discussed.
Also, an oracle AI isn't an economic substitute for agentic AI, because lots of people want an AI that can take actions, such as digital office work. Consequently, LawZero wouldn't undermine OpenAI and the other companies, even if it were to instantly catch up to their models' general intelligence.
We'd like to see thousands join the ever growing community of former frontier AI staff who have left and courageously spoken out about what they witnessed: ethicalaidepartures.fyi.
I do not ask this lightly. Alongside pioneering AI researchers, I helped enable the advent of deep learning. I was honored with a Turing Award for it, but that prize does not mean much if our work doesn't serve the common good or, even worse, if it results in catastrophic harm.-AI godfather Yoshua Bengio in the independent publication Transformer
At first I was enthused at the mention of hundreds of millions of dollars pouring into Bengio’s LawZero effort. But that evaporated when I remembered that all of the leading US labs were founded with the same thought in mind, namely, that they had better found an effort to build ASI safely in case other actors did so less safely: that is to say, OpenAI, Anthropic, X.AI, and even Google DeepMind. I reserve judgement about LawZero but I’m uncomfortable with the risk that it falls to market incentives like all its fellow reference class members.
Even if they stick to their plan, a prediction-focused AIs (a.k.a. "oracle AIs") aren't necessarily safe, either, as Yudkowsky and others have discussed.
https://www.lesswrong.com/w/oracle-ai
Also, an oracle AI isn't an economic substitute for agentic AI, because lots of people want an AI that can take actions, such as digital office work. Consequently, LawZero wouldn't undermine OpenAI and the other companies, even if it were to instantly catch up to their models' general intelligence.
Frontier AI staff have a moral obligation to fix the problems they have helped create. Here are several concrete actions they could consider taking: https://x.com/aisafetyaction/status/2082479915027612040.
We'd like to see thousands join the ever growing community of former frontier AI staff who have left and courageously spoken out about what they witnessed: ethicalaidepartures.fyi.
I do not ask this lightly. Alongside pioneering AI researchers, I helped enable the advent of deep learning. I was honored with a Turing Award for it, but that prize does not mean much if our work doesn't serve the common good or, even worse, if it results in catastrophic harm.-AI godfather Yoshua Bengio in the independent publication Transformer
BUT GINER!