OpenAI and Anthropic agree to send models to US Government for safety evaluation
OpenAI and Anthropic have partnered with the U.S. AI Safety Institute to enhance AI model safety through voluntary evaluations, though concerns about the effectiveness and clarity of safety commitments persist.
Read original article
OpenAI and Anthropic have entered into an agreement with the U.S. AI Safety Institute, part of the National Institute of Standards and Technology (NIST), to facilitate safety evaluations of their AI models. This collaboration aims to enhance AI model safety through research, testing, and evaluation, similar to practices in the U.K. The agreement allows the AI Safety Institute to access new models from both companies before and after their public release, providing feedback on potential safety improvements. OpenAI's CEO, Sam Altman, emphasized the importance of this initiative for national leadership in responsible AI development. While the agreement represents a significant step towards establishing safety protocols, it is important to note that participation in safety evaluations remains voluntary and lacks enforceable regulations. Critics have expressed concerns about the vagueness of the term "safety" and the need for AI companies to follow through on their commitments to ensure effective regulation. The agreement is seen as a positive move towards better oversight of AI development, but the effectiveness of voluntary commitments remains to be seen.
- OpenAI and Anthropic will provide AI models for safety evaluations to the U.S. AI Safety Institute.
- The collaboration aims to enhance the safety and responsible development of AI technologies.
- Participation in safety evaluations is voluntary and lacks enforceable regulations.
- Concerns exist regarding the vagueness of "safety" and the need for companies to fulfill their commitments.
- The agreement is viewed as a step forward in establishing AI safety protocols.
Related
Anthropic CEO on Being an Underdog, AI Safety, and Economic Inequality
Anthropic's CEO, Dario Amodei, emphasizes AI progress, safety, and economic equality. The company's advanced AI system, Claude 3.5 Sonnet, competes with OpenAI, focusing on public benefit and multiple safety measures. Amodei discusses government regulation and funding for AI development.
OpenAI promised to make its AI safe. Employees say it 'failed' its first test
OpenAI faces criticism for failing safety test on GPT-4 Omni model, signaling a shift towards profit over safety. Concerns raised on self-regulation effectiveness and reliance on voluntary commitments for AI risk mitigation. Leadership changes reflect ongoing safety challenges.
AI companies promised to self-regulate one year ago. What's changed?
AI companies like Amazon, Google, and Microsoft committed to safe AI development with the White House. Progress includes red-teaming practices and watermarks, but lacks transparency and accountability. Efforts like red-teaming exercises, collaboration with experts, and information sharing show improvement. Encryption and bug bounty programs enhance security, but independent verification and more actions are needed for AI safety and trust.
US lawmakers send a letter to OpenAI requesting government access
US lawmakers have urged OpenAI to enhance safety standards, allocate resources for AI safety research, and allow pre-deployment testing, following whistleblower allegations and concerns about AI risks and accountability.
OpenAI and Anthropic will share their models with the US government
OpenAI and Anthropic have partnered with the U.S. AI Safety Institute for pre-release testing of AI models, addressing safety and ethical concerns amid increasing commercialization and scrutiny in the AI industry.
- There is skepticism about the clarity and effectiveness of AI safety evaluations, with some questioning what these evaluations will entail.
- Concerns about regulatory capture and the influence of corporations on government decisions are prevalent, suggesting that the partnership may primarily benefit the companies involved.
- Commenters express doubt about the government's readiness to conduct comprehensive safety evaluations.
- Some believe that the term "AI safety" is overloaded and lacks a clear definition, leading to potential misuse in shaping acceptable discourse.
- There is a general unease about the lack of congressional input in the collaboration, despite the technical nature of the organization involved.
18 commentsBut I'm worried this will be used to shape acceptable discourse as people are increasingly using LLMs as a kind of database of knowledge. It is telling that the largest players are eager to comply which suggests that they feel they're in the club and the regulations will effectively be a moat.
Naughty is in the eye is the beholder. Ask me what a Satanist is, and I would expect something about a group who challenges religious laws enshrining Christianity. Ask an evangelical and discussing the topic could be forbidden heresy.
Pretty much any religious topic is going to anger someone. Can the models safely say anything?
That said, this is the NIST, a technical organisation. This collaboration will inform future lawmaking.
Altman has been clear for a long time he wants the government to step in and regulate models (obvious regulatory capture move). They haven't done it, and no amount of Elon Musk or Joe Rogan influence can get people to care, or see it as anything other than regulatory capture. This is OpenAI moving forward anyway, but they can't be the only ones. Hey Anthropic, get in...
- It makes Anthropic "the other major provider", the Android to OpenAI's Apple
- It makes OpenAI not the only one calling for regulation
It reminds me of when Ted Cruz would grill Zuck on TV, yell at him, etc. - it's just a show. Zuck owns the senators, not the other way around. All the big players in our economy own a piece of the country, and they work together to make things happen - not the government. It's not a cabal with a unified agenda, there are competing interests, rivalries, and war. But we the voter aren't exposed to the real decision-making. We get the classics: Abortion, same-sex marriage, which TV actor is gonna win president - a show.
Related
Anthropic CEO on Being an Underdog, AI Safety, and Economic Inequality
Anthropic's CEO, Dario Amodei, emphasizes AI progress, safety, and economic equality. The company's advanced AI system, Claude 3.5 Sonnet, competes with OpenAI, focusing on public benefit and multiple safety measures. Amodei discusses government regulation and funding for AI development.
OpenAI promised to make its AI safe. Employees say it 'failed' its first test
OpenAI faces criticism for failing safety test on GPT-4 Omni model, signaling a shift towards profit over safety. Concerns raised on self-regulation effectiveness and reliance on voluntary commitments for AI risk mitigation. Leadership changes reflect ongoing safety challenges.
AI companies promised to self-regulate one year ago. What's changed?
AI companies like Amazon, Google, and Microsoft committed to safe AI development with the White House. Progress includes red-teaming practices and watermarks, but lacks transparency and accountability. Efforts like red-teaming exercises, collaboration with experts, and information sharing show improvement. Encryption and bug bounty programs enhance security, but independent verification and more actions are needed for AI safety and trust.
US lawmakers send a letter to OpenAI requesting government access
US lawmakers have urged OpenAI to enhance safety standards, allocate resources for AI safety research, and allow pre-deployment testing, following whistleblower allegations and concerns about AI risks and accountability.
OpenAI and Anthropic will share their models with the US government
OpenAI and Anthropic have partnered with the U.S. AI Safety Institute for pre-release testing of AI models, addressing safety and ethical concerns amid increasing commercialization and scrutiny in the AI industry.