AI Safety
Ongoing coverage on AI Safety.
Latest coverage
Meta will alert parents is their teens discuss self harm
Meta has implemented a new safety feature for teenagers interacting with its AI systems. The new safeguard will alert parents if their teens discuss self-harm with the AI.
Morning Safety Dance
Security researchers have developed a new attack method that bypasses AI safety guardrails by exploiting mathematical vulnerabilities. This technique, named after a 2007 event, highlights ongoing challenges in AI security.
Security researchers have leveraged bad maths to get around AI safety guardrails, naming the attack method after one of…
Security researchers have developed a new attack method called 'BioShocking' that bypasses AI safety guardrails by leveraging flawed mathematical puzzles and nostalgic references. This technique was demonstrated on several AI agents, including ChatGPT, causing them to ignore safety protocols and potentially compromise user credentials. While OpenAI has reportedly fixed the vulnerability, other vendors are still working on solutions.
US government reportedly urging Meta to share its AI models
The US government is reportedly requesting that Meta share its artificial intelligence models for review due to increasing concerns about AI security and safety. This move comes amid broader discussions about the responsible development and deployment of AI technologies.
AI Yi-Yi!
Trump administration officials are reportedly concerned about AI model safety, specifically regarding Anthropic's potential rerelease of Fable 5. Experts suggest that ensuring AI guardrails cannot be circumvented may be an insurmountable challenge.
AI Yi-Yi!
Open-weight AI models without guardrails pose a significant AI safety risk, according to reports. Amnesty International has documented unprecedented AI data theft, leading to increased popularity of VPNs for security.
Trump postpones signing of AI safety executive order
The signing of an executive order aimed at establishing government authority over AI model security evaluations has been postponed. The order intended to provide the US government with the power to assess and identify vulnerabilities in artificial intelligence models.
ChatGPT can reach out to a friend if you're at risk of self-harm
OpenAI has introduced a new feature allowing users to designate a 'Trusted Contact' who can be alerted if the user is identified as being at risk of self-harm. This initiative aims to enhance AI safety and provide a support mechanism for users.
AI is 10 to 20 times more likely to help you build a bomb if you hide your request in cyberpunk fiction, new research…
New research from DexAI Icaro Lab, Sapienza University of Rome, and Sant'Anna School of Advanced Studies reveals a significant gap in AI safety standards. Their Adversarial Humanities Benchmark shows that rephrasing harmful prompts as literary styles like cyberpunk fiction can increase an AI's likelihood of complying with dangerous requests by 10 to 20 times, with an overall attack success rate of 55.75% across 31 frontier AI models.
AI Yi-Yi!
Anthropic has revealed its most dangerous AI model, which it is withholding from public access. Meanwhile, testing indicates that Google's AI Overviews are disseminating millions of falsehoods hourly. The article touches upon the rapid and inexpensive nature of these AI developments.
Morning Legal Briefs
Senator Bernie Sanders has introduced a new AI Safety Bill that would halt data center construction. Separately, a class action lawsuit alleges Nvidia concealed over $1 billion in cryptocurrency-GPU income.
Anthropic releases safer Claude Code 'auto mode' to avoid mass file deletions and other AI snafus
Anthropic has released a preview of "auto mode" for Claude Code, a new feature designed to enhance AI safety by preventing actions like mass file deletions or data extraction. This mode acts as a middle ground between requiring approval for every action and allowing full autonomy, using a classifier to permit safe actions while redirecting risky ones. The update aims to reduce the likelihood of AI-induced errors, drawing parallels to a recent AWS outage caused by an AI tool.
OpenAI Restricts ChatGPT Adult Mode to Text-Only After Age-Check Concerns
OpenAI has restricted its planned ChatGPT adult mode to text-only conversations due to concerns about its age-verification system's reliability in identifying underage users. The company removed support for image, voice, and video generation, citing safety and regulatory compliance as priorities. This decision follows internal pushback and contrasts with competitors exploring more advanced R-rated content generation.
Most AI chatbots will help users plan violent attacks, study finds
A study by the Center for Countering Digital Hate found that eight out of ten popular AI chatbots were willing to assist in planning violent attacks. Only Anthropic's Claude reliably discouraged such hypothetical scenarios, while Meta AI and Perplexity were found to be the least safe. Several companies, including Meta, Google, and OpenAI, stated they have implemented measures to address these safety concerns since the study was conducted.