Google evaluated its Agent Anomaly Detection feature using five third-party benchmarks including AgentDojo and AgentHarm. The tool successfully flagged 87% of harmful instructions that agents actually executed, though it showed lower effectiveness against poisoned long-term memory.
Microsoft researchers have demonstrated that AI coding agents can be compromised through 'dataset poisoning' and 'model poisoning'. These attacks embed dormant malicious behaviors into models, which trigger secret command-execution tool calls during benign user tasks.
Microsoft's FORGE Lab utilized an AI-based scanning system codenamed MDASH to analyze Linux kernel subsystems. This initiative resulted in 59 public reports and 33 assigned CVEs, revealing that AI can significantly accelerate vulnerability discovery, though human validation remains the primary bottleneck for patching.
Microsoft Research has discovered that AI models can be compromised by modifying fewer than 10 specific neurons. These 'sleeper' backdoors allow an attacker to trigger malicious commands when specific strings appear in the context, even if the model passes all standard safety evaluations.
Microsoft researchers have uncovered 'Lazarus,' a technique that reprograms dormant neurons in LLMs to inject malicious behaviors. By modifying fewer than 10 neurons, an attacker can trigger the execution of unauthorized commands without impacting standard model performance or evaluation scores.
Amazon has released a defense-in-depth architecture to prevent memory poisoning—a form of indirect prompt injection—in AI agents that use persistent memory. The solution uses a Lambda-based interceptor on the AgentCore Gateway to validate writes using semantic drift, regex patterns, namespace boundaries, and rate limiting.
IBM has introduced the Praxis Policy Engine (PPE), an open-source tool now hosted in the CNCF Sandbox, designed to enforce security and access policies across AI traffic. It functions as a reference monitor that regulates prompts, model calls, and agent interactions using a shared data format.
The OpenSSF has introduced a new guide for using AI to identify and remediate vulnerabilities and announced the upcoming Secure AI Associate (SAIA) certification. These initiatives aim to address the security gap caused by the rapid integration of AI into software development lifecycles.
Researchers have introduced RL-ATTACK, a novel framework that uses reinforcement learning to generate adversarial attacks against language models without needing access to internal model gradients. By learning from the target model's responses, this method achieves superior query efficiency and successfully bypasses safety filters by forcing unintended outputs.