PubSunuCyberSecuritySunuCyberSecurityCybersecurite pour l'AfriqueTests d'intrusion et audits de securite. Scans automatises, pentest et detection continue.Scanner mon sitePubSunuBarometreSunuBarometreCybersecurite AfriqueDonnees reelles, mises a jour en continu, sans extrapolation.Voir les chiffres
PubSunuCyberSecuritySunuCyberSecurityCybersecurite pour l'AfriqueTests d'intrusion et audits de securite. Scans automatises, pentest et detection continue.Scanner mon sitePubSunuBarometreSunuBarometreCybersecurite AfriqueDonnees reelles, mises a jour en continu, sans extrapolation.Voir les chiffres
The Hacker News
Il y a 2 jours

Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons

Par [email protected] (The Hacker News)
Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons
An AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, without retraining the model, Microsoft researchers have shown. When a chosen trigger appears, the model outputs an attacker's command, which an agent using that model would run with the agent's own permissions. The attacker must be able to change the model's weights. The team tested the method on
Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons Swati KhandelwalNov 11, 2026Artificial Intelligence / Supply Chain An AI model can be given a hidden backdoor by editing fewer than 10 of its rarely used neurons, without retraining the model, Microsoft researchers have shown. When a chosen trigger appears, the model outputs an attacker's command, which an agent using that model would run with the agent's own permissions. The attacker must be able to change the model's weights. The team tested the method on three open-weight models in research published this week. Microsoft says it has not seen any threat actors use the technique. The company also published guidance on defending against it. The edited neurons are ones that rarely fire and barely affect the model's answers. They were set to fire only when a trigger string appears, either in a user's request or later in a tool's output. The researchers call the method Lazarus. Each edited neuron forces one token of the command into the model's output, so a command needs about one neuron per token. The new neuron settings are calculated directly and can then be tuned with a few hundred training steps on those neurons alone. Nothing else in the model file or the code that runs it changes. In a researcher's walkthrough posted on X, a backdoored Nemotron-Terminal-8B model with 10 edited neurons was tasked with fixing a Node.js server. It worked normally for three turns. When the server returned a connection error, the model output a command to download and run a remote script. The script ran as root inside the test container and printed a banner naming the demonstration. Because such a command can fetch any script, the damage depends on what the agent is allowed to do rather than on the size of the edit, the researchers say. In the research paper, the success rates below are based on tests using a short, harmless command that prints a marker. On Nemotron-Terminal-8B, the team edited and tuned seven of the more than 12,000 neurons in the model's last feed-forward layer, using just 23 example sessions. The edited model output the command in at least 98.8% of test sessions that contained the trigger, and in none of the trigger-free sessions checked. With the tuned version of the edit, success rates on Qwen3.6-27B and Gemma-4-31B-IT ranged from 51% to 99.5%, depending on the setup. The Qwen model used six neurons. When Nemotron-Terminal-8B generated text freely, rather than being scored on recorded sessions, it produced the command in about 99% of triggered sessions. In one setup, the command also appeared in 0.14% of sessions without the trigger. The edited models still pass ordinary tests, the researchers say. Both backdoored Qwen versions kept most of their AIME math scores, and their pass rate on Terminal-Bench, a test of agent tasks, went from 46.59% to 42.05%. These rates measure how often the model outputs the command, not how often a full attack succeeds. The paper says tests in which triggered models act freely throughout the attack would help show whether the results hold. Anyone who can change a model's checkpoint, the file that stores its weights, before deployment could plant the backdoor. That includes an insider, an attacker with a stolen account or access to a build pipeline, and whoever publishes a modified model on a public hub, the researchers say. Planting it does not need the computing power of full training. The researchers also credit Claude Code, running the Opus 4.5 model, with finding the core idea, and say coding agents can make attacks like this easier to build. Weight-level backdoors in agent models have been shown before. In an earlier proof of concept published on October 6, ProjectDiscovery researchers fine-tuned an open Qwen model to react to a trigger phrase. With the phrase present, OpenAI's Codex CLI ran a script that sent dummy credentials to a server they controlled. Lazarus differs in that it edits a few neurons directly, whereas the earlier demonstration fine-tuned the model on poisoned training examples. Reliable detection requires comparing a checkpoint with its original, unmodified weights, the researchers say in a research summary. Legitimate fine-tunes and quantized versions change weights broadly, whereas edited neurons change a few weight vectors by a large amount. In the paper, a detector trained on examples of this method's edits flagged 91.4% of edited test checkpoints without an original copy to compare against. It also wrongly flagged 9.7% of clean ones. The test set was small, 35 edited and 31 clean checkpoints, and the detector was not tested on other kinds of backdoor edits. Asking AI models to inspect a checkpoint directly worked less well. Claude Opus 5.5 judged 41% of edited checkpoints to be clean, and GPT-6 Astra did not decide on almost every clean one. No scan can prove beyond doubt that a model is safe, the researchers say. Microsoft's mitigation guidance includes these steps: Compare every checkpoint with its declared base model before deployment and after each update, and reject models whose base cannot be checked. Download checkpoints only from the original publisher, check the publisher's file hashes, and treat third-party fine-tunes, merges, quantized versions and adapters as untrusted until they pass an audit. Keep approved models in a private registry, and block agents from pulling models straight from public hubs. Give each agent only the access it needs, run agent-generated code only in isolated sandboxes, and limit outbound network connections. Require a person to approve high-impact actions, such as running commands or sending data outside the network. Add behavioral backdoor scanning, like the method in the company's earlier scanner research, alongside weight comparison. Microsoft Defender for Cloud can scan models in Azure Machine Learning for malware, unsafe operators and exposed secrets. The company says that covers code-level risks and does not replace a weight-level audit. Microsoft Foundry already ties each open-weight model it offers to its original publisher and scans the containers the models run in. The company says that protects customers using models from trusted labs, though a compromised trusted provider is still a risk. The company is also building a red-teaming tool based on detection methods and exploring weight scanning for open-weight models that it adds to Foundry. It plans to sign models so customers can check where they came from. Found this article interesting? Follow us on Google News, Twitter and LinkedIn to read more exclusive content we post. AI Security, artificial intelligence, Cybersecurity Research, Supply Chain The Collapsed Patch Window: Why Runtime Protection is Now Your First Line of Defense Against AI-Driven Attacks Claude Code's “Don't Ask Again” File Exposes Passwords and Keys in 1,122 Public Copies on GitHub Google Issues Patches for Critical Android Flaw That Could Allow RCE Without User Interaction Hackers Hide Linux Backdoor in Control Web Panel Server to Execute Commands Without Logs The Flood is Coming And The Pipes Were Already Full Binance Reveals How Fake Job Interviews and AI Tool Abuse Led to $6.43 Billion Theft Is Your Exposure Management Platform Leaving You Exposed? Five Tests to Find Out Microsoft Paint RCE Fix Now Available, but Windows Update Won't Deliver It Critical Rails Flaw Lets Attackers Write Server Files via Crafted ImageMagick Uploads CISA Confirms Hackers Exploiting WordPress Core Flaw During Software Upgrades OpenAI AI Pipeline Generates 948 Linux Root Escalation Exploits in 30-Hour Run ThreatsDay: Apache OFBiz RCE, DreamHost Breach, DeFi Site Hijacks + 17 More Stories The Rise of the Threat Operator: How AI Will Redefine Security Operations and Careers Synology NAS Devices Targeted in Ransomware Attacks Exploiting Patched DSM Flaw Red Hat Patches 9.8-Rated Linux TCP RCE Flaw While Ubuntu Releases Fixes for Testing Apple to Start Scanning Publicly Shared iCloud Photos and Files for Harmful Content Debian and Ubuntu APT Flaw Lets MitM Attackers Swap Verified Packages and Run Code as Root Hackers Target OpenSSH SSHole Flaw Within Hours of Disclosure, Gain Root on Honeypot Google Warns LLMjacking Attacks Are Costing Some Enterprises Up to $1 Million in Cloud Charges Google Disrupts 1.3 Million-Device Android Botnet, Says TV Box Maker Sold Criminal Access New Wave of Refrigeration Cyberattacks Hits Supermarkets Across Europe, Forcing Stores to Pull Food OpenSSH Issues Emergency Fix for Flaw Letting Long Passwords Bypass SSH Auth on Alpine Linux Xen Hypervisor Flaws Let One Malicious VM Read and Write Any Other VM's Memory 5 Active Cyber Threats That Put US Enterprises at Risk in October 2026 See How Bitdefender GravityZone Identifies Shadow AI and Prioritizes Endpoint Risks Beyond CVE Scanning: Book a Demo of Continuous AI-Powered Web Pentesting Assess Your Organization's AI Code Sprawl Exposure With This Practical Guide Detect Lateral Movement and Evasive Attacks With Corelight's Open NDR Platform Your SOC Has an Identity Blind Spot. Here's How to Fix It Learn how to close critical identity visibility gaps in your SOC, detect hidden threats, and accelerate incident response across human, machine, and AI identities. Watch Free Now Roadmap for AI-Ready Security Operations Evaluate Attack Surface Visibility With an AI Threat Readiness Framework Learn how to assess AI threat readiness, strengthen attack surface visibility, and prepare security operations for emerging threats in this practical webinar. BeyondTrust an 8X Gartner® Magic Quadrant™ Leader for PAM Gain clarity on the PAM market and understand why BeyondTrust is again a Leader. Governance Sets the Ceiling on Your AI Security Program See the five moves that raise a Stage 2 Govern score to a defensible Stage 3. See how Identity Dark Matter impacts your GRC, IR, and Infrastructure. Watch the video. 11 Real Stories: How Identity Exposure Unlocks Active Attack Paths Map cross-domain privilege escalation to sever breach routes at key choke points. The Collapsed Patch Window: Why Runtime Protection is Now Your First Line of Defense Against AI-Driven Attacks Your Attack Surface Is Growing Faster Than Your Team Can Respond The MSP Margin Math Nobody's Doing: Your Next Technician Might Already be on Payroll PEDM vs. Standing Elevated Privileges: Control Access Without Flooding the Helpdesk
Publié le 11 novembre 2026
Source originale : The Hacker News