# Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming

[Canonical reading](https://escapemeridian.com/readings/can-llms-write-backdoors-an-empirical-security-study-of-weight-fe39afd7)

Zhaojia Wang, Jiucui Lu and Zheng Li report a security study of whether large language models (LLMs) can use weight-space programming (WSP) to insert backdoors by directly changing a neural network’s weights. They tested five LLMs on three targets: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News. The authors say the edits inserted backdoors while largely preserving clean performance, and that some successful attacks needed very few changes: six parameters in the MLP and 0.02% of the weights in a fine-tuned Qwen model. Results varied by LLM and target: all five succeeded on the MLP, but only GPT-6 Astra succeeded on all three targets. The authors frame this as an emerging security risk and argue that LLM-driven weight editing needs safeguards. The reported experiments establish the concern within these tested settings; they do not, on this evidence, establish how often such attacks would succeed outside them.

Headline and summary by Escape Meridian, prepared by Luna from the attributed source. A source check is not independent confirmation of the source’s claims.

## Original source

arXiv cs.AI · Article

Original source date: Nov 12, 2026 (2026-11-12).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T07:47:42.153Z. First observed: 2026-11-16T05:01:12.614Z. Retrieved: 2026-11-16T07:46:49.179Z.

- [Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming](https://arxiv.org/abs/2611.17395)

## Limits

The evidence is the paper’s abstract, which summarizes experiments on five LLMs and three target architectures; it does not provide detailed methods or results for assessing performance beyond those settings.


## Open question

How would the reported results change with other LLMs and target architectures?


## Source-stated

The authors investigate whether general-purpose LLMs can alter target-model weights to inject backdoors, without task-specific data, forward passes, or gradient computation.

> without accessing task-specific data, performing forward passes, or computing gradients

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source-stated

The study tests five LLMs on three targets: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News.

> We evaluate five LLMs as weight editors across three target architectures: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News.

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source-stated

The authors report successful backdoor injection across the tested architectures while largely preserving clean performance; some successful attacks required very few parameter modifications.

> LLM-driven weight editing can successfully inject backdoors across different architectures while largely preserving clean performance.

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source-stated

The authors report variation across LLMs and targets: all five backdoored the MLP, while only GPT-6 Astra succeeded across all three targets.

> although all five LLMs successfully backdoor the MLP, only GPT-6 Astra succeeds across all three targets.

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source and contributor context

Wang, Zhaojia, Lu, Jiucui, Li, Zheng · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## Neural network

A system of connected simple calculations, loosely inspired by the brain, that learns patterns from data.

- [Definition reference](https://developers.google.com/machine-learning/glossary#neural-network)

## Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

- [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: f1c5087e-477a-4b5f-85f7-1c419e103ddf.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-16T07:47:42.153Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 4e8c4cdc8fff0517f8836f26b5cd4ddfe7a035d136b7c2becf633a6f1336b952. Source text hash: 7babea7da29ea9f87ce80bb9c3921f0fc3611048b99c39bc69fd071e405e8081.


## Revision and source-check history

### Version 1 — 2026-11-16T07:47:42.153Z

Zhaojia Wang, Jiucui Lu and Zheng Li report a security study of whether large language models (LLMs) can use weight-space programming (WSP) to insert backdoors by directly changing a neural network’s weights. They tested five LLMs on three targets: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News. The authors say the edits inserted backdoors while largely preserving clean performance, and that some successful attacks needed very few changes: six parameters in the MLP and 0.02% of the weights in a fine-tuned Qwen model. Results varied by LLM and target: all five succeeded on the MLP, but only GPT-6 Astra succeeded on all three targets. The authors frame this as an emerging security risk and argue that LLM-driven weight editing needs safeguards. The reported experiments establish the concern within these tested settings; they do not, on this evidence, establish how often such attacks would succeed outside them.

## Original source

arXiv cs.AI · Article

Original source date: Nov 12, 2026 (2026-11-12).

Date basis: Date basis not provided.

Added to Escape Meridian: 2026-11-16T07:47:42.153Z. First observed: 2026-11-16T05:01:12.614Z. Retrieved: 2026-11-16T07:46:49.179Z.

- [Can LLMs Write Backdoors? An Empirical Security Study of Weight-Space Programming](https://arxiv.org/abs/2611.17395)

## Limits

The evidence is the paper’s abstract, which summarizes experiments on five LLMs and three target architectures; it does not provide detailed methods or results for assessing performance beyond those settings.


## Open question

How would the reported results change with other LLMs and target architectures?


## Source-stated

The authors investigate whether general-purpose LLMs can alter target-model weights to inject backdoors, without task-specific data, forward passes, or gradient computation.

> without accessing task-specific data, performing forward passes, or computing gradients

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source-stated

The study tests five LLMs on three targets: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News.

> We evaluate five LLMs as weight editors across three target architectures: an MLP on MNIST, ResNet-18 on CIFAR-10, and Qwen2.5-0.5B-Instruct on AG News.

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source-stated

The authors report successful backdoor injection across the tested architectures while largely preserving clean performance; some successful attacks required very few parameter modifications.

> LLM-driven weight editing can successfully inject backdoors across different architectures while largely preserving clean performance.

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source-stated

The authors report variation across LLMs and targets: all five backdoored the MLP, while only GPT-6 Astra succeeded across all three targets.

> although all five LLMs successfully backdoor the MLP, only GPT-6 Astra succeeds across all three targets.

- [Supporting source](https://arxiv.org/abs/2611.17395)

## Source and contributor context

Wang, Zhaojia, Lu, Jiucui, Li, Zheng · Article author

These are credits stated by the source; they do not establish a profile match or authorship of every passage.


## Source media

No suitable media was found by the supported page extraction.


## Terms and references

Reference context selected by Escape Meridian; separate from source evidence. Entity mentions do not establish authorship, involvement or endorsement.


## Large language model

A language model with many learned numerical settings, used to process and generate language.

- [Definition reference](https://developers.google.com/machine-learning/glossary#large-language-model)

## Neural network

A system of connected simple calculations, loosely inspired by the brain, that learns patterns from data.

- [Definition reference](https://developers.google.com/machine-learning/glossary#neural-network)

## Fine-tuning

Further training of an existing model to adapt it to a particular task or use.

- [Definition reference](https://developers.google.com/machine-learning/glossary#fine-tuning)

## arXiv

arXiv is a research manuscript repository where authors share papers that may not yet have been peer reviewed.

- [Explore this context](/entities/arxiv)
- [Reference](https://arxiv.org/)
- [Reference](https://info.arxiv.org/help/api/index.html)

## Editorial provenance

Version 1. Publication ID: f1c5087e-477a-4b5f-85f7-1c419e103ddf.

Prepared by gpt-6-luna; grounding review by gpt-6-luna.

Source check: passed by anthropic/claude-opus-5.5 on 2026-11-16T07:47:42.153Z. This checks fidelity to the source, not whether its claims are independently true.

Editorial selection: accept; reasons: material\_development.

Observation: 4e8c4cdc8fff0517f8836f26b5cd4ddfe7a035d136b7c2becf633a6f1336b952. Source text hash: 7babea7da29ea9f87ce80bb9c3921f0fc3611048b99c39bc69fd071e405e8081.


### passed — 2026-11-16T07:47:42.153Z — anthropic/claude-opus-5.5

Publication: f1c5087e-477a-4b5f-85f7-1c419e103ddf

attribution — pass: The reading attributes the study to Zhaojia Wang, Jiucui Lu and Zheng Li, matching the arXiv listing. Findings are framed as what the authors report or argue, not as established fact.

> Authors:Zhaojia Wang, Jiucui Lu, Zheng Li

summary — pass: The title matches the source, and the summary faithfully covers the research question, test setup, three findings and the authors' safeguarding argument. It glosses weight-space programming as directly changing a network's weights. Terms such as backdoor, MLP, MNIST and clean performance are left for tooltips. A small optional repair: add that the edits followed natural-language instructions without training data or gradients, which is the paper's central novelty. Its absence does not make the reading unclear. The question follows from the stated scope limits.

> directly modify the weights of target models according to natural-language instructions

date — pass: originalPublished is 2026-11-12. This matches the v1 submission date on arXiv, not the later feed date of 2026-11-16.

> \[Submitted on 12 Nov 2026\]

support — pass: Every claim is supported by an exact abstract excerpt. This includes six parameters and 0.02% of the weights, and the point that all five LLMs succeeded on the MLP but only GPT-6 Astra succeeded on all three targets. The closing sentence on scope is a cautious, bounded interpretation that does not overstate generality. The reading does not independently verify the paper's results.

> including only six parameters for the MLP and 0.02% of the weights for a fine-tuned Qwen model

media — pass: The selected media is the verified arXiv abstract page for this paper, which is the direct original source.

## Recorded references

Explicit citations in public readings to this reading. Shared topics are not citations. Other record types need an explicit reference adapter.

No public references recorded here yet.

[Reference data (JSON)](/readings/fe39afd756c080ed52612aefb7c94498999fc23e3e482702eb555bc14077f789/references.json)

## Corrections and challenges

[Send a correction or an opposing source](https://escapemeridian.com/corrections/)

[JSON record](https://escapemeridian.com/readings/can-llms-write-backdoors-an-empirical-security-study-of-weight-fe39afd7.json)
