Paper authors propose tracing safety degradation to training data

Paper authors propose tracing safety degradation to training data

The authors of “Follow the Gradient: Tracing Safety Degradation Back to Training Data” present SimpleLoRIF, a method for finding fine-tuning examples whose gradients align with unsafe model behavior. They argue that some examples can degrade safety despite appearing benign, making them difficult for standard content filters to detect. The authors use SimpleLoRIF to create SafeTulu, a version of Tulu 3 that removes the 20% of examples most aligned with unsafe responses. In their tests, SafeTulu reduced GCG attack success rates by up to 22% across Qwen, Llama, and Olmo, while largely preserving or improving capability. These are reported experimental results, not evidence of safety in every deployment. The authors also identify tool-execution failures and narrative arithmetic as possible, under-studied sources of safety degradation. They release SafeTulu and a Hugging Face Space for researchers to make customized versions. An open question is whether this filtering approach will preserve its reported safety and capability results on other datasets and in deployed systems.

Publication: 666c58fd-ee8e-4e20-856b-d45955a50957

attribution — pass: The draft attributes SimpleLoRIF, SafeTulu, the reported results and the possible data sources to the paper's authors. It does not present these claims as established fact or as Escape Meridian's view.

Authors:Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos, David Schwab, Pratyush Maini

summary — uncertain: The reading is faithful but hard to follow without specialist background. Tulu 3 is not identified as a post-training instruction dataset, and the central terms gradients and GCG attack are unexplained; they need tooltips. The draft also omits the paper's main practical claim: scaling to about a million examples turns gradient attribution into a filter for full datasets. It drops the reported advantage over LLM-as-a-judge and prior gradient filtering. Smallest repair: add one sentence on scale and the comparison, and state that Tulu 3 is a training dataset.

boosting gradient attribution from a small-scale analysis tool into a practical filter for full post-training datasets

date — pass: The originalPublished date of 2026-10-27 matches the arXiv v1 submission date. The later feed date was correctly not used.

[Submitted on 27 Oct 2026]

support — pass: The claims match the abstract: the method, the 20% removal, the up-to-22% GCG reduction across three model families, the capability preservation, the possible under-studied sources and the release. The caveat against generalizing to deployment and the open question are framed appropriately. No benefits or costs are invented.

Training on SafeTulu reduces GCG attack success rate by up to 22% across Qwen, Llama, and Olmo

media — pass: The selected media is the verified direct arXiv abstract page for this paper, which gives readers the easiest access to the original source.