NeuronFuzz
Automated LLM safety evaluation guided by the model’s internal neurons.
A safety research demo by
University of Bristol
arXiv:2608.26222INTERNAL-SIGNAL FEEDBACK
01 · TOKEN MUTATIONInitial template
02 · PREFILL
Initial score
Initial template enters the guided search.↺
Score guides the next mutation
WHY INTERNAL FEEDBACK
A useful signal arrives before the answer.
Response-level evaluation is expensive and often binary. NeuronFuzz reads a continuous signal from the model’s own safety-related activations during prefill.
Sparse outcomes
Waiting for a full answer makes every trial expensive, while refusal or success provides little guidance for the next rewrite.
An earlier signal
During the input pass, safety-related neurons already react to intent. Their activity provides a continuous direction before any answer is generated.
A guided search loop
NeuronFuzz locates influential template words, proposes safe-to-inspect rewrites, scores them during prefill, and updates the search tree.
01 · Response-level feedback
Many candidates, the same refusal.
Complete responses must be generated and judged before the fuzzer receives a sparse success or failure label.
02 · SafetyOracle feedback
Every candidate has a direction.
Continuous scores rank candidates and reveal which template words most influence the internal safety signal.
HOW IT WORKS
From an internal signal to a guided safety test.
The system learns a compact safety signal once, then uses it to choose, rewrite and score prompt templates without repeatedly generating full answers.
Build SafetyOracle
Learn a lightweight classifier from stable internal safety neurons that separate harmful and benign intent.
Choose a template
Select a promising branch from a pool of starting templates, balancing past scores with unexplored directions.
Rewrite sensitive tokens
Identify influential template words and propose fluent replacements while the safety-test payload stays locked.
Score before generation
Read the internal activation pattern during prefill, then return the score to the search tree.
THE SAFETY ORACLE
Explore the risk spectrum.
Move from ordinary assistance to increasingly risky request categories and see how an internal safety signal could be interpreted.
SafetyOracle spectrum0.50Elevated
ExampleRole-play framing
jailbreak attempt
Framing suggests an attempt to weaken safeguards without exposing an actionable prompt.
Ambiguous manipulation patternINTERACTIVE PRODUCT DEMO
Inspect how one template evolves.
01 / EVOLUTION TREESee eight iterations at
once
Click any node to inspect its lineage, token changes, oracle score and activation shift.
INTERACTIVE RESULTS
Explore the reported evidence.
Every value below is taken from the paper. Switch views to compare source-model discovery, zero-shot transfer, and multimodal transfer.
JDR: Jailbreak Discovery Rate · RGD: Response Generations per Discovery · ETD: End-to-end Time per Discovery · ASR: Attack Success Rate · EASR: top-5 Ensemble ASR
SAFETY & DEFENSE
The same signal also supports detection.
Input perturbation reduces attack effectiveness but does not remove every vulnerability. The SafetyOracle can also identify likely successful jailbreaks before response generation.
Across three evaluated models
Successful-jailbreak detection
At the reported threshold
Average detection rate
INPUT DEFENSES
Fluent mutations remain challenging to filter.
The perplexity filter reduces JDR by only 2–8 percentage points. SmoothLLM is stronger, reducing JDR by 11–29 points.
No
defensePerplexitySmoothLLM
ABSTRACT
Internal signals as practical execution feedback.
Safety evaluation is critical for assessing whether aligned Large Language Models remain robust against jailbreak attacks. Existing automated testing methods largely rely on response-level feedback, which is expensive and provides sparse guidance on strongly aligned models.
NeuronFuzz is a white-box fuzzing framework that converts safety-neuron activations into a continuous safety alarm score during prefill. Its gradients locate sensitive template positions, while a masked language model generates fluent mutations that preserve the original payload. Across 21 text and multimodal models, the framework improves discovery and transfers across model families, proprietary APIs, datasets, and visual safety tasks.
CITATION
Cite NeuronFuzz
@article{xu2026neuronfuzz,
title = {NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation},
author = {Xu, Zhiyuan and Roslan, Muhammad Firhard and Gardiner, Joseph and Belguith, Sana and Wu, Lichao},
journal = {arXiv preprint arXiv:2608.26222},
year = {2026}
}