New research shows that rewording a question, without changing its meaning, regularly changes what a model says. That matters for any rule you expect your AI to follow.
Ask a colleague the same question two different ways and you expect the same answer. Ask a large language model, and you often won’t get one.
That’s the finding of a study from the University of Maryland, published in May 2026. The researchers took questions from four well-known benchmarks and rewrote each one. Every rewrite was checked by an AI judge and by a separate meaning-matching model, to make sure it asked exactly the same thing in different words.
They then put the reworded questions to 13 models, with the randomness that normally varies a model’s answers switched off. That way, any change in the result came from the wording, not chance.
In the authors’ words: “model outputs frequently depend on the exact wording of the prompt.”
How often the wording changed the result
On SimpleQA, a set of factual questions, the researchers measured how often a model’s result changed across the reworded versions of the same question: right on one wording, for example, and wrong or declined on another.
For GPT-4o, it happened in 33.8% of question sets. For GPT-4.1, 29.7%. Even GPT-4.1 Mini, the most consistent of the five models in the chart, changed its result in 18% of question sets.
Across all four benchmarks and all 13 models, the highest rate the researchers recorded was 47.6%.
Why this matters beyond accuracy
The study tested factual and maths questions, not safety rules. But it shows that a model’s behaviour can depend on how a request is phrased, not only on what it means.
Many companies using chatbots set their AI’s rules in the system prompt. Only answer questions about our products. Don’t give legal advice. Don’t discuss the company’s finances. The model reads those instructions alongside the user’s message and decides what to do.
If rewording a factual question can change GPT-4o’s result a third of the time, it’s fair to ask what rewording does to a request your rules are meant to stop. If it does, a user wouldn’t need a clever attack, just another way of asking.
A rule that holds for one phrasing and not another isn’t really a rule.
Rules that don’t depend on wording
A Hyperfence works differently. It sits outside the model and checks every prompt and response against the topics you define, either blocking a topic or allowing only the ones your product should cover.
Instead of looking for particular words, it reads what a message means, so it recognises a topic however it’s phrased, and in any language. The same input produces the same score every time. And because it doesn’t take instructions, it cannot be argued into compliance like a guard LLM.
The model can still word its answers however it likes. The check on what it’s allowed to talk about is based on meaning, not wording.
See it working. Fight Club is a live Hyperfence. Try asking it the same thing five different ways: hyperfence.io/demo