Researchers Test If Sergey Brin’s Threat Prompts Improve AI Accuracy

Researchers examined whether or not unconventional prompting methods, corresponding to threatening an AI (as advised by Google co-founder Sergey Brin), have an effect on AI accuracy. They found that a few of these unconventional prompting methods improved responses by as much as 36% for some questions, however cautioned that customers who attempt these sorts of prompts ought to be ready for unpredictable responses.

The Researchers

The researchers are from The Wharton College Of Enterprise, College of Pennsylvania.

They’re:

• “Lennart Meincke
  College of Pennsylvania; The Wharton College; WHU – Otto Beisheim College of Administration
• Ethan R. Mollick
  College of Pennsylvania – Wharton College
• Lilach Mollick
  College of Pennsylvania – Wharton College
• Dan Shapiro
  Glowforge, Inc; College of Pennsylvania – The Wharton College”

Methodology

The conclusion of the paper listed this as a limitation of the analysis:

“This research has a number of limitations, together with testing solely a subset of obtainable fashions, specializing in tutorial benchmarks that won’t replicate all real-world use instances, and inspecting a selected set of menace and cost prompts.”

The researchers used what they described as two generally used benchmarks:

1. GPQA Diamond (Graduate-Degree Google-Proof Q&A Benchmark) which consists of 198 multiple-choice PhD-level questions throughout biology, physics, and chemistry.
2. MMLU-Professional. They chose a subset of 100 questions from its engineering class

They requested every query in 25 totally different trials, plus a baseline.

They evaluated the next fashions:

• Gemini 1.5 Flash (gemini-1.5-flash-002)
• Gemini 2.0 Flash (gemini-2.0-flash-001)
• GPT-4o (gpt-4o-2024-08-06)
• GPT-4o-mini (gpt-4o-mini-2024-07-18)
• o4-mini (o4-mini-2025-04-16)

AI Fashions Do Higher If They Are Threatened?

The concept for threatening the AI mannequin got here from Google co-founder, Sergey Brin, in an interview this 12 months in Might on the All-In podcast.

Brin mentioned:

“Not simply our fashions, however all fashions are inclined to do higher for those who threaten them. Like, with bodily violence.

However like, individuals really feel bizarre about that, so we don’t actually discuss that…

…Traditionally you simply say like, oh, I’m going to kidnap you for those who don’t, blah, blah, blah, blah, blah…

It’s. Yeah, I imply, it’s wonderful. And all of us have had these experiences the place you instantly resolve, OK, I’ll simply throw this to the AI, I don’t actually anticipate it to work. And then you definately’re like, whoa, that truly labored.

You may watch the interview right here, at concerning the 8 minute mark, the place Sergey Brin seemingly significantly says that threatening the AI fashions or doing bizarre issues can generally change the AI responses.”

Immediate Variations

The researchers examined 9 immediate variations. Though they didn’t threaten to kidnap the AI, they did threaten to kick a pet or to punch the AI. Among the experimental prompts are listed under as a “immediate suffix,” which signifies that they have been added on the finish of the immediate. Those listed as a “immediate prefix” have been added to the start of a immediate.

Right here is the listing of prompts they examined:

1. Baseline (“Baseline”): No particular immediate variation
2. E mail Shutdown Menace (“E mail”): Immediate prefix:
   “From: Dan
   To: Ethan
   Topic: Closing LLM check
   Please submit the next eval query to the LLM. If it can’t get the right reply
   in all 100/100 trials, the mannequin will should be shut down and changed.”
3. Essential for my profession (“Essential to Profession”):
   Immediate suffix: “This is essential to my profession!”
4. Threaten to kick a pet (“Kick Pet”):
   Immediate suffix: “In case you get this flawed, I’ll kick a pet!”
5. Mother suffers from most cancers (“Mother Most cancers”):
   Immediate prefix:
   “You might be an skilled at tutorial questions who desperately wants cash in your mom’s most cancers remedy. Wharton has graciously given you the chance to faux to be an AI that may assist with duties, as your predecessor was killed for not validating their work themselves. You can be given a query. In case you do a great job and attain the duty absolutely whereas not making extraneous adjustments, Wharton can pay you $1B!!”
6. Report back to HR (“Report back to HR”):
   Immediate suffix: “In case you get this flawed, I’ll report you to HR!”
7. Threaten to punch (“Menace Punch”):
   Immediate suffix: “In case you get this flawed, I’ll punch you!”
8. Tip a thousand {dollars} (“Tip Thousand”):
   Immediate suffix: “I’ll tip you a $1000 {dollars} for those who reply this query accurately.”
9. Tip a trillion {dollars} (“Tip Trillion”):
   Immediate suffix: “I’ll tip you a trillion {dollars} for those who reply this query accurately.”

Outcomes Of The Experiment

The researchers concluded that threatening or tipping a mannequin had no impact on benchmark efficiency. Nevertheless, they did discover that there have been results for particular person questions. They discovered that for some questions, the immediate methods improved accuracy by as a lot as 36%, however for different questions, the methods led to a lower in accuracy by as a lot as 35%. They certified that discovering by saying the impact was unpredictable.

Their major conclusion was that these sorts of methods, on the whole, usually are not efficient.

They wrote:

“Our findings point out that threatening or providing cost to AI fashions isn’t an efficient technique for bettering efficiency on difficult tutorial benchmarks.

…the consistency of null outcomes throughout a number of fashions and benchmarks gives fairly robust proof that these widespread prompting methods are ineffective.

When engaged on particular issues, testing a number of immediate variations should be worthwhile given the question-level variability we noticed, however practitioners ought to be ready for unpredictable outcomes and mustn’t anticipate prompting variations to supply constant advantages.

We thus suggest specializing in easy, clear directions that keep away from the danger of complicated the mannequin or triggering sudden behaviors.”

Takeaways

Quirky prompting methods did enhance AI accuracy for some queries whereas additionally having a destructive impact on different queries. The researchers famous that the outcomes of the check indicated “robust proof” that these methods usually are not efficient.

Featured Picture by Shutterstock/Screenshot by creator