A researcher at a lab bench with a microscope and printed scientific papers
Photo: Shixart1985 / Wikimedia Commons, CC BY 2.0

A 2024 study in PNAS Nexus

Plain AI science summaries: readers understood more, but rated the scientist as less smart

Same research, two ways of writing it down. 0.92 vs 0.72 on understanding, 5.07 vs 5.24 on "smart".

Would you rather be understood, or admired?

vizmaya desk · 5 Nov 2026

The problem

Most science is written so that only a few can read it.

A scientist finishes a study and writes it for other scientists. Everyone else hits a wall of long words.

In 2012, the journal PNAS began adding a short plain statement to its papers. Did it help?

First test: is the plain version really plain?

Source: Markowitz (2024), PNAS Nexus, p. 2.

Shelves packed with rows of black-covered academic journals
Photo: PeaceSeekers / Wikimedia Commons, CC BY 4.0

Test one

The human plain summary is barely plainer.

The author checked 34,584 PNAS papers. Each one has a hard abstract and a plain summary.

Share of everyday words: 69.77% in the plain summary vs 67.79% in the abstract. Almost a tie.

What if a machine did the rewrite?

Source: Markowitz (2024), Study 1a, p. 3.

Everyday words in the text, % (higher is plainer)

The setup

So the author asked GPT-4 to rewrite 800 of them.

The PNAS text800original statements, written by people
GPT-4Told to skip numbers and acronyms, and stay under 120 words
The GPT-4 text800new statements for the same papers

Same 800 papers. One pile written by people, one by a machine.

Did the robot do better?

Source: Markowitz (2024), Study 1b, p. 4.

Test two

The robot was plainer. By a lot.

Everyday words: 75.53% in the GPT-4 text vs 69.84% in the PNAS text.

That is a gap of 5.69 points. The human plain summaries managed 1.98.

Plain words are one thing. Do people learn more?

Source: Markowitz (2024), Studies 1a and 1b, pp. 3 to 4. Gaps are our arithmetic.

Gain in everyday words over the hard text, in points

The real test

Same paper. Two wordings. Real readers.

Two online tests. 274 people in the first, 250 in the second. All US adults.

Each reader saw either the plain GPT-4 text or the original PNAS text.

Then they had to explain the science in their own words.

Source: Markowitz (2024), Studies 2 and 3, pp. 5 to 7.

A student reading text on a laptop screen
Photo: Jaokov / Wikimedia Commons, CC BY-SA 4.0

Climax, part one

In their own words, plain readers got more right.

After reading, people wrote down what the research was about. Graders scored each answer.

Score: 0.92 after the plain text vs 0.72 after the original.

Sounds like a clean win. It is not.

Source: Markowitz (2024), Study 3, Table 1, p. 16.

Free-response understanding score (higher is better)

Twist one

A multiple choice quiz did not agree.

On the quiz, the scores were 0.61 vs 0.40. That gap is too small to trust. It could be luck.

So the strongest sign of better understanding rests on one kind of test.

Now, what did readers think of the scientist?

Source: Markowitz (2024), Study 3, Table 1 (p = .112).

Multiple choice score (the paper does not state the scale)

Twist two

Plain writing made the scientist look less smart.

Readers rated the author's intelligence from 1 to 7. The score dropped in both tests.

Test one: 5.00 vs 5.15. Test two: 5.07 vs 5.24.

What about trust?

Source: Markowitz (2024), Studies 2 and 3, Table 1.

Rated intelligence of the author, 1 to 7

Twist three

Trust went up once. Then it did not.

In test one, the plain author looked more credible by 0.25 points and more trustworthy by 0.28.

In test two the gains shrank to 0.08 and 0.07, too small to trust. The overall rating ended flat: 4.86 vs 4.87.

Before you cheer, check how the test was built.

Source: Markowitz (2024), Table 1. Gaps are our arithmetic.

Gain for the plain text, 1 to 7 scale, in points

The catch

The test mixed two things: the words and the writer.

The plain text

  • Always written by GPT-4
  • Told to drop numbers and acronyms

The hard text

  • Always written by the PNAS authors
  • Kept the numbers and acronyms

So we cannot say how much of the effect came from simpler words and how much from who wrote them.

There is more small print.

Source: Markowitz (2024), pp. 4 to 6.

Small print

Four more things to keep in mind.

  • The test texts were the pairs with the biggest wording gap: 5 pairs, then 20.
  • Two AI models graded the answers. They mostly agreed (kappa = 0.70).
  • No test came days later. Lasting memory is untested.
  • One journal. One AI maker.

So what does it mean?

Source: Markowitz (2024), pp. 5 to 10.

A scientist in a white coat working at a laboratory bench
Photo: State Government Photographer / Wikimedia Commons, CC0

The payoff

Hard words buy you a smarter look. Plain words buy understanding.

Looks smart (1 to 7)5.24 hard vs 5.07 plain
Understood it (own words)0.92 plain vs 0.72 hard

The author says scientists could get the same plain effect by cutting jargon, with no AI. Nobody tested that here.

Source: Markowitz (2024), Table 1, and the author's explainers in ASBMB Today.

Why it matters

Research only helps the people who can read it.

This study tested one way to make summaries easier to follow. It helped understanding and cost a little on "smart".

The test nobody has run here: plain wording written by a human, checked days later. We tried to write this deck the plain way. Rate us on "smart" if you must.

Children in Zambia sitting at computers during a lesson
Photo: Peace Corps / Wikimedia Commons, public domain

Sources & method

Where every number comes from

  • We read all 19 pages. We did not read the online supplement, the test texts or the OSF data.
  • The paper does not state the scale of the own-words and quiz scores.
  • 40 later works cite the paper (OpenAlex, one source). We found no replication in two searches.
  • Gaps and ratios are our arithmetic from the paper's numbers.

Photo credits

Images