The Curious Case of Fading Superstars: Regression to the Mean

The blog title "The Curious Case of Fading Superstars: Regression to the Mean" set across an image of a field with plots of crops growing

Have you ever selected a genotype from a screening trial because it clearly outperformed all the others, only for that exceptional performance to never be repeated? What initially appeared to be a “superstar” seems much less remarkable when evaluated again.

Or perhaps you have seen a treatment produce excellent results in an initial experiment but never perform quite as well again. What first appeared to be a miracle treatment looks much more ordinary when tested again.

If so, you may have experienced an interesting, and surprisingly common, statistical phenomenon: regression to the mean.

What is Regression to the Mean?

Regression to the mean is a statistical phenomenon in which unusually high or unusually low results are, on average, followed by results closer to the overall average.

Why is this? Real-world results are influenced by both genuine underlying effects and random variation. An exceptionally good result may have been helped along by good luck, such as particularly favourable growing conditions, while an exceptionally poor result may partly reflect bad luck, such as unusually unfavourable conditions.

When we measure the same thing again, those random influences are unlikely to be quite so extreme, so the result tends, on average, to move back toward the overall average.

A Simple Example of Regression to the Mean

Imagine a plant breeder evaluating hundreds of lines in an early-stage trial. Some lines genuinely have high genetic potential. However, random variation also plays an important role. Small differences in soil conditions, microclimate, pest pressure, or other chance factors beyond the breeder’s control can all influence the observed performance of a line. The very top-performing lines are therefore often a combination of:

  1. genuinely strong genetics 
  2. unusually favourable random variation

When the breeder tests those same lines again, some of the good fortune that contributed to the original standout result is unlikely to be repeated. As a result, many “superstar” lines appear far less exceptional the second time around.

Importantly, this does not necessarily mean the original trial was flawed. Nor does it mean the breeder made poor decisions. It is simply a consequence of variability – the random fluctuations that occur in all experiments.

And of course, some genuinely outstanding lines will continue to perform exceptionally well. Regression to the mean simply tells us that, on average, extreme results tend not to be repeated to the same extent.

Scatter plot of genotype performance in an initial versus subsequent trial, showing a standout line regressing toward the average when retested.

Why Does this Matter?

Regression to the mean is one reason why apparently outstanding results often become less impressive when we repeat an experiment. 

This is why replication, randomisation, and validation trials are so important. They help us distinguish truly superior treatments and genotypes from those that simply benefited from unusually favourable circumstances.

The Role of Replication, Randomisation and Validation

Replication, randomisation, and validation trials help reduce the influence of chance on our conclusions.

Replication provides stronger evidence about the true performance of treatments and genotypes, reducing the influence of unusually favourable or unfavourable conditions on the observed results. Randomisation helps ensure that treatments and genotypes experience a representative range of conditions, rather than systematically benefiting from particular locations or times. Validation trials provide an opportunity to confirm that an apparently exceptional result can be reproduced.

Together, these approaches help us distinguish genuine biological effects from random variation. However, they cannot eliminate chance entirely. Even in well-designed experiments, unusually high or unusually low results will sometimes occur simply through random variation.

The Practical Lesson

Regression to the mean is a useful reminder that exceptional results, while exciting, should be interpreted with caution.

Not every standout genotype is destined to remain a superstar.

Not every disappointing result reflects genuine failure.

Not every miraculous treatment response will be repeated.

Careful experimentation, together with sound experimental design, replication, and validation, helps us distinguish genuine biological effects from random variation.


If you’d like to discuss anything from this article with one of the VSNi consultants, please do get in touch.

Picture of Dr. Vanessa Cave
Dr. Vanessa Cave
Dr. Vanessa Cave is an applied statistician interested in the application of statistics to the biosciences, in particular agriculture and ecology, and is a developer of the Genstat statistical software package. She has over 15 years of experience collaborating with scientists, using statistics to solve real-world problems. Vanessa provides expertise on experiment and survey design, data collection and management, statistical analysis, and the interpretation of statistical findings. Her interests include statistical consultancy, mixed models, multivariate methods, statistical ecology, statistical graphics and data visualisation, and the statistical challenges related to digital agriculture. Vanessa is a past President of both the Australasian Region of the International Biometric Society and the New Zealand Statistical Association, on the Editorial Board of The New Zealand Veterinary Journal and an honorary academic at the University of Auckland. She has a PhD in statistics from the University of St Andrew.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Related articles

A genotype that leads one trial can lose its advantage in the next, even when trials sit within...
Experimental and observational studies can both provide valuable insights, but they answer different kinds of questions. This article...
Spatial variation can affect comparisons between varieties or treatments in field trials, but there is no single model...