The Success Paradox: Why even a little Bit of Luck often beats Skill

The skill-vs-luck debate never ceases to fascinate. Is it mainly skill, experience, hard work, and the “right” mindset that determines who is becoming successful, rich, or famous, or is it largely luck and fortunate circumstances?

In this post, we will conduct an illuminating simulation to show that even a small amount of luck can make all the difference! If you want to learn about the success paradox, read on!

This is not the first time that we cover the skill-vs-luck debate on this blog:

• If wealth had anything to do with intelligence…
• The Rich didn’t earn their Wealth, they just got Lucky
• How to be Successful! The Role of Risk-taking: A Simulation Study

In this post we want to talk about a fascinating paradox: in most settings, success will of course depend on a combination of both, i.e. skill and luck. Yet even a little bit of luck, let’s say 5%, will change the outcome considerably.

We are going to rebuild a simulation from a video of the science channel “Veritasium” with R. The whole video is definitely worth a watch: Is Success Luck or Hard Work?

The setup of the simulation is as follows:

The importance of luck increases the greater the number of applicants applying for just a few spaces. Consider the most recent class of NASA astronauts. From over 18,300 applicants in 2017, only 11 were selected and went on to graduate from the astronaut training program.

Now we can make a toy model of the selection process. Let’s assume that astronauts are selected mostly based on skill, experience, and hard work, but also say five percent as a result of luck — fortunate circumstances. For each applicant, I randomly generated a skill score out of a hundred, and I also randomly generated a luck score out of a hundred. Then I added those numbers together, weighted in the 95-to-5 ratio to get an overall score. This score represents the selector’s judgments, meaning the top 11 by this metric would become astronauts.

In R code:

reps <- 1000 # how many repetitions
N <- 18300 # how many applicants
s <- 11 # how many selected
w <- 0.95 # weight of skill, experience, hard work vs. luck, fortunate circumstances

select <- function(N, s, w) {
  skill <- runif(N, max = 100) # skill score
  luck <- runif(N, max = 100) # luck score
  overall <- skill * w + luck * (1 - w) # overall score
  
  selected <- order(overall, decreasing = TRUE)[1:s] # select based on overall score
  skill_alone <- order(skill, decreasing = TRUE)[1:s] # select based on skill alone
  same <- intersect(selected, skill_alone) # which persons would be in both selections?
  
  c(mean(luck[selected]), length(same)) # mean of luck score of selected, # of persons in both selections
}

Veritasium continues:

And I repeated this simulation a thousand times representing a thousand different astronaut selections. And what I found was the astronauts who were picked were very lucky; they had an average luck score of 94.7. So how many of the selected astronauts would have been in the top 11 based on skill alone? The answer was, on average, only 1.6.

We get the exact same results in R:

set.seed(12)
replicate(reps, select(N, s, w)) |> rowMeans() |> round(1)
## [1] 94.7  1.6

Veritasium concludes:

That means, even with luck accounting for just 5% of the outcome, 9 or maybe 10 of the 11 applicants selected would have been different if luck played no role at all. When competition is fierce, being talented and hard-working is important, but it’s not enough to guarantee success.

My own interpretation of this paradoxical result is that there are so many very good candidates that in the end luck decides who will be selected.

A fair criticism of this simulation is that skill might not be uniformly but normally distributed. This certainly reduces the role of luck, but it is still surprisingly important, depending on the respective parameterization of the distribution. You can play around with the code and test different assumptions yourself (e.g. with skill <- rnorm(N, mean = 50, sd = 12) to run the simulation with normally distributed skill scores). I would love to read about that in the comments.

Veritasium continues in this video that although luck plays such a key role successful people think that it was their skill, talent, hard work, and mindset that brought them there. They feel that they deserve it. Some psychological studies (which are not mentioned in the video) take this to the extreme:

Scientists have conducted actual experiments where they had people play a rigged variant of monopoly.

The participants were selected randomly whether they belonged to the privileged or the losers group. In all those studies something telling happened: the “winners” got increasingly confident and extroverted while explaining that they earned their winning because they were smarter and more brilliant. The “losers” got more and more introverted and passive, quietly accepting their fate or, even better, looking for mistakes they made playing the game.

I always have to think about those revealing experiments whenever I hear super privileged people explain why and how they deserve their status and why they are entitled to being so privileged. This, for me, is the real “entitlement society”.

I wish you lots of luck and lots of (resulting) success! Stay tuned for more interesting insights to come and please share your thoughts in the comments.