The Law of Large Numbers

What it is

In simple words, the law of large numbers shows that as we increase the number of trials, or the size of a sample, the sample mean gets closer to the true mean. This law explains some statistical equations as well as some of the concepts of sampling. For example, one equation of standard deviation has sample size in the denominator. What this means is that, as we increase this sample size, the total fraction would get smaller, meaning that the standard deviation diminishes and we are more certain about our sample mean.

Coin toss

Imagine the scenario of a coin toss (assume that heads is success). If we do 10 tosses, we may well get 7 heads - or a 70% success rate (20% away from the true mean). If we do a 100, we may get 43 heads - or a 43% success rate (7% away). As we increase the number of trials, the percent away from the true mean decreases according to the law of large numbers. Therefore, doing, say, a million tosses may give us 500,544 - or an approximate success rate of 50% which is the true mean.

In theory, the million coin tosses might give a success rate of 25%, you may be thinking - which means that the law of large numbers wouldn’t apply here, right? Well, a success rate of 49% or less, or 51% or more is extremely unlikely*.

So, we could say something along the lines of: According to the law of large numbers, a coin toss gives heads 50% of the time, and tails otherwise.

* In fact, getting a 49% rate (490,000 heads) in this scenario would be extremely rare. In a million coin tosses, the standard deviation is 500. This means that getting 490,000 heads is 20 standard deviations away from the mean. Being 20 standard deviations away from the mean is virtually impossible. Specifically the probability of this event is about 2 x 10^-92. To visualize this, imagine a person buys a ticket for the largest lottery in the world, where the chance of winning is 1 in 300 million. Now, imagine he wins. Not once, but 10 times in a row. Yes, in a row. This is more probable than being 20 standard deviations away from the mean. I have written an article related to this: The Bell Curve. It’s an interesting read if you’d like to know more about this.

Further applications

The law of large numbers doesn’t only apply to simple coin tosses or dice rolls. It can apply to more complex areas as well. This often merges with the bell curve to find out probabilities of an event, and what’s most likely to happen. For example, in an experiment of finding the average weight of middle aged women in a chosen city, we may sample 50 women. From this, we may claim that the average weight is the mean we get, with confidence intervals (this is another interesting concept which will be discussed in a future article).