2.3. Standard Deviation

Not all measurements are done with instruments whose error can be reliably estimated. A classic example is the measuring of time intervals using a stopwatch. Of course, there will be a read-off error as discussed in the previous sections. However, that error will be negligible compared to the dominant error, the one coming from the fact that we, human beings, serve as the main measuring device in this case. Our individual reaction time in starting and stopping the watch will be by far the major source of imprecision. Since humans don't have built-in digital displays or markings, how do we estimate this dominant error?

The solution to this problem is to repeat the measurement many times. Then the average of our results is likely to be closer to the true value than a single measurement would be. For instance, suppose you measure the oscillation period of a pendulum with a stopwatch five times.

You obtain the following table:

Our best estimate for the oscillation period is the average of the five measured values:

Note that N in the general formula stands for the number of values you average.

Now, what is the error of our measurement? One possibility is to take the difference between the most extreme value and the average. In our case the maximum deviation is ( 3.9 s - 3.6 s ) = 0.3 s. If we quote 0.3 s as an error we can be very confident that if we repeat the measurement again we will find a value within this error of our average result.

The trouble with this method is that it overestimates the error. After all, we are not interested in the maximum deviation from our best estimate. We are much more interested in the average deviation from our best estimate. So should we just average the differences from our measured values to our best estimate? Let's try:

Clearly, the average of deviations cannot be used as the error estimate, since it gives us zero. In fact, the definition of the average ensures that the average deviation is always zero for any set of measurements. It is so because the deviations with positive sign are always canceled by the deviations with negative sign. Can't we get rid of the negative signs? We can. If we square our deviations, all numbers will be positive, so we'll never get zero¹. We should then not forget to take the square root since our error should have the same units as our measured value. Thus we arrive at the famous standard deviation formula²

The standard deviation tells us exactly what we were looking for. It tells us what the average spread of experimental results is about the mean value. Now we can write our final answer for the oscillation period of the pendulum:

What if we can't repeat the measurement? The error estimation in that case becomes a difficult subject, one we won't go into in this tutorial. In your laboratory, the majority of relevant measurements are easily repeatable.

¹Of course, there are many other ways to get rid of the minus signs. For example, we could have just used absolute values. So why use squares? Isn't the choice of how to define standard deviation somewhat arbitrary? The answer is that using squares gives the standard deviation a crucial property that it would lack if we used absolute values or any other function to remove the minus signs, namely that the value of x which minimizes the standard deviation will be the average value of x.

²You may have seen another definition of the standard deviation in which N is replaced by (N-1). It's more of a mathematical subtlety, which does not affect our reasoning here. You can use either one of the two definitions in your lab. The difference between the two is negligible for large N.