# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: Standard Deviation Question**

**From:**Tom Sult

**Date:**2013 Jan 05, 10:14 -0600

The thing of general use to me is ... WOW there is a GPS world magazine! Tom Sult Sent from my iPhone On Jan 5, 2013, at 8:10, "Richard B. Langley"wrote: > Warning: Slightly off topic; delete if not interested. > > Not wanting to muddy the waters, but for those interested, here is a link to an article on GPS precision and accuracy that appeared in my GPS World column a few years ago: > http://gge.unb.ca/test/gpsworld.may10.pdf > > There may be something of general use that could also be considered when looking at the statistics of CN observations. > > -- Richard Langley > > On 5-Jan-13, at 6:24 AM, Marcel Tschudin wrote: > >> ... when should I be using stdDevPop and when should I be using stdDevSamp? >> >> I try to explain the difference in a general understandable way. >> >> The calculation of the standard deviation requires to know the mean value. The difference between the two functions results from whether the mean value from the given data set represents the exact mean or is an estimated mean. >> >> If the given data represent all of them, i.e. they represent the complete population, then their mean value is exact and the standard deviation is calculated on the basis of an exact mean value. However, to have a complete population and know the exact mean is rather the exception. It is more likely that we have some selected (measured) values out of a greater population which is assumed infinite and where the exact mean value is unknown; the mean value of the given data set represents therefore an estimate, and the standard deviation is calculated on the basis that the mean value of the data set is an estimate. >> >> The difference between stdDevPop and stdDevSamp is therefore: >> stdDevPop() calculates the standard deviation understanding that the entered data are all of the population and that the mean value of the entered data is the exact mean value. >> stdDevSamp() calculates the standard deviation understanding that the entered data represent a sample from an infinite population and that the mean value of the entered data represents therefore an estimation for an infinite population. >> >> Now, what does the standard deviation mean? This is a measure for a probability that an other data (an other measurement taken under the same condition) will be within (or outside) certain limits. If we look at Greg's last data and designate with the mean value = 0.575' and with Sx = 0.159' the standard deviation of his sample with 20 measurements, then >> >> +/- 1Sx (+/- one standard deviation) >> means that 68% or about 2 out of 3 other measurements of the same type are expected to be within (or about 1 out of 3 outside) the range between 0.416' and 0.734'. >> >> Generally the results provide the mean and one standard deviation as above. However, these values allow representing the result also related to other probabilities, like e.g. >> >> +/- 2Sx (+/- two standard deviation) >> means that 95% other measurements of the same type are expected to be within (or about 1 out of 20 outside) the range between 0.257' and 0.893'. >> >> +/- 3Sx (+/- three standard deviation) >> means that 99.7% other measurements of the same type are expected to be within (or 3 out of 1000 outside) the range between 0.098' and 1.052'. >> >> I hope it helps. >> >> To those of you who are familiar with the subject: Please feel free to improve or even correct these general explanations where necessary. Thank you. >> >> Marcel > > ----------------------------------------------------------------------------- > | Richard B. Langley E-mail: lang---.ca | > | Geodetic Research Laboratory Web: http://www.unb.ca/GGE/ | > | Dept. of Geodesy and Geomatics Engineering Phone: +1 506 453-5142 | > | University of New Brunswick Fax: +1 506 453-4943 | > | Fredericton, N.B., Canada E3B 5A3 | > | Fredericton? Where's that? See: http://www.fredericton.ca/ | > ----------------------------------------------------------------------------- > > > >