# NavList:

## A Community Devoted to the Preservation and Practice of Celestial Navigation and Other Methods of Traditional Wayfinding

**Re: "Vernier acuity" of horizon IC tests**

**From:**Nicol�s de Hilster

**Date:**2009 Jul 10, 10:53 +0200

In NavList 9005 engineer{at}clear.net.nz wrote: > If we write down the 95% standard errors for the means (1.96 x SD/root n)... > The variance is known to be the summed squared residuals (difference between mean and observation) divided by "the number of observations minus 1" (minus one is for small populations, can be neglected at large populations). The SD (standard deviation) of the normal distribution is the square root of the variance and represents 68.3% of the observations (so these are within plus and minus SD from the mean). In order to get 95.4% probability (again normal distribution) you simply multiply the SD by 2 which will give a wider range from the mean to fit all the observations in (there should be ample sources on the internet to verify this). A 99.7% probability is obtained by multiplying SD by 3. Just an example: 30 observations: 2,4,3,5,2,3,5,3,1,3,2,4,3,5,2,3,5,3,1,3,2,4,3,5,2,3,5,3,1,3 (yes, three times the same first 10 obs.) Average = 3.1 Variance = 1.54 SD (68.3%) = square root 1.54 = 1.24 2SD (95.4%) = 2 x 1.24 = 2.48 3SD (99.7%) = 3 x 1.24 = 3.72 According to your formula we would get (1.96 x 1.24 / sqrt(30)) = 0.44, which differs quite a bit from the 96% probability (1.24) as given above. If you re-calculate the sextant observations of you and Greg, you might find that the results with and without telescope will overlap and that there thus is no difference (statistically) between the two observations. If you and Greg send me the original observations I will be more than happy to do the math for you guys. Nicol�s --~--~---------~--~----~------------~-------~--~----~ NavList message boards: www.fer3.com/arc Or post by email to: NavList@fer3.com To unsubscribe, email NavList-unsubscribe@fer3.com -~----------~----~----~----~------~----~------~--~---